UPDATE: To everyone who suggested YUNO, thank you so much. This seems like it is about to make my journey much easier. It is basically almost exactly what I was looking for, but I was unaware that it existed.
Thank you ALL for your suggestions, actually. It’s a bit overwhelming for an almost complete noobie but I an going to look into all of the suggestions in time. I just saw that there were several mentions of YUNO so I decided to make that one of the first things I investigated.
So, about two months ago, I had a very eye opening experience. As the result of a single misconfigured security setting on my Android, I was locked out of my Google Account on my phone AND all of my PCs. I had no access whatsoever to Google, or any of the literally hundreds of services that I get through Google.
This is when I realized that I relied entirely on Google/Android because those two days were actually very difficult, being cut off from media, services, passwords, everything, from the past almost twenty years of my life, could be taken away from me in an instant. The decades of my life that were locked away in my Google Account included hundreds of thousands of pictures, almost a hundred thousand audio tracks, several hundred books, several hundred apps, thousands of videos, etc. ad infinitum. Unfortunately, very little of this material was backed up at that point. That is my fault. Also, the misconfigured security setting was my fault as well.
The amount of data, media, memories, services, etc. that would have been lost is actually endless and it would have affected my life in several ridiculously negative ways.
Luckily, in the end, I was able to get my access back and then basically immediately grabbed all of the several terabytes of information and media of mine that they had, and that I was almost locked out of. I have it all in my house now on a drive in my computer, with a backup made on another disconnected disk.
I then decided that no corporation was ever going to have such an insanely high level of influence on and control over my entire life and my media ever again. That experience was actually very scary.
I’ve been trying to get into SelfHosting, but am finding it quite daunting and difficult.
There is a LOT of stuff that I have to learn, and I am mostly unsure of where to even begin. I know basically nothing about networking.
I need to learn the very basic stuff and work my way up from there, but everything that I’ve seen on the Internet assumes that the reader already has a basic to intermediate understanding of networking and the subjects that surround it. I do not, but I am going to learn.
I just need someone to show me where to start.
Thanks in advance for any assistance!
I’d recommend starting by hosting a nextcloud instance.
- Get a desktop computer, pretty much anything will do but having room to add more HDD is important.
- Install Linux distro like Ubuntu or something
- Get a static IP so your IP doesn’t change
- Setup a router port forwarding rule so that an outside address points to your nextcloud instance.
Then do some optional steps:
- Automatically turn on PC when power comes back on (BIOS setting)
- Startup script that runs nextcloud on startup
- Install docker to manage services like nextcloud
- Add some remote desktop thingy to manage your server from your laptop (ssh is also good but a steeper learning curve)
- Get a NAS for storing data with redundancy.
- Have some other form of backup like your current Google account, cloud provider or one of your mates with a similar setup.
That’s pretty much what you need to start hosting your own files, then later on you can setup a email server, media server like Jellyfin, homepage and everything.
Just go one step at a time and when you hit an issue you can and should ask Google or ChatGPT. Remember, everything exposed to the Internet is vulnerable so take security seriously. Always have everything protected by a decently long password, pairing requirement with your server confirming adding a device or an API key.
I wanna say thank you for making this post OP. I’ve got a spare laptop that I want to try to turn into my own cloud server but I find the endeavour similarly hard as well. I’ll be looking at the tips in the comments. Good luck OP!
Good luck to you, as well friendly stranger!
This post inspired me to audit my firewall rules, had put it off for too long
if it seems daunting, which it is!, maybe it would be a good piont to ask yourself if you really need to run a server and these self hosted web apps in the first place. i did for about 10 years but i realised at some point i didnt need half of it.
if youre planning on having multiple users or want to share one of the services (like real time editing of files or passwords etc) then thats where self hosted stuff makes sense to me, but if not then syncthing can do a lot without needing any complicated setup
keepass is a good example. or note taking apps like jopin or obsidian where the data is store in plain text and where you can choose where the data is stored works great with syncthing
for about a year before i did any self hosted stuff i was running only syncthing on my laptop and phone without any server so its do-able and you can get started right now and worry about getting a server later.
tailscale is a huge help as well and is very easy to setup. say you repurpose an old laptop as a server for now and install whatever services on it, jellyfin for example, you will only be able to access that when you are on your home network but not when you are away, and thats where tailscale comes in. as long as its installed on each device you should be able to connect to your apps/services from anywhere.
basically you can start small and then over the coming months and years as you learn more you will get more confident about moving onto more complicated setups
Instead of self hosting, why not try better offerings?
Most anything you probably use Google for, you can do with a disroot account. Riseup is a great group, with many similar services (not all). The tildeverse also has myriad replacement services.
Just try to support them, financially.
I have no idea what Riseup, Tildeverse, or Disroot are but I will certainly look into it.
Amazing, thanks for sharing. I understand Riseup and Disroot and the missions of each, but I’m having a bit of trouble with Tildeverse , likely because I am not well versed in *nix operating systems as of yet. I’m going to commence to reading up on all of these!
Oh, yeah, I guess I kinda tossed that out there, as they do host a ton of servives. However, its very welcoming as an onboard ramp to learn about *nix stuff. Just ease yourself into that, while exploring Disroot and RiseUp :)
I really wish people would realize the level of dependency, and thus leverage, these companies have encouraged us to give them, before they learn it first hand.
Yeah, it was dumb. I should have thought about it long before what occured, but I didn’t. But, in the end, I definitely learned my lesson.
You’re not dumb, we are all being brainwashed into sticking our asses in the air and convinced we won’t get fucked.
As the result of a single misconfigured security setting on my Android, I was locked out of my Google Account on my phone AND all of my PCs.
Just a heads up on what you are getting yourself into, if you fuck up your self hosted setup badly enough there is no recovery.
That isn’t necessarily intended to scare you off from self hosting, just that the first and most important lesson to learn is to have a good system of backups that are backed up automatically, are easy to recover from, and are separated enough from other copies of the data that if something goes terribly wrong one copy will survive.
Thanks for the advice. Currently, I have a backup of all of my important data on a high capacity HDD that is completely disconnected from any devices. There is no real way to automate backups with that setup, but it’s what I am working with at the moment.
To piggy back off of Arcayne’s comment
I’d recommend setting your sights on the 3-2-1 rule. 3 copies of your data, 2 different mediums, 1 off-site. Hetzner Storage Box is a good cheap offsite option.
I don’t backup everything to a 3-2-1. I at least have everything on a 2-1 backup scheme. Two copies, one offline. My personal photos and videos are on a proper 3-2-1. I can redownload my movies and music. I can’t retake pictures from my childhood. It saves a lot of money doing it that way. My personal media is less than 1 TB, whereas my whole collection is many TB and would be prohibitively expensive to 3-2-1.
I’d recommend setting your sights on the 3-2-1 rule. 3 copies of your data, 2 different mediums, 1 off-site. Hetzner Storage Box is a good cheap offsite option.
Backup. I use Backblaze personal which is $179 for two years of ‘unlimited’ storage. All my important self hosted data is duped to some old 2.5" external drives connected to my work machine that then is backing up to Backblaze. I also have 1yr retention, so any deleted file is accessible for up to 1yr.
After backups are sorted, stick with the OS you know best. If Windows (I hope not), then HyperV for VMs is good. Try the official Nextcloud VM from Hanson IT. Nextcloud is a good catch-all, but it’s beaten by other specific tools. I now host all I need from specific Docker containers: photos, calendar, email backup etc etc
But I would say Docker. Docker desktop if Macos or Windows if your thing. Get to know docker and the world of self hosting is your oyster.
As what others say, keep it all to your home network and tread carefully when trying to remote access it all.
Honestly? Don’t do the whole switch, or even a big switch from a few services to another.
Start small. Very small. Try doing just one service you rely on, like your images or music. Immich just announced their first stable release. I use navidrome for my music. Make sure to test these on a copy of your data, not your actual data.
Once you’ve got one service working as you want it to do, then you can try your hand at another service. This way, you don’t get stuck trying to do everything all at once.
It may be worth considering how much (if any) you want to spend at the start, too. That’ll inform your next immediate task; setting up basic backups for your data. A spare drive is a good start, but it may be worth keeping another one at your parents house, or similar.
That is exactly my plan, to have this set as a long term goal with several incremental micro-goals, as opposed to attempting to do it all in a weekend. I figure making it a long term thing gives me much more of a chance to actually learn what it is that I am doing. Plus, at my level (no real networking knowledge to speak of) trying to do this in a weekend sounds like a nightmare, lol.
An “I Wish I Had …” addition to this: maybe start something like Wiki.js or Joplin that you will then use to document what you do as you go. Keep a record of your process so that you can work your way back if you need.
Great suggestion! I will definitely implement something like that.
You seem to imply you also want to selfhost some email service. But that’s sadly one of the few things that will always be better at a trusted third party email provider.
Besides that it seems the most important thing you want is pure data storage, and that kind of selfhosting is not hard. In many cases one would not even consider it as part of “selfhosting” as it can be as simple as a local NAS or external HDD.
So my question is what do you actually want to accomplish? Because I think for a lot of your concerns you don’t even need to go and host something.
I don’t want to replace email, per se. More so, the services that come along with it. Stuff like photo storage and sync, notes and reminder sync, calendar sync, and a lot of the hundreds of other small things that I have just been leaving up to Google that would cause a huge problem for me if I lost access.
I have the password situation handled with KeePassXC on my PCs and KeePassDX on my phone, but that’s about as far as I have come as of yet.
Keepass is a great way of password management, I use keepass as well. I also use syncthing to sync my password database across all devices and then I have the server acting as the “always on” device so I have access to all passwords at all times. Works amazing because syncthing can also be setup so when a file is modified by another device, it makes a backup of the original file and moves it to a dedicated folder (with retention settings so you can have them cleaned every so often). Life is so much easier.
For photo access you can look into immich, its a little more of an advanced setup but, I have immich looking at my photos folder in syncthing on the server, and using that location as the source. This allows me to use one directory for both photo hosting and backup/sync
Do you know anywhere that I could find a guide on how to set that up correctly? I’ve used SyncThing in the very recent past and it ended up being not good because of my own ignorance. I have syncthing on my phone and my desktop, but neither of them are syncing anything to anywhere. I’m terrified that I will mess something up and possibly lose data, or end up with a gigantic mess of files that do not belong in the synced directory. That’s what happened the only time I ever did use Syncthing. For some reason, it basically transferred a gigantic directory of things that were not relevant at all from my PC to my phone, filling up all of the storage and it was a hassle to get rid of everything that was not supposed to be there. I REALLY want the KeePass databases on my phone and on my main PC to communicate and update each other though. Entering everything in twice has been frustrating.
I haven’t used a guide aside from the official getting started with syncthing page.
It should be similar to these steps though, I’ll use your desktop as the origin device.
- install syncthing on all devices you want to be syncing with
- on your desktop syncthing page, click “add remote device” and add the device ID of your phone(found on your phones syncthing app), you can also add any other device you want to have communications with (you will need to approve this action on the phone as well so be on the lookout for a notification)
- make a backup of your current keepass file just in case these steps shouldn’t cause files to change but, since the end goal is syncing two devices that you have mentioned have differences with files with the same name better safe than sorry
- create a keepass share on one of the devices (the folder path of this file should be wherever your keepass file is stored on your device. If this file is in a folder with a bunch of other files, you may want to move the file to it’s own subfolder or you will end up sharing all of the files in that path)
- under file versioning chose what type of file version control you want. I prefer staggered since it when a remote device changes the file it moves the old file to a folder, and then deletes them according to the settings
- at this point you should double check the name of your mobile devices keepass file name, if its the same as the name of the db on the desktop, you should rename it prior to continuing. Keepass should be able to detect a file conflict and rename it on it’s own but, better safe than sorry.
- share the folder with the device you want to sync it(your phone in this case)
- Your phone should get a notification that a device wants to share something with it. Approve it, be careful not to clear it because it’s a pain in the butt to get that notification back if you accidentally deny or swipe it away, the mobile app isn’t /amazing/ with it’s UI (but it has gotten better)
- once approved configure it to where you wanted the file to be on your mobile device.
- You should be done at this point. Syncthing should be automatically syncing the keepass files between the two
Some things you may want to keep into consideration. Syncthing only operates when there are two devices or more that are online. I would recommend if you are getting into self hosting a server, having the server be the middle man. If you end up going that route these steps stay more or less the same, it’s just instead of sharing with the phone, its sharing with the server, and then moving to the server syncthing page and sharing with the mobile. This makes it so both devices use the server instead of trying to connect to each other. Additionally, if you do go that route, I recommend setting your remote devices on the server’s syncthing instance to “auto approve” this makes it so when you share a folder to the server from one of your devices, it automatically approves and makes a share using the name of the folder shared in the syncthing’s data directory. (ex. if your folder was named documents and you shared it to the server, it would create a share named “documents” in where-ever you have it configured to store data). You would still need to login to the server instance in the case of sharing said files to /another/ device, but if your intent was to only create a backup of a folder to the server, then it removes a step.
Another benefit that using the server middleman approach is that if you ever have to change a device later on down the road, you are only having to add 1 remote device to the server instance, instead of having to add your new device onto every syncthing that needs access to that device.
Additionally, if you already have the built in structure but it isn’t seeming like it is working, some standard troubleshooting steps I’ve found helpful:
- if trying to share between devices, make sure that there is at least two devices that are connected as remote devices active in order to sync
- If above is true, make sure the folder ID’s are the same between both devices. that is how syncthing detects folders that should be sync’d
- If also true, make sure the devices are being seen as online in remote devices. If it isn’t showing as online, the connection is being blocked somewhere, verify you don’t have a firewall or router blocking it somewhere.
That is so awesome. Thank you so very much for taking the time out to inform me on this matter.
Note taking? Joplin. It’s been amazing for me and replaced Evernote after they started with their BS.
Photo storage. Been using Photo prism but I’m open to a better one.
As someone who went through this process after trumps 2nd term and power i can give you my process:
-angrily unsubscribe all big tech subscriptions -make a protonmail and tutamail account, realize I like proton suote more and decide to subscribe -transfer all passwords to proton suite -download all photos and other data to an external drive. TURNS OUT THIS TAKES SEVERAL DAYS WTF -angrily order a rasp-pi and an external SSD -use step by step tutorials to install docker and immich. Fall in love -gradually (via help of google and GPTs) become confident enough with command line to start managing the server headless over SSH
Fast forward 6 months: My router os now running OpenWRT, my network access is always through ProtonVPN. My external devices are connected via wireguard to the router when not on home wifi. My main server is now an old office mini pc running about 10 services. Im using borg for nightly snapshots(its a bit like apple time machine) and after that everything is backed up to another server at a friends house via rsync and ssh. I have a third mini computer whose purpose is to be my tv’s UI with access to services like the national broadcasts web ui and muäy own jellyfin and invidious (adless youtube client) The tv does not have an internet connection anymore.
how well does invidious work on the TV? how easy are the controls?
I use a small wireless logitech keyboard-mousepad so it works very well. I had to make exceptions in the router for googles video severs to bypass the vpn though.
yeah not all invidiouses allow proxying the video, and at some it used to be off by default
Re-investing in a new platform full of tools (Proton suite) isn’t in my opinion a rational answer. My answer is self host vaultwarden, self host your file storage, and choose between Proton and Tuta for mail, and use your own domain name so you can take your email address with you should you move.
In my opinion No-one should ever store any form of personal data implicitly on someone else’s computer.
Oh i have vaultwarden as well nowadays
When you’re connecting to your local network, say from outside your home, you connect through proton vpn? I’m wanting to self host, but I thought I would have to switch between proton and whatever VPN the local machine is using? Say immich for instance
My phone is on a wireguard tunnel into my router which puts my wireguard vpn in the same forewall zone as my home LAN. Internet access is routed through the tunnel and then through another tunnel to protonvpn and from there to the www. It was a bit elaborate to set up but it works. Wouldnt really recommend the setup for everyone, it was a bit of a pain in the ass to get working. I used Openwrt and policy based routimg plus wireguard for the tunnels into and put of the router.
Yikes. Before you dip into any of the self-hosting, take and get a WD Gold drive - from Western Digital directly (wd.com) - do NOT go through Amazon or NewEgg or any third party merchant. Send in the warranty that goes with it and register the drive (this is for covering the off chance it’s a DOA unit) Then get a good quality enclosure to pop the drive into and take your time and back up EVERYTHING onto that new HD.
Don’t use an SSD.
You want a spinning platter drive, as this is backup only, so once it’s full with all of your content, it gets dated and labeled and popped into a drawer for safe keeping. If you have countless terabytes of data, get more drives and swap them into the enclosure, date and incrementally fill. A fine tip sharpie to note what’s on the drive is fine, or if you’re obsessively anal about it, make a spreadsheet with that info… If your drives are kept dry and stored with care they will last for DECADES…
The truth if being honest here - I’m a data hoarder and most of the stuff I’ve tucked away since I first came online (in 1999) is now on drives that I maybe spin up once a year. I used to have the notion that it was critical that all my shit was accessible all the time and I ended up dropping money on networked storage… and over time, realized that as long as I knew where the files were, DID have the most important stuff - family photos and scans - tucked away not only in long term storage, but on multiple drives in multiple machines, (home, work, laptop) it was okay not have it served up instantly.
Just reading your post made me go cold inside - I can only imagine what you were going through until it got sorted. From a bonafide old school data hoarder… Please, back your shit up locally. Use enterprise drives.
Then sort a self-hosting soultion.
Which software do you use for backups?
I do my backups manually.
As I have run unsuported Mac installs for the last 20 years, I started a long time ago, automatically partitioning my OS drives and making storage volumes to work off of.
The storage volume in the computer will have subfolders for the type of data - music, video, photos, etc.
When my storage volumes fill, I will pull my latest backup drive out of storage, hook it up then go into each storage subfolder, sort by date and add everything that’s newer than what’s in the backup drive. (which is actually how Apple’s Time Machine backups work - incrementally sorted by date - but I’ve had this method since the start, so I just stuck with it)
I just make sure to take note of how many files/folders I’m adding to the backup drive and note what it has at the start, then at the end, as a double-check of it all, before I clear the storage drive on the computer. (I did not do this and lost almost a years worth of music rips, waay back in 2003. Rebuilt the music I lost then iTunes threw a wobbler and lost the library for me. FML…)
The longest backup will ALWAYS be the initial one if you’re dealing with a first time backup. The rest, once you work out how to organize your files, is academic.
What I’ve found is that your tastes will change, you grab content you think you’ll want to hold onto forever… and then years later, you realize it’s low-bitrate, low-resolution, too pixellated… whatever… and you decide to delete it.
With the software doing the backups for you - it’s too easy to just let it rip and go have dinner while it works and you end up with files that you’d otherwise get rid of. Part of being a data hoarder is not keeping everything forever. There’s a ton of garbage online. Tastes change as you get older… You want to curate that shit so you can keep what’s most important - like family stuff.
And really good porn.
Several detailed, easy to understand and very good pieces of advice! Thank you! I have definitely saved your comment for referencing throughout this process!
Install Virtualbox (or some software to handle virtual machines).
Install Debian (or some other OS of your choice, I won’t judge if you prefer Windows).
Update your OS (apt update && apt upgrade -yon Debian).
Take a snapshot of your VM’s current state after updating. Saves a lot of time if you mess up or want a clean slate.Now you decide on what you want. Do you want to install n8n or Node-RED for automation? Do you want to use Immich for pictures? Paperless to save papers in a digital format? Audiobookshelf to listen on your books or podcasts? Jellyfin to stream your media? Set up a Minecraft or Factorio server?
Once you have decided on what you want to do, try to do it in your virtual machine.
Once you understand how to set it up and configure it to your liking, decide on how you want to host it. I took an office computer, added a few HDDs and replaced the case with a bigger one and it’s now my home server, but any old laptop will do. Just make sure to take backups.I used to have a Dell R710 and a virtual machine for each service I hosted, but I have moved to docker because it as simple as taking the often provided compose file, tweaking it a bit (where to store data etc) and running it with
docker compose up -d.It’s a great idea to run it in a VM first, because I’m bound to make mistakes along the way. Awesome advice! I’m definitely going to be referencing your comment throughout the process! Saved!
I had a Raspberry Pi once and wanted to move file from the current folder to some other folder. I typed
mv /* /path/to/folder/and move everything in the root directory and down to this other folder.EDIT: Meant to say that snapshots are cheap backups. I ended up reinstalled the OS.
Correct would have been dot slash:
mv ./* /path/to/folder/
I will probably get flogged by this answer but here it goes:
I’d throw you right into the deep end: get a spare machine (an old laptop or PC) and install proxmox on it. Play around, breaks shit, delete the container/VM and start over.
Grab stuff from the Community Helper Scripts and see new stuff, try alternatives, see what works for you and don’t be afraid of breaking stuff.
It takes a bit longer and some basic concepts might fly over your head, but the stuff you learn like this, you learn by heart.
It’s been a few years since I started tinkering with a laptop with a busted video output circuit. Now I serve NextCloud and Immich to my family, keep receipts and documents neatly organised on Paperless, have a decent arr stack and a bunch of extra goodies. All from “a PC without video? Might as well make a server” now with a proper machine with several drives on ZFS pools, health checks and redundancy.
Its a helluva rabbit hole.
Isn’t that how everyone does it? 😬
I was unaware that those Community Helper Scripts existed! They should definitely be helpful at some point down the road!
Good luck!
At first, you have to decide what do you need. You can selfhost almost everything, but in my opinion there is no need to do so.
Second thing is hardware to host it. I saw a few comments recommending NAS. It is of course good thing, but my suggestion is just building your own NAS. You need only decent computer to do it.
The easiest way is just installing TrueNAS on it - with that you can setup file sharing and your apps via docker.
But what apps would you need/want? I can recommend a few from my stack:
- vaultwarden - for storing passwords, 2FA codes
- immich - for storing photos, videos, autoupload from phone
- adguard - for getting rid of ads, tracking They are really easy to deploy.
As an alternative to file shares via SMB, nextcloud is really good option. It’s google drive on steroids. Also includes photo gallery with great app on android/ios with autoupload option.
Nice! Yes, photo storage and backup as well as note sync, reminder sync, calendar, etc. are all very important micro services to me.
I think Nextcloud is a really good option for you. It includes everything you mentioned.
