A while ago, I realized how pervasive big tech spyware was. It’s everywhere. In search engines, word processors, even the OSes themselves. Upon this realization, I decided to try to get away from that spyware, and researched how to do that. This is when I came upon self-hosting. There are certain open-source programs that allow you to do the same thing the big tech services allow you to do, but you host them yourself, so you control them. There was just a slight problem: I had no servers.
This is when I learned about “the cloud” and how you could rent servers from companies and then use those to host your software. I tried it, but realized I was just hosting my own services on big tech servers, so it wasn’t helping. I’d been wanting a raspberry pi cluster for a while at that point, but never had the motivation to get one, so I waited until my birthday, collected the money gifted to me by my relatives, and bought 4 raspberry pi 4s with just 1GB of RAM, as well as all the equipment to run them (an 8-port network switch, power cables, etc.). Since then, my cluster has grown. Little by little, I’ve collected money and bought new parts. Now, I have 8 raspberry pi 4s (four 1GB, four 2GB), a Pine H64, a RockPro64, and my old 2012 Mac Mini running Debian. I also have an old 2011 MacBook that I’ve revived and patched to run the newest macOS which I will use to compile and test apps for iOS and macOS, as well as a Radxa Rock5 Model B with 16GB RAM on the way that I will be adding as well.
Originally, whenever I wanted to run a new service, I’d just kind of find a server and stick it on there, then manually configure everything and hope I remember where it’s running. Now, I have a Nomad cluster with Consul and Traefik handling everything automatically for me. If I want to run a new service, I just make a Nomad config for it, and Nomad finds a free server with enough resources, downloads it, configures it, runs it, and then publishes it to Consul, from where Traefik automatically picks it up, sets up routing rules, acquires a TLS certificate, and exposes the service. Everything happens automatically. If a server goes down, Nomad will run the services that were running there on a different server and Traefik will reconfigure itself to match.
This is what my setup looks like:
At the top is my Mac Mini, a WiFi antenna for Home Assistant, and a RockPro64, on the shelf under that, there are 8 raspberry pis, under that is a network switch and a Pine H64 running the reverse proxy, and under that is a UPS that lasts over an hour in the event of power loss. On the right is my 3D printer, which is connected to one of the Pi 4s for OctoPrint.
This has been very useful for me. Not only does it mean I own my data AND my services, it also means outages don’t affect me and my services are always very fast and reliable. I’ve had numerous times where Github was down and most people couldn’t do any work, except me because I have my own Gitea instance, for example. I think anyone with the expertise, time, and resources to do this should do it.
Where to start? Also how much have you invested in this
I started with an 8-port network switch, 4 raspberry pi 4s, and the cables to connect everything together. I used powerline for ethernet because this is in my room and my router is in a completely different location (now I juxt have a very long ethernet cable I got for pretty cheap from Monoprice). The cost of the first setup was a couple hundred dollars.
Over time, I added more and more. I was and still am a teenager (I’m 17 right now), so I just saved money gifted to me on birthdays and new year to build this incrementally.
I also had some things lying around like the old mac mini (my first computer that my parents bought me in 2012 when I was 7 years old).
I don’t know exactly how much the whole thing costs because I built it incrementally, but it shouldn’t be too difficult to calculate based on the parts you can see in the photo.
This is quite the setup, but how to you deal with the fact that your ISP can still see which websites you’re accessing? Did you also build your own custom search engine?
The ISP can still see what websites I’m accessing, yes, but they only get IP data, so the most they can know is that I connected to some site’s IP. They do not have a clue what I actually did there because nearly all sites (including my own services) are secured via TLS encryption.
I did build a metasearch engine, though I am not using it now because it kept getting rate limited. I am now using SearXNG.
Great
