A real-world production migration from DigitalOcean to Hetzner dedicated, handling 248 GB of MySQL data across 30 databases, 34 Nginx sites, GitLab EE, Neo4j, and live mobile app traffic — with zero downtime.
0? My energy company says I’m using power equivalent to a family of eight. And it’s just wifey , the servers and me. I had cops here asking if I grow weed 😁
So unless you steal power, it surely isn’t close to 0 😁
A trick I realized a few years ago: Caddy has a module you can build it with that does WOL. So I was able to run a Caddy reverse proxy that woke up my higher powered server on demand, and let it go back to sleep when I wasn’t using it. Might be a bad idea for a database sever, but for my uses it was pretty simple and effective.
Is it that your service/website is on both (low powered server and high powered one) or is it only on the high powered? So, it’s like
the lower powered server knows it needs help (sounds a bit surreal to me, but perhaps it’s doable)
or the lower powered server does not serve anything, but wakes up the high powered when the thing is accessed?
I guess that’s the 2nd thing, but it’s very cool indeed! That way you can really have very convenient things for free, as it’s super cheap to run any hardware for a very while on demand. I don’t mind waiting a minute or even two when I need to access something very infrequently and don’t want to run my server 24/7. I do exactly that, but I wake up it via LAN manually.
The low powered server is the Caddy server, all it does us act as a reverse proxy for everything in my house, giving it an SSL cert and doing things like WOL. The caddy config basically just says “Here’s your reverse proxy target, if you don’t get a response within one second, send a WOL packet, wait a couple of seconds, then try again”.
The only requirement is for you to do a custom build of caddy (this is done with a dockerfile), and to have WOL enabled on the high power server.
It means the first web request for services on the high power server might take a few seconds, but everything after that is smooth.
Is it that high power server takes a few seconds to boot? What’s the hardware you have there? I’m curious that’s the average boot time for an average high power server? I do use heavily obsolete devices for my personal servers (think of DDR-2 era devices with Intel Atom or sometimes core 2 duo devices) usually without even SSDs. With an SSD, my desktop devices (all DDR-3 era with SATA-3 disks) boot within 20…30 seconds, which is good enough for me. I assume the more modern devices would be quicker, but [single-digit, I assume] seconds sounds very good. To me, that sounds like it’s a no-brainer to have this feature. I was thinking whether I can wait minutes for something I need occasionally to boot. Seconds is just too fast. I think that delay is tolerable even for a commercial / production server, where the expectations are just different.
I’m not shutting it down, just sleeping it. My high powered server is a gaming PC from a few years ago. Running Linux, my best case scenerio for cold boot would still be 10-20 seconds, but wake from sleep is near instant.
Sleeping is something I haven’t think of, but it’s a really great way of saving power! Thanks for your input, I’d definitely replicate your approach! (When it comes to scale a bit.)
More likely your system is more sophisticated, I have just joined the hobby, so to say. But I am sure you can go much cheaper than that with bare metal. If I’d really need to host something, I’d rather buy a real server, and invest in solar power instead of paying some rent. Was a happy Digital Ocean customer, before I realised I can do the same with a Raspberry Pi. I was buying a couple of Pis a year for them. Right now, de-facto one Pi can host everything I really need. I regret I wasted about half a thousand on nothing. Could have bought a great NUC instead of wasting money on the cheapest VM for years.
Yeah solar power is much more affordable these days. I live in a vehicle so I have a 500w panel on the roof charging a 200ah lithium battery. I only use a laptop and steam deck, but could easily upscale. The whole system, including the victron controllers & shunt and the 2k inverter came out around £700, but I’m pretty sure stuff has only got cheaper since I bought it. I have way more power than I need in summer, though there are maybe two months in winter when I have to charge everything in daylight. I could always add a small wind generator if I needed. Renewables are totally feasible these days
Oh no, I host everything myself, I would never use a cloud-based anything. A pi does a lot, but it wouldn’t do for all my stuff. Jellyfin, AgentDVR, PaperlessNGX and especially LLamaCPP do need a lot of ressources :)
My solar covers more than my entire electric bills in the mild weather months.
I was thinking of moving to more modern systems with modest power consumption too. One of the systems I picked up is basically a case-as-a-heatsink with no fans.
My electric bill last month was $15. It eventually is financially worth it.
Don’t know how hard you went on your home lab, I use office rather than datacenter equipment and it’s quite and plenty for my needs. For my professional test/dev needs I have such equipment at work, so I don’t need to home train on gear for the sake of competence in the field.
15 bucks is impressive, yes…i just currently don’t care very much for the bills. Dunno even if solar would be great here, we’re not on the overly sunny side 😁
I don’t work (and never really did), so home is all there is. Office surely is enough most of the time, especially considering I wouldn’t want to pay license fees or be online with stuff. Professional stuff can be pretty shitty in that regard. So I use whatever fits my criteria best.
It grew more and more to be honest, and it already comes close to be a part-time job that costs money and has no salary 😑
This is why im doing my homelab on low powered processors (5825u NAS boards). Runs way cooler and is way more efficient. Same performance as my 9900kf gaming PC cpu wise.
I was on very low-powered too. As I’m a damn cheapskate. But then the performance started to piss me off and i went nuts. Plus a fat gpu for LLAMACPP for the smarthome. Also I’m gaming A LOT. So it adds up :)
That’s fair. I have a 6800xt in my pc and I use that for my llm.
That said, I think I’m slightly misleading you. My current setup is not a 5825u. It’s my old laptop with an i7-8550u. I’m going to move it to the 5825u soon.
I think I’m at a family of four or five, but I’m alone with my dogs and my weed and my servers. Being able to legally self-host your own drug supply is great.
We’re (krauts) allowed to do that too now. But a friend does, so I don’t need to. I would suck at it anyway. Last time i did that was 30yrs ago. Planted A SHITTON, and when i reaped it, I had like 5 fat ass ikea-bags full of tasty weed. Had them dry in a room I never use and…forgot them. All went bad. I should stick to non-living things like servers :-)
How does running a server, assuming it’s used some amount of internet bandwidth, handle residential internet speeds? If I’ve got a gig up and down, can I reasonably run like a jellyfin for my friends?
I only use jelly for wifey+me, so it’s overseeable. A gig UP surely does help. depending on your source-material, how many watch CONCURRENTLY and how much needs to be transcoded. So hard to say, but 3-5 people should be good with that.
My isp hasn’t complained. I have fiber at my house and symmetrical gigabit. You should be fine if you transcode to reduce bandwidth needed as you still use your own internet. But I’m not running jellyfin on my home server.
Most servers you rent are only going to have 1Gbps internet speeds too unless you’re paying extra, so if you’ve got symmetrical gigabit at home, you’re 100% good to go, except for maybe higher downtime than a datacenter. My fiber at home seems to go out for a bit overnight occasionally as they’re doing maintenance.
0? My energy company says I’m using power equivalent to a family of eight. And it’s just wifey , the servers and me. I had cops here asking if I grow weed 😁
So unless you steal power, it surely isn’t close to 0 😁
So you, wife and your 6 digital kids, checks out.
If you put it that way…kinda 😬
I realised I don’t need my servers being online 24/7, so for me that’s Raspberry Pi and equivalents, plus powering on computers on demand.
A trick I realized a few years ago: Caddy has a module you can build it with that does WOL. So I was able to run a Caddy reverse proxy that woke up my higher powered server on demand, and let it go back to sleep when I wasn’t using it. Might be a bad idea for a database sever, but for my uses it was pretty simple and effective.
Get a load of this guy not using his high powered server 24/7/365.
Oh wow, that’s really cool! I do use Caddy too.
Is it that your service/website is on both (low powered server and high powered one) or is it only on the high powered? So, it’s like
I guess that’s the 2nd thing, but it’s very cool indeed! That way you can really have very convenient things for free, as it’s super cheap to run any hardware for a very while on demand. I don’t mind waiting a minute or even two when I need to access something very infrequently and don’t want to run my server 24/7. I do exactly that, but I wake up it via LAN manually.
The low powered server is the Caddy server, all it does us act as a reverse proxy for everything in my house, giving it an SSL cert and doing things like WOL. The caddy config basically just says “Here’s your reverse proxy target, if you don’t get a response within one second, send a WOL packet, wait a couple of seconds, then try again”.
The only requirement is for you to do a custom build of caddy (this is done with a dockerfile), and to have WOL enabled on the high power server.
It means the first web request for services on the high power server might take a few seconds, but everything after that is smooth.
Is it that high power server takes a few seconds to boot? What’s the hardware you have there? I’m curious that’s the average boot time for an average high power server? I do use heavily obsolete devices for my personal servers (think of DDR-2 era devices with Intel Atom or sometimes core 2 duo devices) usually without even SSDs. With an SSD, my desktop devices (all DDR-3 era with SATA-3 disks) boot within 20…30 seconds, which is good enough for me. I assume the more modern devices would be quicker, but [single-digit, I assume] seconds sounds very good. To me, that sounds like it’s a no-brainer to have this feature. I was thinking whether I can wait minutes for something I need occasionally to boot. Seconds is just too fast. I think that delay is tolerable even for a commercial / production server, where the expectations are just different.
I’m not shutting it down, just sleeping it. My high powered server is a gaming PC from a few years ago. Running Linux, my best case scenerio for cold boot would still be 10-20 seconds, but wake from sleep is near instant.
Sleeping is something I haven’t think of, but it’s a really great way of saving power! Thanks for your input, I’d definitely replicate your approach! (When it comes to scale a bit.)
Oh. Okay. That comes close to 0. Mine runs 24/7, just because it would take too long to power down and up all machines, VMS, switches etc 😁
More likely your system is more sophisticated, I have just joined the hobby, so to say. But I am sure you can go much cheaper than that with bare metal. If I’d really need to host something, I’d rather buy a real server, and invest in solar power instead of paying some rent. Was a happy Digital Ocean customer, before I realised I can do the same with a Raspberry Pi. I was buying a couple of Pis a year for them. Right now, de-facto one Pi can host everything I really need. I regret I wasted about half a thousand on nothing. Could have bought a great NUC instead of wasting money on the cheapest VM for years.
Yeah solar power is much more affordable these days. I live in a vehicle so I have a 500w panel on the roof charging a 200ah lithium battery. I only use a laptop and steam deck, but could easily upscale. The whole system, including the victron controllers & shunt and the 2k inverter came out around £700, but I’m pretty sure stuff has only got cheaper since I bought it. I have way more power than I need in summer, though there are maybe two months in winter when I have to charge everything in daylight. I could always add a small wind generator if I needed. Renewables are totally feasible these days
Oh no, I host everything myself, I would never use a cloud-based anything. A pi does a lot, but it wouldn’t do for all my stuff. Jellyfin, AgentDVR, PaperlessNGX and especially LLamaCPP do need a lot of ressources :)
My solar covers more than my entire electric bills in the mild weather months.
I was thinking of moving to more modern systems with modest power consumption too. One of the systems I picked up is basically a case-as-a-heatsink with no fans.
Never bothered with solar. Tbh the only benefit I see there is integrating it into the smart-home 😁
No fans would be great, but it’s not just the PC that makes noise. It’s the switches and NASs and UPSs…
My electric bill last month was $15. It eventually is financially worth it.
Don’t know how hard you went on your home lab, I use office rather than datacenter equipment and it’s quite and plenty for my needs. For my professional test/dev needs I have such equipment at work, so I don’t need to home train on gear for the sake of competence in the field.
15 bucks is impressive, yes…i just currently don’t care very much for the bills. Dunno even if solar would be great here, we’re not on the overly sunny side 😁
I don’t work (and never really did), so home is all there is. Office surely is enough most of the time, especially considering I wouldn’t want to pay license fees or be online with stuff. Professional stuff can be pretty shitty in that regard. So I use whatever fits my criteria best.
It grew more and more to be honest, and it already comes close to be a part-time job that costs money and has no salary 😑
This is why im doing my homelab on low powered processors (5825u NAS boards). Runs way cooler and is way more efficient. Same performance as my 9900kf gaming PC cpu wise.
Edit: this hasn’t happened yet
I was on very low-powered too. As I’m a damn cheapskate. But then the performance started to piss me off and i went nuts. Plus a fat gpu for LLAMACPP for the smarthome. Also I’m gaming A LOT. So it adds up :)
That’s fair. I have a 6800xt in my pc and I use that for my llm.
That said, I think I’m slightly misleading you. My current setup is not a 5825u. It’s my old laptop with an i7-8550u. I’m going to move it to the 5825u soon.
I think I’m at a family of four or five, but I’m alone with my dogs and my weed and my servers. Being able to legally self-host your own drug supply is great.
We’re (krauts) allowed to do that too now. But a friend does, so I don’t need to. I would suck at it anyway. Last time i did that was 30yrs ago. Planted A SHITTON, and when i reaped it, I had like 5 fat ass ikea-bags full of tasty weed. Had them dry in a room I never use and…forgot them. All went bad. I should stick to non-living things like servers :-)
ich habe meins nicht vergessen
Mine blew my socks off, but i smoke so little that I have to grow once a year max.
I probably vape even less. But also rarely, so I’m not a measure for anything 😁
I do it mainly because I love to.
Who doesn’t 😁
How does running a server, assuming it’s used some amount of internet bandwidth, handle residential internet speeds? If I’ve got a gig up and down, can I reasonably run like a jellyfin for my friends?
Easily
I only use jelly for wifey+me, so it’s overseeable. A gig UP surely does help. depending on your source-material, how many watch CONCURRENTLY and how much needs to be transcoded. So hard to say, but 3-5 people should be good with that.
My isp hasn’t complained. I have fiber at my house and symmetrical gigabit. You should be fine if you transcode to reduce bandwidth needed as you still use your own internet. But I’m not running jellyfin on my home server.
I was running it on a couple hundred Mbps up for a while, and gig up is fine
Most servers you rent are only going to have 1Gbps internet speeds too unless you’re paying extra, so if you’ve got symmetrical gigabit at home, you’re 100% good to go, except for maybe higher downtime than a datacenter. My fiber at home seems to go out for a bit overnight occasionally as they’re doing maintenance.