Keeping services running is fairly trivial. Getting to parity with the operationalization you get from a cloud platform takes more ongoing work.
I have a homelab that supports a number of services for my family. I have offsite backups (rsync.net for most data, a server sitting at our cottage for our media library), alerting, and some redundancy for hardware failures.
Right now, I have a few things I need to fix:
- one of the nodes didn't boot back up after a power outage last fall; need to hook up a KVM to troubleshoot
- cottage internet has been down since a power outage, so those backups are behind (I'm assuming it's something stupid, like I forgot to change the BIOS to power on automatically on the new router I just put in)
- various services occasionally throw alerts at me
I have a much more complex setup than necessary (k8s in a homelab is overkill), but even the simplest system still needs backups if you care at all about your data. To be fair, cloud services aren't immune to this, either (the failure mode is more likely to be something like your account getting compromised, rather than a hardware failure).
You're spending that much time on it because you're doing too much. Your use of the term "homelab" is telling. I have:
* A rented VPS that's been running for ~15 years without any major issues, only a couple hours a month of maintenance.
* A small NUC-like device connected to the TV for media. Requires near-zero maintenance.
* A self-built 5-drive NAS based around a Raspberry Pi CM4 with a carrier board built for NAS/networking uses. Requires near-zero maintenance.
* A Raspberry Pi running some home automation stuff. This one requires a little more effort because the hardware it talks to is flaky, as is some of the software, so maybe 2-3 hours a month.
The basics (internet access itself) are just a commodity cable modem, a commodity router running a manufacturer-maintained OpenWRT derivative, a pair of consumer-grade APs reflashed with OpenWRT, and a few consumer-grade switches. There's no reason for me to roll my own here, and I don't want to be on the hook for it when it breaks. And if any of the stuff in the bulleted list breaks, it can sit for days or weeks if I don't feel like touching it, because it's not essential.
And yes, I've hard hardware failures and botched software upgrades. They take time to resolve. But it's not a big burden, and I don't spent much time on this stuff.
> I have a much more complex setup than necessary
Yup.
> Getting to parity with the operationalization you get from a cloud platform takes more ongoing work.
You don't need this. Trying to get even remotely there will eat up your time, and that time is better spent doing something else. Unless you enjoy doing that, which is fine, but say that, and don't try to claim that self-hosting necessarily takes up a lot of time.
It's definitely mostly a hobby, but I also want to get something close to the dependability of a cloud offering.
I started small, with just a Raspberry Pi running Home Assistant, then Proxmox on an old laptop... growing to what I have now. Each iteration has added complexity, but it's also added capability and resiliency.
I love self-hosting and run tons of services that I use daily. The thought of random hardware failures scares me, though. Troubleshooting hardware failure is hard and time consuming. Having spare minipcs is expensive. My NAS server failing would have the biggest impact, however.
Other than the firewall (itself a minipc), I only have one server where a failure would cause issues: it's connected to the HDDs I use for high-capacity storage, and has a GPU that Jellyfin uses for transcoding. That would only cause Jellyfin to stop working—the other services that have lower storage needs would continue working, since their storage is replicated across multiple nodes using Longhorn.
Kubernetes adds a lot of complexity initially, but it does make it easier to add fault tolerance for hardware failures, especially in conjunction with a replicating filesystem provider like Longhorn. I only knew that I had a failed node because some services didn't come back up until I drained and cordoned the node from the cluster (looks like there are various projects to automate this—I should look into those).
An iron ring does not technically make you an engineer in Canada. It just says you graduated from an engineering program. A P.Eng, which is a professional engineer's license is something you acquire after multiple years of experience and testing.
We've become really close friends with a couple of families on our block. It definitely helps that we all have kids the same age, but a big reason for this is that our kids entertain each other, freeing the adults (most of the time) to interact with each other.
I also sing in a small choir that is as much a friend group as it is a music ensemble. Most of us have kids, but not everyone; while I can't speak for those in the group without kids, I think one reason it works so well is because none of us have parenting responsibilities while we're together as a choir (we joke that rehearsals are a great time for us to escape from our families).
> I'd bet a lot of money that however the system is implemented the police and fire get special treatment when it comes to process (i.e. asking permission before they go somewhere planes might be) and that is part of what lead to this.
I highly doubt that any system would intentionally give ground vehicles of any kind special treatment on an active runway.
It's also a crowded and super mature space space between JFrog (Artifactory) and Sonatype (Nexus). They already support private PyPI repositories and are super locked in at pretty much every enterprise-level company out there.
A commodity yes, but could be wrapped in to work very nicely with the latest and greatest in python tooling. Remember, the only 2 ways to make money are by bundling and unbundling. This seems like a pretty easy bundling story.
Yeah you'd think so but somehow JFrog (makers of Artifactory) made half a billion dollars last year. I don't really understand that. Conda also makes an implausible amount of money.
Most of the companies that spend $$$$ with them can't use public registries for production/production-adjacent workloads due to regulations and, secondarily a desire to mitigate supply chain risk.
Artifactory is a drop-in replacement for every kind of repository they'll need to work with, and it has a nice UI. They also support "pass-through" repositories that mirror the public repositories with the customization options these customers like to have. It also has image/artifact scanning, which cybersecurity teams love to use in their remediation reporting.
It's also relatively easy to spin up and scale. I don't work there, but I had to use Artifactory for a demo I built, and getting it up and running took very little time, even without AI assistance.
Yeah I mean I understand the demand. My previous company used Artifactory. I just don't understand why nobody has made a free option. It's so simple it seems like it would be a no brainer open source project.
Like, nobody really pays for web servers - there are too many good free options. They're far more complex than Artifactory.
I guess it's just that it's a product that only really appeals to private companies?
Both Artifactory and Sonatype have somewhat restricted open-source options, which is part of their "get a foot in the door" product-driven sales strategy.
There are no competing open-source projects because such projects would need to provide more value than Artifactory/Sonatype OSS, which are both already huge projects, just to be considered.
JFrog has a free version. It's called the JFrog Container Registry. Lots of features are missing and you can't use the Artifactory API that it ships with, but it's there.
There are also several free registries out there: Quay, Harbor, and Docker's own distribution. They all have paid versions, of course.
From my understanding there are a lot of companies that need their own package repositories, for a variety of reasons. I listened to a couple podcasts where Charlie Marsh outlined their plans for pyx, and why they felt their entry into that market would be profitable. My guess is that OpenAI just dangled way more money in their faces than what they were likely to get from pyx.
Having a private package index gives you a central place where all employees can install from, without having to screen what each person is installing. Also, if I remember right, there are some large AI and ML focused packages that benefit from an index that's tuned to your specific hardware and workflows.
Private artifact repositories also help to mitigate supply chain risk since you can host all of your screened packages and don't have to worry about something getting removed from mvn-central, PyPI, NPM, etc.
Plus the obvious need for a place to host proprietary internal libraries.
> a lot of companies that need their own package repositories
Every company needs its own package repository. You need to be able to control what is running on your environment. Supply-chain risk is very, very real and affects anybody selling software for a living.
This is besides the point that in the real world, not every risk is addressed, at least in part because available resources are diverted to address larger risks.
How many people use that simple pip repo daily? If the number is not in the high hundreds, or a few thousands; maybe nothing. But once you get up there, any kind of better coordination layer is useful enough to pay money to a third party for, unless maintaining a layer over pip is your core competency.
That starts to get into a very philosophical space talking about human action as deterministic or not. I think keeping to the fact that the artifacts (ie code) we are working off will have deterministic effects (unless we want it not to) is exactly the point. That is what lets chaotic human brains communicate with machines at all. Adding more chaos to the system doesn't strike me as obviously an improvement.
It is not at all clear that the mapping is for purposes other than the AR features in the game itself though. In fact Niantic advertised the scanning field research as helping them make richer experience at PokeStops (which they did).
Niantic was much more upfront about this with Ingress, so people who know the company's history will likely guess that Pokemon Go is serving the same purpose, but for someone coming into the game without that background, there is nothing in the game itself that indicates that data is being collected for other commercial purpose.
Right, but it sounds like the data collection itself was pretty well communicated. So nobody should be surprised it gets used for some other (legal) purpose than was originally intended.
I have a homelab that supports a number of services for my family. I have offsite backups (rsync.net for most data, a server sitting at our cottage for our media library), alerting, and some redundancy for hardware failures.
Right now, I have a few things I need to fix: - one of the nodes didn't boot back up after a power outage last fall; need to hook up a KVM to troubleshoot - cottage internet has been down since a power outage, so those backups are behind (I'm assuming it's something stupid, like I forgot to change the BIOS to power on automatically on the new router I just put in) - various services occasionally throw alerts at me
I have a much more complex setup than necessary (k8s in a homelab is overkill), but even the simplest system still needs backups if you care at all about your data. To be fair, cloud services aren't immune to this, either (the failure mode is more likely to be something like your account getting compromised, rather than a hardware failure).
reply