Yeah sure, my process to learn was literally I went down to a local computer store, I bought a cheap desktop computer and 6 cheap hard drives, an HBA, a RAID controller, a bunch of cables, went back to the office and installed Ubuntu Server onto it and practiced several "skills", like how to setup automated backups using rsync, how to physically install these components, how to install mdadm for software RAID etc... comparing software RAID to using a RAID controller. I setup several drills for each task involving various types of failures and how to manage them to the point that it was part of muscle memory.
Some drills I practiced were setting up RAID 10 and on hardware failure having an email sent to me, so I went through the process of getting RAID 10 working using 4 of the drives, and then I would physically pull a hard drive out of the system as it was running to simulate a failure, and then I swapped in a new hard drive and ensured that the RAID rebuild process took place.
Once I was confident enough, I went to thinkmate.com and bought two JBOD expansion chassis each of which supported 78 drives and filled each of them up with 5 TB Seagate drives to get a total of 390 TB. The JBODs are managed by a 1U server running Ubuntu Server and software RAID using mdadm. I also bought 8 compute servers that were considered high density and could fit in 4U. I got a Cisco router for Internet and I networked everything within the data center together using a used Infiniband switch that I bought off of ebay. I also got a KVM so I could remotely access all of the systems. If there's one thing that I would change today, it would be to use ZFS.
I remember comparing the cost of doing this DIY setup to some other premade solutions like EMC and the cost was astronomical, like 3-4x what I ended up having to pay. I also even remember watching Linus Tech Tips and Level1Techs on Youtube and they both had good content about how they managed their storage that was fairly reasonable but still slightly on the pricey side, nevertheless I learned quite a bit from it.
At any rate, the bottom line is that it's not trivial to learn all this stuff by any means and I remember having some serious frustrations due to just how bad and demoralizing some error messages can be, but it's not thaaaat difficult either and in my situation my company is self funded, no venture capital or outside investors of any kind so every dollar my company spends is a dollar out of my own pocket. You better believe when I'm starting my business I'm not going to just blow 100s of thousands of dollars extra unless I absolutely have to.
About 5 years ago I managed to use my own storage system to ditch Dropbox in favor of Nextcloud, which is just leaps and bounds superior. I remember I got so frustrated with Dropbox because I wanted to just do something as simple as create a sub-directory and grant only read only permissions to some accounts. I also remember wanting to do some simple things like create a fake account with very limited permissions that could be used in our conference room for presentations or demos but the only way to do that with Dropbox would be to create an actual account and have to pay the full price for it.
Nextcloud works amazingly well, has all kinds of cool plugins, and gives our organization a lot of flexibility that Dropbox doesn't and I can make as many accounts as I feel like for whatever reason I feel like.
That's a lot more involved. We're a quant firm that have servers colocated globally at various exchanges, so our data is distributed across a lot of systems. The main and original purpose of this storage system is to provide one centralized repository of all of this global data so that we can run backtests and perform analytics locally as opposed to having to hit our Australian system when we need ASX data, and then hit our German system when we need Xetra data, etc etc...
So different backups happen in varying directions, with the colocated servers using the main storage system to back themselves up, and the main storage system in turn backing portions of itself onto the colocated servers.
Not everything gets backed up, however, and most of what does get backed up can be compressed quite significantly. We mostly just back things up that can't be recovered from third parties or that are proprietary to our organization.
