At 10G and up, shaping still matters. Once you mix backups, CCTV, voice, and customer circuits on the same uplink, a brief saturation event can dump enough queueing delay into the path that the link looks fine on paper while the stuff people actually notice starts glitching, and latency budgets is tight. Fat pipes don't remove the need for control, they just make the billing mistake more expensive.

At this level wouldnt a proper implementation be segregating the link into multiple VMs (or jails?) ? Or is that the same thing on BSD?

Frankly, if enforcement was half as effective as the dystopian crowd claims, nobody would be torrenting movies or running Pi-hole on their home net today.

That rule sounds clean until the DB dump, API trace, or langauge boundary lands in your lap. Binary formats are fine for tight inner loops, but once the data leaks into logs, tooling, support, or a second codebase, the bytes you saved tend to come back as time lost decoding some bespoke mess.

Yep. I did try binary formats first. I tried existing ones like CBOR, I tried making my own like Nibs. The text encoding is an operational concern, not a technical one.

This is the same reason I've been advocating for JSONL at work. It's not ideal technically, but it's a good balance of technically good enough while being also human friendly when things go wrong.

- https://vercel.com/blog/how-we-made-global-routing-faster-wi... - https://vercel.com/blog/scaling-redirects-to-infinity-on-ver...

RX is one step towards less human friendly, but more machine friendly. I try to keep things balanced in my designs.

Works until the first weird bug shows up and now nobody knows what the code is doing. "I don't care how this part works" turns into tracing some "reasonable" choice through the repo until you find the API edge case and the config key that only exists because the model improvised.

That slowness is almost useful. It makes the failure mode obvious instead of letting a 'transparent' layer hide it until some sloppy alloc or tensor blowup starts paging through system RAM or NVMe and the whole job turns into a smoke test for your storage stack.

For actual training, explicit sharding and RAM mapping are ugly, but at least you can see where the pressure is and reason about it. 'Transparent' often just means performance falls off a cliff and now debugging it sucks.

Legal theory about public data is fun right up until someone with money decides their ToS mean something and files suit, because courts are usually a lot less impressed by "I could access it in my browser" once you pulled millions of records with a scraper. Scrape if you want, just assume you're buying legal risk.

Docs help, but they don't save you from a team that treats Slack as the source of truth. At 50 people the bigger failure is ownership: nobody knows which page is current, nobody gets time to fix stale docs after the process changes, and after a few rounds of that the wiki becomes a museum piece new hires learn to ignore.

Configure Slack to only retain messages for a month or so. People will stop relying on it for such.

Slack is communication, not documentation.

It should be, but I’ve also seen few teams having everything on slack. Better ones at least pinned stuff

Treating either as gospel is lazy, Perlis was pushing back on dogma and Pike on theory, while legacy code makes both look cleaner on paper.

SQLite is fine right up until you want concurrent writers. Once you need multiple users, cross-host access, or anything that looks like shared infra instead of a local cache, the file-locking model stops being cute and starts setting the rules for the whole design. For collaborative versioning, Postgres makes more sense.

For a distributed VCS, what would be the need for such things? Even if it were a really big project, how many writes could be going on that this becomes a bottleneck? I don't see it but maybe you have a situation in mind.

In the current environment, even a distributed VCS may have concurrent agents modifying it on different branches.

The problem i faced is mostly importing large repos. But normal use should be fine.

The thing people tend to gloss over is how CoW shines until you need to update the base image, then you start playing whack-a-mole with stale memory and hotpatching. Snapshots give you a magic boot, but god help you when you need to roll out a security fix to hundreds of forks with divergent state.

Fast startup is nice. If the workload is "run plain Python on a trusted codebase" you win, but once it gets hairier the maintenance overhead sends you straight back to yak shaving.

Wouldn't you need to restart a process anyways if there's a security update? Sounds like you'd just need to kill all the VMs, start up the base again, and fork (but what do I know).

That is very true. We use copy on write for exe.dev base images right now, and are accumulating a lot of storage because of version drift.

We believe the fix here is to mount the base image as a read-only block device, then mount a read-write block device overlay. We have not rolled it out yet because there are some edge cases we are working through, and we convinced ourselves we could rework images after the fact onto a base image.

Right now our big win from copy-on-write is cloning VMs. You can `ssh exe.dev cp curvm newvm` in about a second to split your computer into a new one. It enables a lot of great workflows.