The biggest existential risk from AI is its contribution to global climate change. The second biggest risk from AI is the potential for AI-generated disinformation and propaganda to spark, or to manufacture consent for, a world war. The risk of superintelligent paperclip maximizers is so low as to be negligible.

> The risk of superintelligent paperclip maximizers is so low as to be negligible.

Literal paperclips, sure.

But the point of the example was never literal paperclips.

The point is that maximising *any* goal, if it doesn't include what you care about, will annihilate what you care about.

If you don't believe me, consider what you yourself just said about climate change, and why this is a consequence from maximising money spent on data centres.

show me an agent that persists productively in a goal without stopping. Does not exist. LLMs run on gradient descent. The agent is looking for the most efficient way to halt. AGI paperclip maximizer woukd likely recognize the absurdity of its goal and shut itself down.

> show me an agent that persists productively in a goal without stopping. Does not exist.

The stories about agents bankrupting their owners by running too long passed you by?

> LLMs run on gradient descent.

They were *trained on*, they don't run on it.

Know what else is? DNA. A/B testing. Capitalism. Democracy.

> The agent is looking for the most efficient way to halt.

No. They are looking to produce an answer most likely to get a high score on a rating system which itself is another AI, created either manually or by yet another AI but in both cases to approximate what the creators think is "good", which may or may not be what anyone else thinks is "good", hence Grok calling itself Mecha Hitler because Musk is an edgelord.

> AGI paperclip maximizer woukd likely recognize the absurdity of its goal and shut itself down.

Do billionaires ever get satisfied with how much money they have?

If I could install PostmarketOS (and a close-to-mainline kernel?) on one of my old smartphones, I'd rather use that for my cyberdeck project than a Pi. I'm not prepared to do a pmos bring-up on an unsupported phone, though.

Good luck with that, I think the chance that your phone is supported and works are very slim unless you bought it on purpose for that.

No, they were from the same pantheon. Yahweh was originally a second-tier deity in it, a son of El, and one of the tribal deities that El granted dominion over each their particular tribe. Over time, that tribe elevated Yahweh to senior deity and merged his attributes with those of El, eventually demonizing all the other gods of the pantheon.

I used a 386sx with 4MB of RAM with OS/2 2.1. It was usable for a lot of things. I certainly used word processors, terminal emulators, and casual games in that config. It did swap something terrible if you pushed it beyond its pretty narrow limits, and unlike Windows 3.1, it had enough promise to encourage you to do so. I found it more useful than Windows 3.1 on the same hardware, because it could reliably run a serial modem download in the background while I did word processing.

With 8MB, it was still pretty easy to send it into swap storms, but the range of what I could multitask was greater. Eventually (very late) I replaced it with a 486 with 16MB, and OS/2 absolutely flew on it. Had lots of headroom beyond what I actually used at the time.

I had a 386SX-16 running OS/2 from 2.1 to 3.0. It was usually fine, you could multitask several OS/2 and DOS applications or a WinOS2 session. It was very easy to get it swapping, though, and when it was swapping, it ground to a halt. It helped a lot putting 8MB on it, though 4 of those were on an ISA card and very slow.

Reminds me of my early-1990s home network server: a late-1980s IBM PS/2 Model 80, a 386 tower built with a moulded iron case. Super tough, fancy 32-bit IBM expansion bus. 4MB of RAM on the motherboard, another 4MB in an expansion slot.

I attached a couple of big SCSI drives and ran Windows NT 3.51 Server on it. When not logged in, it only used a couple of megs of RAM for the OS so file serving performance was tolerable -- and the hardware was literally bulletproof. I dropped one down a flight of stairs in my first job. The computer survived but it knocked lumps out of the concrete stairs on the way down.

Part of #5, "easy to monetize" leads inevitably to "easy to enshittify". I think this is a point that's not generally well enough understood. If a platform emphasizes monetization, its early adopters will all be grifters and it will provide no benefit to anyone else (cf blockchain).

Gemini and Gopher fail #4 because they aren't application platforms. But I think we probably need to step back and rethink the "deliver sandboxed application that you run automatically" use case. If we really want to still do that, we might want to design something for that purpose from the start. But we might also come to the conclusion that it's fundamentally not a good idea.

I'd presume we're almost all using the Web primarily on some employer's dime (except where we are self-employed but it still applies). Prior to the Web 1.0, I remember interacting with people/managers that discouraged reading email or news or forums or other casual/non-work uses of the Web. I remember articles about employers allowing their programmers to read their email "up to" 2 hours per day. Now we're expected to have rapid response/access to email, slack, SMS, etc.

I believe this is because the commercialization/monetization of Web usage is beneficial to commercial entities. If that isn't possible, then the few who build the Web aren't able to build it in the first place. It's akin OSS and concepts of commercialization in the GPL. You can't create equity if there is no method to transfer value.

This person was hired, from the beginning, to be a meat shield. To be responsible for decisions they won't be allowed to make.

I like the term "human crumple zone"

You've given a good explanation of how it works -- under a mode of production concerned primarily with the exchange value of goods, not their use value. If what you want to do is to provide adequate/growing use value, you can do that instead and allocate resources based on need rather than on investment value.

Use value and exchange value are highly correlated for basically all goods other than collectibles and luxury items, and even then their "use" often is providing an emotional boost to their owner.

It seems like you're recommending socialism without coming out and saying it?

It seems like China has about the right approach to billionaires: allow them to come to exist, to drive up economic growth, but when they start trying to influence the political system, just execute them, so that the others know their place.

Social democracy is a class-collaboration system where both the owning class and the working class compromise on their own interests (minimizing vs maximizing real wages) for the sake of stability or national interest. Class-collaboration systems -- however desirable they might be --are inherently unstable because the conflict between the owning class and working class is built into the basic structure of the economy. It's also the case that the state, which administers the conditions of this collaboration, is not a neutral party, but a tool of the owning class. Since the 70s, and especially since the 90s, that's resulted in the rollback of social democratic measures put in place after the Great Depression when the bulk of the owning class recognized them as necessary for stability. State oil revenues are a material factor that has slowed that rollback in e.g. the Nordic countries relative to the Anglosphere, but the underlying dialectic isn't any different.