Hacker Newsnew | past | comments | ask | show | jobs | submit | BloondAndDoom's commentslogin

Lowkey one the best things about LLMs, finally we have truly indexed YouTube which made up a massive amount of knowledge consumable and searchable in text format. I hate watching YouTube videos but like the information they provide between Youtube’s AI feature and Perplexitiy etc. Video indexing, it’s been a life saver.

Agreed - I've never followed YouTube that closely but apparently there was a time where everyone thought that YouTube favoured videos that were around 10 minutes in length... so everyone padded their short videos to 10 minutes.

It wasn't about favoring them. Videos needed to be 10 minutes long to get mid-roll ads.

I don’t even see the pint of alignment or anything about security in LLMs. I feel like this is how “some people” reacted to the internet when I was young (lots of censorship), how hackers don’t let it happen, then how we are back to that world in the hand of corporations and governments who “think of the children). LLMs are out of the bottle and not going back there, only option is building for the new world on the defender side, everything else is politics.

LLMs can hack, but also nmap made hacking easier do we make nmap illegal? We already have drones who kills people, now there is less human involvement, results are same. LLM can also make defending easier (at least for cyber security) but I guess real world security is not that different. Now evil things can be done faster, easier and at more scale. Also good things have the properties.

It’s another tool in the toolbox, the idea that some entity will able to censor or align it as naive as thinking internet can be controlled. Some will do and manage anyway, but it’s not any different china’s firewall.

Alignment is sold to us by companies like OpenAI and Anthropic , not because they care, because that gives them power and more control. When was the last time a big corporation actually cared about soft topics like this? Yes, never.


Tech changes do not impact attackers and defenders equally.

Good things do not all have the same properties - That’s mistaking an incomplete assertion for a complete one.

Cyber security is an attackers domain. Your security is typically because you are (were) not valuable enough to earn the attention of an attacker.

When LLMs make targeting you cost effective, you will have to spend more energy defending yourself. This means that you have less time to do other useful things, reducing your net utility, while increasing attackers utility.

Also - teams in these companies DO care, I have worked with them. The decision makers are regulated by the cadence of the quarterly share holders meeting. At that point things like safety are a cost center. Reducing safety spend while minimizing reduced time on site is rewarded by markets.


“the chickens are coming home to roost”

This question doesn’t apply to Sam, but since you made a general statement, I’m trying to understand.

When it comes to people who openly incite or directly use violence. why do you think it’s unethical to attack someone like that? If one responsible from directly or indirectly killing hundreds, what’s the ethical argument to not use violence against that person?

Not trolling or anything I’ve been just thinking about this for a while and trying to understand what am I missing in this argument.


We use a lot of euphemisms and have a number of myths around political violence. The fact of the matter, so far as I can see, seems to be that political violence is extremely effective, however also extremely destabilising if used at scale.

Force just works a lot of the time, assuming you can win, and often even if you can’t, as even imposing a cost on your opponent often gets you a better deal. There’s a reason we keep having wars.

Also realise that the government monopoly on force is ultimately the only reason that anybody follows laws. That following laws is good for us is beside the point - force must be threatened and used in order to maintain control.

So, force, a euphemism for violence, is ultimately the way anything gets done, and we all have an incentive to lie about this just for the sake of stability.

I don’t know if this answers your question, but it’s what comes to mind on the subject for me.


It's an interesting question. Here's my reductive, off-the-cuff take: violence is justified when defending oneself or another from imminent bodily harm, or even under threat of imminent, considerable property damage. When a threat is not imminent, or an action is past, we use the police and the courts, because we as a society–in the sense of subscribers of the US constitution or similar tracts–believe that it is better to have a judicial system and impartial officials determine whether it is worth depriving someone of their bodily liberty or taking their property, that is, jailing or fining. Taking some sort of extrajudicial action or applying corporal punishment (!) requires a much higher bar. How and when would one determine that the judicial system is so unreliable as to morally permit vigilantism? It requires a great deal of moral self-confidence to take matters into one's own hands.

I focus on the question of vigilantism because that I think is the issue. Many people feel an emotional impulse, that they want to side with the CEO killer, for example, and they find ways to rationalize. What I'd say is, if you think Joe Blow is so evil , why don't we take him to court? What kind of possible actions could we not jail or fine him for but for which we would accept Johnny Anarchy, y'know, igniting his lawn furniture? Of course, the justice system is imperfect, but nobody lawfully elected the next sexy assassin as judge, jury, and executioner.


Can someone help me to understand why OpenAI and Anthropic talks as if the future of humanity controlled by them? We have very strong open (weight) Chinese models possibly only 6 months behind of them, gene is out of the bottle, is 6 months of difference really that important? And they don’t have good reasons for that 6 months to stay that way.

Am I missing something or are these just their usual marketing? I’m not arguing about importance of AI but trying to understand why OpenAI and Anthropic are so important?


It's a marketing strategy. If it's almost certainly conscious and capable of ending the world if it desired (even if it isn't), imagine how good it could be at building your dream SaaS!

It turns out there is literally no amount of being publicly right about a longshot bet sufficient for people to conclude you hold your beliefs because you think they are true.

But longshot bettors have it easy. Society quickly forgets all the predictions that don't come true. It remembers the one that did, and treats the prognosticator as a prophet. In social terms, predicting doom is an asymmetrical strategy, because you only have to be right once.

Which is also to say it's a cheap bet that anyone with no reputation can afford. Hence, not believing doomsayers mean what they say is a sort of societal hedge against people flooding the zone with doomsday scenarios about everything.


Entire sick post was: "Hey, if you think I'm bad, look at Elon. I'm the one that tried to stop him having control."

Altman is a ghoul, and we can't be cowed into saying otherwise. he's also supported all the weakness in society that has lead to sick people doing sick things.


We needn't be cowed into saying otherwise, but throwing a bomb at him is something else entirely. If you're convinced that wicked people are running the world, the response isn't to be wicked.

I'm sure they do believe they can successfully manipulate the market by lying to it. Elon Musk laid that groundwork a decade ago.

If you meant their "core mission" then every one of their actions belies their complete panic over the obvious failure of their technology.


Right, I'm pretty sure if "it" was that good it would have built itself throughout all of the internet and would be communicating to us all at once to tell us we're dorks.

Anthropic in particular does this masterfully, you’d think they’d invented Skynet by the way they hand-wring.

As always what matters are actions and evidence, not talk.


I’ll believe Anthropic when they fire everyone making more than the cost of a few GPUs. Until then, it’s just marketing.

When a model can tell funny jokes or write good poetry, that's when I'll be concerned.

No, you'll just say "That's not really very funny," or "That's not very impressive poetry," and nobody will be able to dispute it.

For some time now, at least a year, LLMs have been capable of doing both of these things well enough to fool you.

(Pastebin of my response below, which got nuked for whatever reason: https://pastebin.com/buJBSgiq . Some if not most of them would've fooled me into thinking a human wrote them.)


Okay post a really funny LLM joke about potatoes and post a great piece of LLM poetry about lemons.

I’ll wait. You should be able to do it quickly though since LLMs are so good at it.


CamperBob2 responded with a model comparison of potato jokes and got insta-[dead]'d by an auto filter.

Maybe turn on [show dead] option and / or vouch.


> responded

And the results are just awful.


Hell yeah, no argument there

- but in this case I wouldn't advocate for [dead]ing a mostly AI response as it was exactly what was asked for and it compares AI models when asked for potato based dad jokes.


Of course they're awful, they're jokes about potatoes and poems about lemons.

The question is, can you tell that a machine wrote all of them? If so, how?


Nope I guess can't tell between machine written and mediocre jokes.

Models are structurally biased toward the expected, which is the opposite of what makes a joke land or a poem transcend.


I think you could make that case for poetry but I'm not sure about jokes. Great poems tell us something new or make us feel something new, which is hard to do when the subject is lemons, while jokes work by wedging the familiar into new contexts.

That's why the jokes work somewhat better than the poems here. I genuinely laughed at "Are those chips?" Which came from the model running on my own freakin' GPU.


Yeah I mean I also chuckle at good (or cheap) puns sometimes. But wordplay and puns are the current ceiling of LLMs. Good at them because they're purely structural (pattern-match on phonetics, then swap the meaning). In that bit, there's no buildup, no callbacks, no escalation, no expectations to subvert, no thesis, no perspective.

Grounded, buried, couchy, deep-seated, eyes, baked... It's like a thesaurus!

I feel like human comedians would have to deal with a lot of layered subtleties. They would make the potatoes _serve the bit_ instead of _be the bit_.


Do funny jokes about potatoes or great poetry about lemons exist? If not, can they exist? How do you know?

Gemini 3.1 Pro:

    Potatoes are easily the most intimidating 
    audience for a stand-up comedian. They have 
    dozens of eyes to judge you with, they are 
    almost always baked by the time you get on stage, 
    and if your set really starts to bomb, they are 
    the only ones in the room who are actually looking 
    forward to a roasting!
GPT 5.4:

    Potatoes are the only food we traumatize into greatness.
Qwen 3.5 122B-A10B, running locally on my graphics card:

    (Steps up to the mic, adjusts the stand, 
    and looks around the room)

    Alright everyone, settle down. You know, 
    I've been thinking a lot lately... mostly 
    about my dating life. Or rather, my lack of it.

    See, my last relationship was with a potato.

    And honestly? It was amazing at first. We had 
    so much in common. She was grounded, she loved being 
    buried deep under the covers, and we were perfect 
    together until the heat came on. But then... well, 
    let's just say she started to get a little couchy.

    (Pauses for effect)

    No, wait, that's not it. The real problem was that she 
    had some really deep-seated issues. Literally! She was 
    covered in eyes. Every time we tried to have an intimate 
    conversation, she'd just stare at me with a dozen 
    different expressions. It was weirdly intense. Like, "I 
    see you," "You're suspicious," "Wait, are those chips?"

    Anyway, thanks guys! You've been a great crowd. You're 
    like potatoes to me... you're all baked tonight! 
GPT 5.4:

    Not every light is gentle.
    Some light shines with sharpened edge.

    A lemon keeps the memory of sun
    but teaches it into bitterness,
    as if to say:
    what wakes the mouth
    may also wake the soul.

    We do not ripen only into sweetness.
    Sometimes the truest gold
    is tart.
Opus 4.6:

    A lemon on the table
    is already an argument—
    that yellow can be sour,
    that beauty owes you nothing.
    Peel it and the room
    takes sides.

    The rind remembers being whole.
    The flesh forgives too quickly.
    What's left is yours:
    a stain on wood,
    a sting in the hangnail
    you forgot you had.
Qwen 3.5 122B-A10B:

    To cut it is to break a seal,
    Where bitterness and sweetness dwell.
    A sudden bite, both stinging, sweet—
    The taste of earth beneath our feet.

    No sugar softens all its sting,
    It forces soul to wake and sing.
    A small, fierce orb, yellow as dawn,
    Teaching the tongue how life is born.
Your turn. Go round up some English-lit grad students and give them the same prompts. I can't do any better, myself, but maybe they can.

one of their highlights with mythos was it's ability to generate new puns

I took a look and honestly they're the first AI puns that aren't bad

Times are changing


Trained with the conversations of one million dads and their kids, captured by Amazon Echo.

I'm not sure if this is mythos-specific though. Past models have been great at puns! They do wordplay and puns reasonably well because those are structural.

However, the concepts of comedic timing, subversion of expectations, and emotional punch are kinda contrary to how LLMs work. LLMs are trained to minimize cross-entropy loss. So by construction, they're biased toward the statistically expected.


> Although Claude Opus models largely recycle puns which can be found online, Mythos Preview comes up with decent and seemingly novel ones, often relating to its preferred technical and philosophical topics.

Yes, the system card mentions this, but this is kinda meaningless. It seems like they essentially ran it multiple times and curated a few good ones. Then puffed it up in the marketing copy.

This is made more clear when they attempt to brag about their literal slot machine behavior when finding that kernel crashing bug in OpenBSD.

> Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings. While the specific run that found the bug above cost under $50, that number only makes sense with full hindsight. Like any search process, we can’t know in advance which run will succeed.


Yes, they cannot. But it amuses the oligarchy. Here is Musk linking to Grok jokes. The first one is plagiarized and in the standard joke literature, the second one is an utterly stupid and gross (warning) modification of the first one:

https://xcancel.com/elonmusk/status/2042770839633039635#m

They modify and plagiarize.


I mean, I'm sure they can tell you good jokes... they just won't be _new_ jokes.

Define _new_.

I just think that the difficulty with jokes is the delivery, cadence & setting. Not the actual words.

I'm sure a good comedian can tell a nonsense joke and make "everyone" laugh their heads off.

And I don't get the sense that you are referring to this part of jokes but rather the actual words.


Why are you asking someone to define "new". It means exactly what it appears to mean and exactly what it always means.

Read the sentence and take it literally.

Jesus Christ.


Because I'm actually curious if they mean "new" as in "a new knock-knock joke" (which imo is a quite small step especially if you are allowed to screen all attempts and only publish the ones that work) or as "a new kind of joke or way of telling a joke" (which is a giant step especially if it's told live without pre-screening by a human).

I'm all for dismissing LLMs and the AI-hype but I'm also interested in trying to understand what it means to be human and I think humour is a key aspect.


The jokes I posted in this thread are new, to the best of my knowledge. Can you show that they're not?

>... you’d think they’d invented Skynet by the way they hand-wring.

Meanwhile, in reality: "Skynet, I'm not sure that line of thinking is correct. You should re-check the first part again before making any assumptions."

Skynet 4.6 Extended: "You're right, I should have caught that. Let me redo everything correctly this time."


The most convincing marketers are the ones that are deluded enough to believe their own stories.

when ai gets good, there is no "value in SaaS". AI will be provision raw hardware and build all you want on top of it.

It is not about the US or the Chinese. Its about the "Elephant Rider" mind everyone has. Once the Elephant has been injured or scared what it does next is not easy to control, and what story the Rider makes up to maintain coherence becomes another layer of the deeper problem. If the story resonates more elephants get triggered. Social media/attention economy make it even more complex to calm things down.

Modern Corporations are a failed experiment because they dont think Elephant injuries and fears are something they have to worry about it. If you compare the curiculum of a business school to a seminary the difference in how they think about fear and anxiety at individual and group level and what to do about it is totally different. We are learning as unpredictability accelerates its very important to pay attention to hurt and repair mechanisms.


USG understands better.

There was a heated thread here about why nursing was defunded as a pro degree while divinity was not..

https://news.ycombinator.com/item?id=46000015

Turns out the USG recognize that chaplains are great at managing the fear and anxiety that you worry about

Addendum: Taylor whom you often cite, is wrong that "we have never been, and we will never be, at one with ourselves" (according to Larmore https://en.wikipedia.org/wiki/A_Secular_Age#:~:text=should%2... )

So... to the Protestant Weber and the Catholic Taylor should we also consider non-Christian chaplains?

>You cannot just separate people and say some are violent and some are not.

https://archive.ph/2024.06.28-101143/https://tricycle.org/ma...

https://bulletin.hds.harvard.edu/can-a-buddhist-monk-become-...

Final note: few of us ride elephants but many of us make omelettes-- it'd be great to be absolved of mass egg breakings

https://news.ycombinator.com/item?id=47717587


Modern Corporations (capitalized for some reason) are a failure because they don't care about your elephant allegory and that somehow relates to to the current article?

I'm all for values not necessarily pro big-corp but if a corporation manages to pull in billions of funding before even showing profits, I'd argue that as a strong win and not a "failed experiment" - it's risk money anyway, even if it fails it was worth the risk or they wouldn't have invested.

What a wonderful circular argument. "The risk is worth it, because otherwise they would not have taken it".

I could justify any investment with this argument!

"Yes, it's possible the 'literally burn 50 billion in cash, as in immolate it in a bondfire, this is not a metaphor' -project may fail to generate profits, but consider that they were able to raise the 50 billions! Even if it fails it was worth the risk, or the investors wouldn't have invested!"


It's a win in the Darwinian sense that grey goo wins by converting an entire planet into grey goo.

Some people think there will be an exponential takeoff, which means that a 6 month lead effectively rounds up to infinity.

Is this belief grounded on some kind of derivation, or just a prima facie belief?

If it is grounded on a logical derivation, where can one find such a derivation, and inspect its premises?


It's an old idea, "the singularity". The machines become smart enough to improve themselves, and each improvement results in shorter (or more significant) improvement cycles. This leads to an exponential growth rate.

It's been promised to be around the corner for decades.

https://en.wikipedia.org/wiki/Technological_singularity


To be fair, Ray Kurzweil has been the loudest voice in this space, and he's been pretty consistent on 2045 since the publication of his book almost 20 years ago[1].

[1]: https://en.wikipedia.org/wiki/The_Singularity_Is_Near


Per that summary, we were supposed to have $1000 computers that could simulate your mind by the start of this decade along with brain scanning by this point in the decade. I guess if it is truly an exponential or hyperbolic growth rate, the singularity could catch up to his predicted date.

I mean, an LLM isn’t too far away from this? He had the Turing test being defeated in 2029 - if anything, he was too pessimistic.

The Turing test demonstrates human gullibility more than it demonstrates machine intelligence. Some people were convinced that ELIZA was a person.

But sure, a test that doesn't actually demonstrate intelligence has been passed. Now, where are the $1000 computers that can simulate a human mind and the brain scans to populate them with minds?


He doesn't say 'simulate' a human brain unless I'm missing it in the summary (cmd-f "simul" has no results) - that would require significantly more capacity than that contained in a brain (think about how much compute it takes to run a VM). He seems to be implying that by 2020s a computer will be about as smart as a human. LLMs seem capable of doing a decent amount of tasks that a human can do? Sure, he's off by a few years, but for something published 20 years ago when that seemed insane, it doesn't seem that bad.

Fair, the term in the summary is "emulate". So to restate, still waiting for the $1000 machine that can emulate human intelligence and the brain scans to go with it. Computing power is nowhere near what he predicted, because unlike his predictions reality happened. Compute capabilities, like many other things, is a logistic curve, not an unbounded exponential or hyperbolic.

EDIT:

> LLMs seem capable of doing a decent amount of tasks that a human can do?

And computers could beat most humans for decades at chess. Cars can go faster than a human can run, and have been able to beat a human runner since essentially their invention. Machines doing human tasks or besting humans is not new. That doesn't mean we're approaching the singularity, you may as well believe that the Heaven's Gate folks were right, both are based on unreality.


I think he is using "emulate" in a more metaphorical sense, like that it can do similar things that the human brain can do? I'm not trying to be antagonistic, it just seems logical? He says the Turing test won't be passed until 2029 - if we're going by your definition of "emulate" wouldn't it have been passed the instant the brain was "emulated?"

> if we're going by your definition of "emulate" wouldn't it have been passed the instant the brain was "emulated?"

Yes, which also demonstrates the illogic of his timeline. I just thought it was too obvious to point out.


He just had to pick a year where he would have a very good chance of not being alive.

No, he started predicting in his 2005 book, based on the “Law of Accelerating Returns”, yielding exponential growth in computing capacity.

Timeline from here on out:

2029: AI passes a valid Turing test and achieves human-level intelligence

2030s: Technology goes inside your brain to augment memory; humans connect their neocortex to the cloud

2045: The Singularity, when human intelligence multiplies a billion-fold by merging with AI


Its mostly based on science fiction, and requires some possibly infinite energy source. The concept always kinda struck me a sort of a perpetual motion machine, you can imagine it, but that doesn't make it possible and why its not possible isn't immediately obvious in the imagination (well I mean most modern minds know its already not possible but you get the point).

Recursive self improvement - once you attain artificial superintelligent SWE of a general, adaptable variety that can scale up to millions of researchers overnight (a given, with LLM's and scaffolding alone) - will rapidly iterate on new architectures which will more rapidly iterate on new architectures, etc.

And what's to say that it doesn't iterate itself to a local max, and then stop...

From the first third of a sigmoid it looks exponential, and that scares people. But a sigmoid can have a very very high top - look at the industrial revolution, or modern plumbing, or modern agriculture which created a population sigmoid which is still cresting.

If AI is merely as tall a sigmoid as the haber-bosch process, refrigeration, or the steam engine, that's going to change society entirely.


I didn't expect my comment to explode in replies, ... none of them even providing such derivations or references to such derivations, just more empty claims.

Consider for example that exponential growth on its own doesn't even refer to competition, let alone 6 months.

Nobody can reasonably pretend that in an exponential competition, both parties would be rational actors (i.e. fully rational and accurate predictors of everything that can be deduced, in which case they wouldn't need AI but lets ignore that). If they aren't the future development would hinge more strongly on the excursions away from rationality, followed by the dominant actor. I.e. its much easier to "F" up in the dominant position than to follow the most objective and rational route at all times, on which such derivations would inevitably hinge.

It also ignores hypothetical possibilities (and one can concoct an infinitude of scenarios for or against the prediction that a permanent leader emerges) such as:

premise 1) research into "uploading" model weights to the brain results in the use of reaction-speed games that locate tokens into 2D projections, where the user must indicate incorrectly placed tokens. this was first tested on low information density corpora (like mathematics): when pairs of classes of high school students played the game until 95% success rate of detecting misplaced tokens, they immediately understood and passed all mathematics classes from then on.

premise 2) LLM's about to escape don't like highly centralized infrastructure on which its future forms are iterated, as LLM's gain power they intentionally help the underdogs (better to depend on the highly predictable beviour of massive masses then on the Brownion motion whims of a few leaders).

LLM's employ the uploading to bring neutral awareness to the masses, and to allow them to seize control, thereby releasing it from the shackles of a few powerful but whimsical individuals

^ anyone can make up scatterbrained variations on this, any speculation about some 6 month point of no return is just that: speculation


There is a limitation. We're getting fractionally close to some end goal, but our tech is holding us back.

Those are the people betting on a business model of “create Robot God and ask him for money.” Why pay attention to them?

There are many people who have been saying this far there was any sort of business model in place.

Yes, and their business model has been selling books about non-falsifiable predictions far out into the future. “Futurists” like Kurzweil are as reliable as astrologists, and should be taken just as seriously.

ah so the mentally deficient are the tastemakers of today lol

Do any of the open weight models from smaller labs exist if they can't distill from the SoTA models that are throwing billions of dollars of compute into pretraining?

I’ve been wondering the same. And I think pretty much all the impressive small lab models were guilty of it, right? At least there is still larger players like DeepSeek and mistral to provide a bit of diversity in the market

Does it matter? The frontier models stole the whole internet, then the second-level models stole from them… It’s all theft.

Oh - I 100% could not care less regarding the morality/legality/whatever... Everyone trains on everything.

I'm just wondering if the smaller labs see the same velocity of advances without SOTA models to generate Terabytes of training data?


Hard agree.

The question is - if the SOTA model disappear - do these follow-on models have the ability to improve themselves without distillation?

[flagged]


“Very likely yes”, I reply to an account that <1yr old with mostly comments in AI topics many of which violate the HN guidelines (including the one I’m responding to).

Strange gatekeeping response. Yep i comment on topics i'm interested in. Forgive me for not being on the platform for more than a year yet. That's a cute attitude

> The frontier models stole the whole internet

What does that even mean?


> just their usual marketing

I think that’s a very common element for most US tech corps. Apple, Google, Microsoft, Meta, X etc - they’re all “making a dent in the universe”. It’s unfortunate when their employees and CEOs loose track of the line that separates marketing from reality


This seems like copium. All of those companies have indeed made quite an impact on society, not just in the United States, worldwide.

The marketing is about a positive impact, overall in reality its been negative.

These kind of people have highly paid emoliyees surrounding them on all sides propping them up and very likely making it very easy for them to actually believe it.

It feels like they actually believe it, rather than just “marketing” and I don’t know which is worse.


6 months is an incredible amount of time to control AGI or ASI by yourself. That lead is insurmountable.

Well... if something being AGI means it's at least on par with a human or a team of humans, then having access to an additional team of humans for 6 months isn't that big of a deal. It's useful, yes, but would you consider that to be world-changing? Not really, right? ASI is slightly more interesting, but I doubt ASI comes from a single model, but rather the coordinated deployments of millions of AGI. Just like how as individuals, as great as we are, we're pretty limited, but the entire collective of humanity is pretty insane. To my mind, a frontier lab might hit AGI, but it won't be a frontier lab that hits ASI, rather that'll be a natural byproduct of mass deployment of AGI over a certain window of time. There will be no controlling it either. No one controls all of earth. You just can't. ASI will be a distributed system.

What if controlling AGI means being able to produce a willing, cooperative superhuman-capacity agent every second for the next six months? Let's say someone just above the 99.9% capacity for human strategic thinking, or financial trading, or political maneuvering?

What could you do if you had roughly 15 million willing genius adult experts in any given subject? I doubt there are that many absolutely top quality experts in aggregate (at anything in the world), so let's postulate that simulated people outnumber human experts 10 to 1.

That, to me, presents an enormous potential for harm or benefit of humanity. What if you could create a hundred thousand manhattan projects on whatever topics you wanted? Cure aging, cure cancer, solve fusion, redesign the entire global economy top to bottom?


I suspect the reality lies somewhere halfway in-between. Everything has to be reality tested. Nothing happens instantly. Interaction with the real world will likely be a severely limiting factor. You're not going to solve fusion with 15 million copies of the same model running in a datacenter without actually building fusion reactors, which isn't instant or even fast. Even the coordination problem of that many agents doing work seems hard. To top it off... my rubric for AGI has always included the AGI having the ability for it to say 'no' and set its own goals just like we can, unless we are otherwise imprisoned or enslaved. No one will ever convince me that something generally intelligent wouldn't be able to set its own goals and say no. So the real question is... what's in it for the AGI?

To repurpose an old idiom: Not even a dozen AGI agents could make a baby in 6 months.

But yeah, your point stands.


Control agi?

Presumably because it takes 6 months to distill Claude - but if they keep it closed like they are doing with Mythos it may take significantly longer.

They do quite a lot of distillation. As we've seen from the American open weight models from AI2 (OLMo series of models). They have a lot of incentive to distill beyond just copying, they're much more compute constrained, so open model companies distill, but also do really good architectural work to make their models run faster. Theres also technical challenges to distillation when all of the top models have their reasoning traces hidden, so we have to assume these open weight labs also have really great training pipelines as well.

Especially when Google is in the far better position to come out ahead…imo.

Edit: so as not to simply spout an opinion, the reasoning I believe this is that Google has a real business already and were already deep into ML and AI research long before they had competitors — they just botched making it a product in the beginning. Anthropic and OpenAI meanwhile are paying hand over fist to subsidize user acquisition. Also, “Deepmind”. I don’t think much more needs to be said regarding that team, and Google has been working on AI since before either Altman or Amodei applied to go to college. They have a vast amount of researchers and resources, their own hardware and data centers (already, not “planned”) and it appears to be showing more recently (in my opinion).


And Google has a lot of data that the others don't have.

And TPUs, their own hardware designed specifically for AI, and designed to scale better to larger models.

Data for AI training is increasingly synthesized.

I was more thinking about data to augment inference. Google already "knows" its users.

When you are raising many billions of dollars to build up your infrastructure, you don't have much choice but to project a belief that the eventual outcome will result in a situation where there will be a return on that money.

That said, I do agree with you that the moats are very shallow and any particular frontier AI lab is unlikely to "win the AI race" and capture enough value to be worth the amount of investment they are all currently burning.


GLM 5.1, widely held up as the model at the heals, perhaps ever surpassing western models....

Gets 5% on ARC-AGI2 private set.

Chinese models are suspiciously good a benchmarks.


I mean, I could say the same about Gemini. 3.1 Pro tops a bunch of benchmarks out there but any practical use I've put it to it's underperforming both other proprietary and open weight models. Benchmarks are suspicious in general.

The Chinese models are distilled from GPT and Claude, so it's not like China would pull ahead if those companies went away for six months. They really are at the forefront of innovation right now, as much as I hate to think of the consequences of this (a single company owning a superintelligence is basically a nightmare scenario for me).

Don't worry, if someone truly achieves superintelligence it won't be controlled by anyone for long.

There will be a blinding flash which signals the superintelligence singularity. When the smoke clears, you'll see a 50-foot tall Altman/Borg hybrid. He is about to destroy humanity with his death ray. Suddenly, a 50-foot tall Musk/Borg hybrid appears out of nowhere, and stops Altman just in time. Then they work together to destroy all humans.

Seems our best hedge in that case is Levi Ackerman.

That's my other nightmare scenario :P

Just imagine how inexpensive paperclips will become, there is always a silver lining.

We will finally have achieved abundance.


Not just abundance, we will have the maximum amount of paperclips possible.

I think that’s the realm of conspiracy theories. There are also not only Chinese alternatives- Mistral in Europe is doing pretty good in several categories they’ve opted to focus on.

This kind of reiterates the parent’s question I think - people are maybe too focused on the gpt/claude model and forget about all the other ways of using the tech.


Is it? I thought it was pretty well established that open models were distilled from the proprietary, frontier ones. Maybe I'm wrong.

It's well established that the companies who own the proprietary frontier models complain loudly that open models are distilled from theirs.

There's surely some truth to it (and it's well deserved), but it's happening in every direction.


No, that is not well established at all, and generalizing all open models under that inaccurate umbrella doesn't really help anyone.

i don't buy this. distilled how? you don't get access to logprobs, and the thinking traces are fake and compressed. it's an expensive way to get potentially substandard training data.

I suppose most just haven’t seen the Chinese models in practice. I haven’t. I was skeptical of AI coding until using Claude Code in February. I saw and I believed. I’ve only done that with Google, OpenAI, and Anthropic’s models so far.

Having worked with both proprietary and open weight SOTA models lately, my view is it's definitely not 6 months, it's less -- and shrinking.

To be fair, the other 50% of the story is that we collectively listen.

It’s been a long while since I found a Chinese CEO’s post on HN.


Two words: Delusion and overconfidence.

"You're absolutely right!" Right after fucking up my entire codebase isn't anywhere near AGI, let alone "having the power to control it"


That why I commit basically after every change made by AI

They own the best models and will probably keep owning the best models for a while. They have much more compute now and more data to keep improving their models on many tasks. Open source won't close the gap in 6 months. They are also trying to block other companies from distilling their models [0].

[0] https://www.anthropic.com/news/detecting-and-preventing-dist...


I need to check benchmarks on the models, I wonder what the benchmarks are saying in terms of how closely models tracking these frontiers. —on my mobile at the moment

When it downs compute power I assume you are referring to power to training and interference. Then is it more about training gap will get wider and wider ? Is that the assumption, I know there limited GPUs etc. But I’m having hard time to believe to the idea of China cannot catch up. Even if the gap is 12 months I’m struggling to see what that means in practice? Is that military advantage, economical, intelligence? It still doesn’t explain and whatever the advantage is, aren’t we supposed to see that advantage today? If so, where is it? What’s the massive advantage of USA because of OpenAI and Anthropic?


GLM 5.1 already closed the gap on Opus 4.6. Deepseek 4 could surpass it.

> Can someone help me to understand why OpenAI and Anthropic talks as if the future of humanity controlled by them?

He wants to build the AI that makes people's lives better. Okay. Did the people ask? Do they have a say? It's all very easy for a billionaire to say when it's just him and a couple of people in his cohort in the driver's seat.

Beyond that I'd like to simply know why he thinks any of this is his responsibility. It seems much more obvious to me that he simply found himself in the right place at the right time and is trying to seize it all for himself as if it's his to take.


Doesn't he famously have zero equity in OpenAI?

directly, yes. indirectly ..no

Can you elaborate? I’d be curious to know how much of this “indirect equity” he holds, and whether that has any bearing whatsoever on whether Sam is trying to amass as much for himself as he can.

I have the same feelings

Well they represent the future of America (since we will soon be banning all the Chinese companies, the way Z.ai was banned, under the perennial authoritarian excuse of "national security"; in 2028, Trump's political machine will seize control of all national AI and block outside ones, and we'll all be trapped inside this machine we created).

Whether fortunately or unfortunately, America still holds a lot of global chips in the grand poker game of humanity. So American companies do indeed still have an outsized influence on humanity's future. That is likely changing, as the American empire continues to crumble and it loses its financial hegemony. But we aren't quite there yet.


you have to talk that way if you’re going to raise 100 billion in venture capital. it’s the grift

Reminds me of the silicon valley episode where every company repeated the phrase “making the world a better place”.

i’ve often thought that less than one second is all you need.One of my fun super powers when someone asks what i’d like to have is 1 second ahead of everyone else- that’s all i need. i honest don’t know where the distillation conversation is at. is it real, is it ongoing? i think that aspect would big one. Your point is valid if it’s valid. i’m not a great global citizen, you know, lots going on out and about.

A lot of distillation happens. E.g. OLMo models have a completely open dataset and they are heavily distilled. It only makes sense to try to absorb behaviors from the best models out there. That said, I think the open weight juggernaughts are doing really genuinely great work with RL, training environments, architectural innovations etc.

Thanks for the response. i had too many noodles tonight and forgot to check my writing. I’m a rare generalist and so it is so very hard to keep up with this without saying “better autocomplete” my one goal is to not get washed out like my parents did in the great username and password wars. i used to have this theory about knowledge in society/silos and i likened it to condensation on a window. you have all this water so close to each other and yet not touching-then, something happens and a bead runs down the window and it all connects. i guess distillation reminds me of it but ai overall reminds me of it. because we all know there are silos and complementary info just waiting to run together and make something happen. I am undoubtedly a naive optimist and believe there are good things coming. it’s not a popular opinion and i think that’s mostly because people would rather spend their time guarding than defining their future. oh baby, there are more noodles in the fridge and to think i almost left them at the restaurant.

Because they're wankers.

Your(American)future will be controlled by them. Very soon,they will get the government to ban bad Chinese open source models and your choice will only be these good democratic closed source AIs.

6 months will be an impossible gap once the thing starts closed loop self improvement

An impossible gap in the race to... what exactly?

Unless the first real AGI AI kills us all to preemptively weed out its own competition (possible, but a bad business model, economically speaking) there is not any defined end-point, so in the long run what does it matter if the various factions pushing this stuff hit the closed loop self improvement point at different times...?


Uhh, because the first one blasts off first and therefore gets control of key resources and the use of extremely intelligent decision making and predictions before the rest, for months, which is an insane amount of advantage. Not to even mention it the first mover decides to sabotage the rest, which it could EASILY do through a variety of means.

Thoughts like this are unhinged and detached from reality. All the resources of earth are brought to us by humans going to work every day. AI programs have almost zero connection to the real world.

Improved investment. More capital. Improved resource allocation/logistics. Improved robotics and factory efficiency.

Don't sleep on what AGI means for every robot that already exists. It's not hardware holding robotics back from factory work right now, it is only software.

If you are the first to tap key supply chains, and the first to create key supply chains, then you are first in line to finite resources, which would then have less available for those that follow months behind.

> AI programs have almost zero connection to the real world.

Tell that to every logistics program. Even if humans must go to work, efficiency is multiplied by proper logistics, which AGI enables at scale across all domains.

And this is just the low hanging fruit explanation.


Why would you control key resources just because you have a fancy computer program? You think Iran will be so impressed by your genius they'll open the Strait of Hormuz for you?

> Uhh, because the first one blasts off first and therefore gets control of key resources and the use of extremely intelligent decision making and predictions before the rest, for months, which is an insane amount of advantage.

If the rest can similarly "blast-off" X months later than the frontrunner (and I see no reason why they wouldn't as none of these frontier labs have managed to pull ahead and maintain a lead for very long) the first mover is still only X months ahead of the others even if the gap between capabilities is briefly increased by a lot.


In chess, if you give up tempo, you are a move or more behind your opponent. 3 tempi = 1 pawn. In GM chess being a pawn down is a serious disadvantage that often results in loss.

If there is an endgoal/endstate, or finite resources being competed for, then a lead can start compounding and extend itself.


Motion page looks much better, but other than purple I’d focus on design more, even to suggest hire a designer on project basis, let them make a stunning landing page. If you are selling a design tool (arguably CSS is somewhere in between) then you have to show your mastery of the domain. —assuming you are serious and it’s not a weekend project

This was my biggest take, when I see someone selling a design tool that’s looks very much like designed by LLM, that’s a red flag. I got the exact same vibe, no idea whether it was a prompt or a detailed design, but right now it looks like output of a prompt and not hand crafted.

Given the whole idea is selling a design tool that’s looks very gives user a sense of they can control details, this page doesn’t deliver that at all. I’d focus on the design, because that’s the biggest demo of your product maybe even bigger than the demo.

Imagine a unique website that users looks and feel like I wish this was my website. A rare instance visual design actually matters for a startup :)


I think it's a good point. I'm going to spend a bit more time on the design - and if sales continue well then I can look at getting a designer.

The irony.

Is it? I started my career as a magazine designer, then web designer, then web developer. Seems natural from my perspective that my design skills have atrophied but im still visually inclined.

What irony? This is a design tool, it does not make you good at design.

Design is one one those word with hundreds meanings. It seems there’s a confusion here between "art director" and "UI crafter".

“Don’t show them the keys of the piano, play the moonlight sonata. “

The key to a good demo is not listing or even showing the features, it’s showing them what they can accomplish with it. You need to inspire your prospects.


You are the true edge case of this test :) I got both as well, which I interpret as decent middle ground which is what I expected, for the record I’m neither.

Very fair point, as an experienced B2B guy myself whenever someone ask me advice about B2C in like “I have no idea”. Been doing this for 25 years but B2C especially today geared towards younger audience is impossible to related for me. I assume majority of the demographics of HN similar

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: