I agree with your conclusion, but that's by design. The goal is not to tell people the truth (how would they even do that). The goal is to give the answer that would have come from the training data if that question were asked. And the reality is that confirmation is part of life. You may even struggle to stay married if you don't learn to confirm your wife's perspectives.
> The goal is to give the answer that would have come from the training data if that question were asked.
Or more cynically, the goal is to give you the answer that makes you use the product more. Finetuning is really diverging the model from whats in the training set and towards what users "prefer".
The loss function is based on predicting the response based on the training data, or based on subsequent RLHF. The goal is usually to make money. Not only does the training data contain a lot of "you're absolutely right" nonsense, but that goal tends to push more of it in the RLHF step.
> You may even struggle to stay married if you don't learn to confirm your wife's perspectives.
I don't dispute that but man that is some shitty marriage. Even rather submissive guys are not happy in such setup, not at all. Remember its supposed to be for life or divorce/breakup, nothing in between.
Lifelong situation like that... why folks don't do more due diligence on most important aspect of long term relationships - personality match? Its usually not a rocket science, observe behavior in conflicts, don't desperately appease in situations where one is clearly not to blame. Masks fall off quickly in heated situations, when people are tired and so on. Its not perfect but pretty damn good and covers >95% of the scenarios.
All this, and yet, people are so angered by the term "stochastic parrot".
I use LLMs every day, I use Claude, Gemini, they're great. But they are very elaborate autocomplete engines. I'm not really shaking off that impression of them despite daily use .
It's weird. It's literally what they are. It's a gigantic mathematical function that takes input and assigns probabilities to tokens.
Maybe they can also be smart. I'm skeptical that the current LLM approach can lead to human-level intelligence, but I'm not ruling it out. If it did, then you'd have human-level intelligence in a very elaborate autocomplete. The two things aren't mutually exclusive.
People are hung up on what they “really” are. I think it matters more how the interact with the world. It doesn’t matter if they are really intelligent or not, if they act as if they are.
Yes, it is. But those distinctions are going to be a lot less relevant with robotics. It won’t matter if it’s impatient or just acting impatient. Feels slighted or just acting like it feelss slighted. Afraid, or just acting afraid. For better or for worse, we are modeling AI after ourselves.
I am hearing this term for the first time but I love it. It is novel and creates a picture. Exactly what Scott Adams says about labels used for persuasion. I usually say "highly trained autocomplete" in discussions at work, but I am going to say "stochastic parrot" from now on.
oh, OK. You should google the term to see where it comes from. it's from someone who is essentially an anti-LLM activist and it's meant as a slur. That's likely why people consider it to be a slur, due to its origins.
> You may even struggle to stay married if you don't learn to confirm your wife's perspectives
Nope. You picked the wrong wife if that is the situation you are finding yourself in. My partner and I accept each others perspectives even if we disagree. I would never date a woman who can't accept that different opinions exist and that we both will sometimes be wrong.
>And the reality is that confirmation is part of life.
Sycophantic agreement certainly is, as is lying, manipulation, abuse, gaslighting.
Those aren't the good parts of life.
Those aren't the parts I want the machine to do to people on a mass scale.
>You may even struggle to stay married if you don't learn to confirm your wife's perspectives.
Sorry what?
The important part is validating the way someone feels, not "confirming perspectives".
A feeling or a perspective can be valid ("I see where you're coming from, and it's entirely reasonable to feel that way"), even when the conclusion is incorrect ("however, here are the facts: ___. You might think ___ because ____, and that's reasonable. Still, this is how it is.")
You're doing nobody a favor by affirming they are correct in believing things that are verifiably, factually false.
There's a word for that.
It's lying.
When you're deliberately lying to keep someone in a relationship, that's manipulation.
When you're lying to affirm someone's false views, distorting their perception of reality - particularly when they have doubts, and you are affirming a falsehood, with intent to control their behavior (e.g. make them stay in a relationship when they'd otherwise leave) -
... - that, my friend, is gaslighting.
This is exactly what the machine was doing to the colleague who asked "which of us is right, me or the colleague that disagrees with me".
It doesn't provide any useful information, it reaffirms a falsehood, it distorts someone's reality and destroys trust in others, it destroys relationships with others, and encourages addiction — because it maximizes "engagement".
I.e., prevents someone from leaving.
That's abuse.
That, too is a part of life.
>I agree with your conclusion, but that's by design
All I did was named the phenomena we're talking about (lying, gaslighting, manipulation, abuse).
Anyone can verify the correctness of the labeling in this context.
I agree with your assertion, as well as that of the parent comment. And putting them together we have this:
LLM chatbots today are abusive by design.
This shit needs to be regulated, that's all. FDA and CPSC should get involved.
That's probably true to some extent, but I'm not completely on board.
> 1. Anything that is in the world when you’re born is normal and ordinary and is just a natural part of the way the world works.
Television and calculators were in the world when I was born, but I never viewed them as "natural". TV always seemed to be a way to distract yourself from the world.
> 2. Anything that’s invented between when you’re fifteen and thirty-five is new and exciting and revolutionary and you can probably get a career in it.
I was happy to get on board with the WWW, the web browser, and widespread email usage. Those were revolutionary technologies with immense values. On the other hand, I'm still not on board with text messaging, phone scrolling, or social media. If I could, I'd eliminate social media from society.
> 3. Anything invented after you’re thirty-five is against the natural order of things.
I'm over 50 and a strong believer in the value of the LLM. It's a work tool that I can use at work and put away when I'm at home (or not, depending on my mood). It's new and exciting and revolutionary and a move in the right direction for humanity.
> Why is that argument always applied against Linux, and never against for instance macOS, which also can't run Windows software?
There's a certain type of technical user that gets joy from coming up with arguments, good, bad, or just pulled out of their butt, explaining why people can't use Linux. I'm not going to spend my day trying to understand people's unusual preferences.
Another day in the post-copyright world. Surely someone somewhere is already using this to test the effect of copyright laws, should we decide to go back to that world.
My interpretation is that Claude did what Knuth considers to be the "solution". Doing the remaining work and polishing up the proof are not necessary to have a solution from this perspective.
The insight is the point of research. Proof isn't the desired product of research, it's simply an apparatus that exists for the purpose of verifying and demonstrating correctness of insight.
Yeah, and I'm not sure what the other guy's argument is. It's Knuth, the primary researcher, who is giving the praise here. I don't see a possible motivation he would have to falsely give accolades to a AI for a problem he presented, then cleaned up to solve.
> I could see a company setting a 6-month timeline initially, so they can reevaluate the program and choose how to evolve their support for open source.
There's nothing about this "for open source". This is for the celebrities of the open source world. "Use our product and let us advertise that you're using it." Nice try, but this is a pretty common marketing strategy, so no point pretending it's about supporting open source. A big name open source project adopting their products provides massive value to the company. Actual support would be giving access to the non-celebrities of the open source world.
> No wonder it a builder has to build something over 2000 SQFT to make it pencil out.
I'm with you up until that. Maybe there are places where you have to build over 2000 sf due to regulations. For the most part, this is an industry talking point to justify building expensive houses on the limited land that gets zoned for residential. It gets repeated a lot.
You can build smaller houses but you can't charge as much for them. I'm not faulting the builders for maximizing profits, but it's still a talking point. (And in the grand scheme of things, it's not the reason housing is unaffordable.)
> I'm with you up until that. Maybe there are places where you have to build over 2000 sf due to regulations. For the most part, this is an industry talking point to justify building expensive houses on the limited land that gets zoned for residential. It gets repeated a lot.
I kind of feel it is the inverse.
If you can build a house for X$/sqft, you have a linear relationship. If it costs 100k _plus_ X$/sqft (for sewer, permits, etc) now you have a floor. You can sell a bigger house for 600k, or a 35% smaller house for 425k, odds are you’ll sell the bigger house quicker. I bet the 325k homes would sell like beanie babies in the 90s in places like sf.
The actual problem, the elephant in the room, is that California is expensive, both by popularity and regulation. This makes for an embarrassing conundrum where California is simultaneously pushing poor people out while trying to subsidize their life via social programs.
> You can sell a bigger house for 600k, or a 35% smaller house for 425k, odds are you’ll sell the bigger house quicker.
Yes, and that's the point I was making in my comment. What's driving the price gap is that you're selling to a group with a much higher capacity to pay when you're building the 2000 sf homes.
But that has nothing to do with it being unprofitable to sell lower-priced homes, it's just less profitable. The talking point makes it sound like the home builders are trying to do good things, but they're victims of the government. No, it's nothing more than a desire to maximize profit, because the folks that are looking for 2000 sf homes are high income households with kids, and they're willing and able to pay a steep premium for that house.
Didn't see this until now, but it's still worth a response:
> You believe that builders are this nefarious cabal.
No, calling them profit maximizers has nothing to do with them being a "nefarious cabal".
When folks come along and ask "Why can't I afford a house?" the builders don't give the honest answer "Because we're maximizing profit, and that means we don't build houses for you." Instead they say "We're trying but the government won't let us build houses for you."
>>> What surprised me the most was the fees that just piled on. I knew the land, labor and materials costs.
Just the sewer (the capacity only, no work done) was $11k. Then add on the park and school fees which both were over $10k. No wonder it a builder has to build something over 2000 SQFT to make it pencil out.
>> The actual problem, the elephant in the room, is that California is expensive, both by popularity and regulation. This makes for an embarrassing conundrum where California is simultaneously pushing poor people out while trying to subsidize their life via social programs.
> I think your comment works by making a bold assumption that construction in California is wildly expensive than elsewhere and that it’s the major driver of cost.
This is a good example of why it's insane that nobody at Mozilla cares that they hire CEOs that have only a LinkedIn page. If you want to visit the website of the Mozilla CEO, you have to create an account and log in. No big deal if it's a CEO of a plastics manufacturing company, but when the mission is fighting against the behavior of companies like LinkedIn, it makes me wonder why Mozilla exists.
The CEO role at Mozilla is unstable. Even if Mozilla didn't require a LinkedIn page, chances are their CEOs would have an up to date account. Also, Mozilla's ARR is mostly their Google partnership.
If you visit the Mozilla website right now, you will see "Break free from big tech — our products put you in control of a safer, more private internet experience."
I don't think Mozilla requires a LinkedIn page. bachmeier is complaining that Mozilla's CEO doesn't have a personal webpage, and only has a LinkedIn page. By not having a personal webpage, and having a LinkedIn page, it appears that Mozilla's CEO doesn't really care about the open web.
The surest sign of incompetence is somebody claiming they are forced into a requirement for perfection when the requirement is simply a basic adherence to virtue
Time for folks to familiarize themselves with Linux distros designed to run on older hardware. My 2009 laptop runs great, with the exception of the browser. Oh and the fact that 32-bit software is harder and harder to find.
Yes, I'm grateful I run Linux. You can get quite a bit done with 4GB RAM and a 6th generation (or even earlier) CPU. All 64-bit. I don't think such ancient hardware will be affected by AI demand to the same degree (though I think we'll still see some prices rise if people stop buying new stuff).
The worry is that at some point the older hardware will stop working.
> Open source is a no-strings-attached gift, and all participants should recognize it as such.
I've always felt this is incorrect. First, because lots of people use open source to further their careers, it isn't. Open source contributions are paid work if you benefit from them in any way. Second, if you use open source yourself, your work is no longer a gift. You're contributing back to the community you've taken from. The person you're being a jerk to because it's a "gift" might be the author of other software you've used.
Why is this down-voted? Everything we do in this industry rests upon somebody else's gift of free/open software, that is the reality of it. I do feel an inclination to contribute back however I can, even if it's just documentation, or submitting a (properly detailed) bug report, for precisely the reason described here.
It is correct. Just because your livelihood might hinge on a specific project doesn't mean that its developers have any obligation to accept your patches. It means you have to be very careful in choosing which project you hitch your wagon. Obligations are created, in order, through law, contracts or personal relationships. If you have neither, you have nothing.
> It is correct. Just because your livelihood might hinge on a specific project doesn't mean that its developers have any obligation to accept your patches.
I think you responded to the wrong comment, since I didn't say anything about being obligated to accept patches.
I responded to "You're contributing back to the community you've taken from" which implies that one has some obligation to accept a patch (calling it a contribution). Just because you're offering doesn't mean that others are obliged to accept or even review it.
reply