Because it seems like, regardless of the announcement, there will always be someone who has the most niche issue with it and manages to make assertions for an entire group of people while only really referencing their personal experience ("and all of the people they know").
I mean, I am the strongest local LLM advocate you will find. I have my GPU loaded with a model pretty much all day, for recreation and work. My job, my livelihood involves running local LLMs.
But it's intense, even with a very finicky, efficient runtime on a strong desktop. Local LLM hosting is not something you want to impose on users unless they are acutely aware of it, or unless its a full stack hardware/software platform (like the Google Pixel) where the vendor can "hide" the undesirable effects on system performance.
I think that's a reasonable generalization to make.
Fair but google does, _supposedly_ have a Gemini model meant to run on phones so it'd presumably be small enough that it wouldn't necessarily be a massive problem. Or, at least, we could get there eventually. Not arguing at this point, you're right. I just think over time we could get there
Running "smart" LLMs locally takes a lot of RAM, a lot of compute, and a lot of disk space.
It produces a considerable amount of heat unless it's run on an NPU, which basically doesn't happen on desktops at the moment.
Hot loading/unloading it can be slow even on an SSD.
Users often multitask with chrome in the background, and I think many would be very displeased to find Chrome bogging down their computer for reasons they may not be aware of.
Theoretically Google could run a very small (less than 2B?) LLM with very fast quantization, and maybe even work out how to use desktop NPUs, but that would be one heck of an engineering feat to deploy on the scale of Chrome.
Honestly that sounds extremely feasible, especially for a feature that isn't on by default. The one the parent comment references in Arc isn't on by default. Also chrome eating up system resources is already a meme and they've been working on using less by sleeping tabs.
