An LLM able to e.g. summarize the gist of a long text conversation in a group chat that I missed, able to understand queries like "what time did X say we'd meet up, and how long would it take me to get there" etc. would be incredibly useful to me.

Sure, most of that is possible server-side – but the appeal would be that local processing could preserve a lot more privacy.