> an LLM can ingest unstructured data and turn it into a feed.
An LLM can try to do that, yes. But LLMs are lossy compression. RSS feeds are accurate, predictable, and follow a pre-defined structure. Using LLMs to ingest data which can easily be turned into an parseable data structure seems strange: use the LLM to do the "next part" of the formula (comprehension, decision making, etc)
I mean that your RSS feed can basically be "Go to https://techcrunch.com/latest/ and use each non-video item as a feed item" or "Go to x.com/some_user and make each tweet a feed item", and the LLM can do a perfect extraction of links from html response blobs.
The only thing you have to do is ensure it can reliably get the response html. Maybe MCP browser + proxy or mirror to seem more human.
I built this for myself. The idea is that each feed is a url + title + a prompt to tell the LLM how to extract the links you want so that it generalizes over all websites.
And each feed item is a canonicalized url + title + a local copy of the content at that url which is an improvement over RSS since so many RSS feeds don't even contain the content.
An LLM can try to do that, yes. But LLMs are lossy compression. RSS feeds are accurate, predictable, and follow a pre-defined structure. Using LLMs to ingest data which can easily be turned into an parseable data structure seems strange: use the LLM to do the "next part" of the formula (comprehension, decision making, etc)
There is also LLMs.txt https://llmstxt.org/ eg https://joshua.hu/llms.txt / https://joshua.hu/llms-full.txt