The basic principal behind this shift is that exposure to bad cholesterol acts like radiation dose. Both the intensity (how high LDL/ApoB is) and the time exposed (how many years) matter.
These newest guidelines from the AHA/ACC, released yesterday, operationalize this partly by using a new set of equations that predict cardiovascular risk over a 30-year period (rather than 10 years).
Very cool. I learned something new about why EMA (exponential moving average) is needed:
> EMA-based training dynamics like JEPA’s don’t optimize any smooth mathematical function, yet they provably converge to useful, non-collapsed representations.
All the papers say EMA avoids “representation collapse” without justifying it. Didn’t realize there were any theoretical results here.
Roughly, when you train a model to make its predictions align to its own predictions in some way, you create a scenario where the simplest "correct" solution is to output a single value under diverse inputs, aka representation collapse. This guarantees that your predicted representations agree, which is technically what you want it to do but it's degenerate.
EMA helps because it changes more slowly than the learning network which prevents rapid collapse by forcing the predictions to align to what a historical average would predict. This is a harder and more informative task because the model can't trivially output one value and have it match the EMA target so the model learns more useful representations.
EMA has a long history in deep learning (many GANs use it, TD-learning like DQN, many JEPA papers, etc.) so authors often omit defense of it due to over-familiarity or sometimes cargo culting. :)
LeCun's technical approach with AMI will likely be based on JEPA, which is also a very different approach than most US-based or Chinese AI labs are taking.
If you're looking to learn about JEPA, LeCun's vision document "A Path Towards Autonomous Machine Intelligence" is long but sketches out a very comprehensive vision of AI research:
https://openreview.net/pdf?id=BZ5a1r-kVsf
Training JEPA models within reach, even for startups. For example, we're a 3-person startup who trained a health timeseries JEPA. There are JEPA models for computer vision and (even) for LLMs.
You don't need a $1B seed round to do interesting things here. We need more interesting, orthogonal ideas in AI. So I think it's good we're going to have a heavyweight lab in Europe alongside the US and China.
BTW, I went to your website looking for this, but didn't find your blog. I do now see that it's linked in the footer, but I was looking for it in the hamburger menu.
Thanks! We need to re-do the top navigation / hamburger menu -- we've added a bunch of new things in the past few months, and it badly needs to be re-organized.
Very interesting. I am keenly interested in this space and coincidentally had my blood drawn this morning.
That said, have you considered that “Measure 100+ biomarkers with a single blood draw” combined with "heart health is a solved problem” reads a lot like Theranos?
FWIW, the single blood draw is 6-8 vials -- so we're not claiming to get 100 biomarkers from a single drop. The point of that is mostly that it just takes one appointment / is convenient.
This is very cool work! I have a quick follow-up: in the biomarker prediction task, what horizon (ie. how far into the future) did you set for the predictions? Prediction is hard beyond an hour, so it'd be impressive if your model handles that.
The prediction task is set up as predicting the next measured biomarkers based on a week of wearable data. So it's not necessarily predicting into the future, but predicting dataset Y given dataset X.
The specific biomarkers being predicted are the ones most relevant to heart health, like cholesterol or HbA1c. These tend to be more stable from hour to hour -- they may vary on a timescale of weeks as you modify your diet or take medications.
Appreciate your work! Healthcare is a regulated industry. Everything (Research, proposals, FDA submissions, Compliance docs, Accreditation Standards, etc.) is documented and follows a process, which means there's a lot of thesis. You can't sneak in anything unverified or unreliable. Why does healthcare need a JEPA\World model?
Regulation is quickly catching up to modern AI techniques; for the most part, the approach is to verify outputs rather than process. For example, Utah's pilot to let AI prescribe medications has doctors check the first N prescriptions of each medication. Medicare is starting to pay for AI-enabled care, but tying payment to objective biomarkers like cholesterol or blood pressure actually got better.
Oh boy. Now we’re entering the fiber era. We’re just leaving the protein era. Before that it was the intermittent fasting era. Before that it was the keto era. The low fat era was probably a few before that.
I hear about fiber constantly all of the sudden. You might be right about it, but how do we know it’s different than. All the past nutrition tends?
Idk about cholesterol, fiber is well known to be very healthy. Same for protein.
Losing body fat will often have the biggest impact by far if one is overweight, though. It also stabilizes blood sugar and has a lot of benefits in general.
Before manufactured insulin shots, the treatment for diabetes was a multi-day oatmeal fast. This has been around for many decades. The only thing that's changed is that you are finally hearing about it.
It is funny how you can break diet/nutrition into generations like this.
I think the trends are a reflection of poor education. Fiber/protein/whatever being important components of a diet isn't new information. But the information is new to folks that never had nutrition explained to them.
I feel like we're due for something really ridiculous next. I've been paying attention to macros, fibre, salt, and having a reasonably varied diet for years; we've done salt, fat, carbs, protein, and now we're doing fibre.
"Eat a varied diet" seems boring but maybe those influencers selling pills made from 500 vegetables were ahead of the curve all along.
It would probably be better to just eat all those different vegetables as part of actual meals to get a varied diet, rather than in pill form.
I was under the impression that more protein and less salt/fat/carbs are still kinda the trend? If more fiber gets added to the mix I guess it is essentially telling people to eat more plants, thus leading to more varied diets overall.
Because the trends are bullshit and nutrition is just not that complicated.
The trends are a strange type of nutrition entertainment for people to read and then ignore in practice. There is some kind of psychological comfort in the knowing you can switch to oatmeal next week while gorging yourself at the Cheesecake Factory.
Oatmeal is good for you. News at a 11. We have known this for at least that last 50 years.
iirc, from older articles, which differ from this nice result, bile acids contain cholesterol(s) and they're generally reabsorbed in the intestines, so the fiber is conjectured to bind with some before reabsorption, expelling the bound fraction of circulating cholesterol in feces.
this result in the paper is very interesting in the conjecture is that the gut microbiome is altered in a beneficial way, and that the effect (with the resulting lowering of cholesterol) persists for weeks after even 2 days of oats.
We know almost nothing about how digestion works, but fiber has the added benefit of lining your intestines, preventing the absorption of some nutrients. It also helps push things through, so they spend less time sitting around being absorbed.
- "cardiovascular mortality ": > eating approximately 50 grams of soy protein a day (no small amount as this translates to 1½ pounds of tofu or eight 8-ounce glasses of soy milk!) in place of animal protein reduced harmful LDL cholesterol by 12.9 percent. [1] Such reductions, if sustained over time, could mean a greater than 20% lower risk of heart attack, stroke, or other forms of cardiovascular disease.
- "risk of cancer": many studies shows breast and prostate cancer reduction, but that is probably more related to isoflavones (Phytoestrogen) than fibers.
makes it sound like this is unrelated to soy specifically and more about displacing less healthful things (like higher saturated fat and caloric animal sources)
That’s fair. Re-reading the citation: tofu and soy milk contains very low amount of fibers so it’s probably not a greet exemple to illustrate "soy protein" if the fibers are at play. Or they aren’t. A dive into the source seems reasonable.
Note that saturated fat is also present in plants based food like peanut butter, although that one also contains tons of fibers (absent in animal sources). Coconut oil on the other hand is a tasty evil.
Long time ago I used a fetal Doppler to detect heart failure in adults. It was not based on LLMs (2016) but on a HMM to learn the hidden states: The heart beats.
It was tested with the Physionet database. It performed quite well, but I didn't invent the algorithm; it came from a Physionet competition.
The most surprising thing I learned that coconut milk is a lot less healthy than I would have guessed (high in lauric acid).
I had always heard that dark chocolate was heart healthy, in spite of being high in saturated fat. In addition to the effect of antioxidants, the stearic acid in dark chocolate quickly converted by your liver into oleic acid (a monounsaturated fat).
I honestly haven't used ChatPRD. But I think the biggest advantaged are the simplicity and core flow of the synthesis model I built! I built it for myself as a PM, so it doesn't have a bunch of the feature cruft.
These newest guidelines from the AHA/ACC, released yesterday, operationalize this partly by using a new set of equations that predict cardiovascular risk over a 30-year period (rather than 10 years).