A good read if the How and Why NNs are working is of interest. Also the HN discussion [1] about this paper [2] is related from my point of view.

[1] https://news.ycombinator.com/item?id=40019217

[2] "From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examples" https://arxiv.org/abs/2404.07544v1

(Edit: format)