What you're saying is outright false. If read llm papers you'll see that a lot of benchmarks used are related to math and logic.

GPT3 paper as an example https://arxiv.org/abs/2005.14165

I recommend you read that paper as it answers a lot of misconceptions you may have about llms.