This is not the case any more and have not been for a very long time. There are plenty of game engines, and some of them are specifically targeting beginning game devs and abstract a lot of that stuff in really high level concepts that require no much more from the developer than some really basic arithmetics and geometry intuition.

In fact, there are so many beginner-friendly gaming engines out there for most languages, that I am convinced that we should start using games as the entry-point for teaching programming languages. It is a beatifully self-contained domain.