Absolutely true. However this comes at the cost of having to not think too hard about issues like "what is a function".
You generally don't run into trouble with 1, x, 1/x, sin(x) and the like. But when you push past the analytic functions, you wind up having to unlearn a lot of ideas so that you can learn an entirely different foundation.
Lest you disregard this as completely useless superstructure, note that basically the entirety of the theory of stochastic processes, starting with Brownian motion, is positively infested with continuous everywhere nondifferentiable functions; while the existence of a nonconstant infinitely smooth function with an identically zero Taylor series is what permits the Berezinskii-Kosterlitz-Thouless phase transition to exist. So while the weird animals of elementary real analysis are perhaps not the most important thing in the world, they are far from irrelevant to it.
You generally don't run into trouble with 1, x, 1/x, sin(x) and the like. But when you push past the analytic functions, you wind up having to unlearn a lot of ideas so that you can learn an entirely different foundation.