It's not as easy... For example, it is possible and legal to use your own number to call from a VoIP provider, so the recipient can call you back on your actual phone.
On the other hand, it should be possible to detect at least a percentage of spoofed caller IDs and block them (e.g. non-existing numbers).
The VoIP provider could forward the call to your phone as a middleman, or there could be 3 numbers(1. Who to bill 2. Calling number 3. Reply-to number) and only #3 is user-configurable.
The idea of using a VoIP provider would be to _avoid_ using one's phone, e.g. to get better rates... And the reply-to number, it's unfortunately not the way telcos work... and changing all the infrastructure for this would be very very expensive.
On the other hand, it should be possible to detect at least a percentage of spoofed caller IDs and block them (e.g. non-existing numbers).