The trolley problem is a simplification of a difficult ethical question. I wish people would stop dismissing it as though whatever simple answer they come up with makes the problem go away. In many situations, a car is going to have to choose between alternatives that each carry different risks. There simply are no universally acceptable simple rules to follow in such cases.
There is one universally acceptable rule: Do not choose your victim. If they are unlucky, they are unlucky. That's all there is to it.
Choosing who should be "unlucky" doesn't solve the problem. By assuming unavoidable failure you effectively give up solving the underlying problem. The solution is to avoid the failure in the first place.
No system can avoid failure 100% of the time. It may find itself going too fast with a unexpected loss of some control - brake failure, intentional sabotage, (imperceptible) oil slick, whatever. However, the system still has some control to choose between several terrible and costly outcomes. It must choose its victim. How should it chose?
Does what humans actually do matter? What matters is what we want the machines to do, which is going to be worked out between buyers, manufacturers, lawyers, lawmakers, and the rest of us.
