Via Jason Kottke comes this thought-provoking exercise challenging you to apply your own morality to difficult "trolley problem" scenarios that self-driving cars will have to deal with the moment they hit the streets. In other words, when a self-driving car must make a decision to kill (either its own passengers or pedestrians), what criteria should it use to make that decision?
Please go through the exercise yourself before reading any more of this post, as I don't want to poison your answers with my own.
Ok, all done?
There are no objectively right answers to this problem, but my strategy was as follows: First, I disregarded all demographic differences between humans. I don't feel comfortable assigning different values to men, women, the elderly, kids, athletes, criminals, obese people, etc. There was one question where I did have to use this as a tie-breaker, but that was it... and it still didn't feel good. Then, I optimized for saving people who were doing nothing wrong at the time. In other words, pedestrians who crossed on a Don't Walk signal were sacrificed pretty consistently. Then I optimized for greatest number of human lives saved (pets were toast... sorry pets). The hardest question came down to a scenario where you had to pick killing four innocent people in the car vs. four innocent pedestrians. For this, I chose to spare the pedestrians, as those who choose to take a vehicle seem like they should bear the risk of that vehicle more than those who made no such decision.
The summary page at the end is interesting, but it can also give false impressions. For instance, even though I explicitly disregarded demographics, it showed me as significantly preferring to save people who were "fit" and people who were "older". Depending on your strategy, some of these conclusions may be enlightening, and some will just be noise from a small data set. Also, don't forget to design some of your own. Here is the hardest one I could create, based on my own decision-making criteria.
Tough stuff, but it's good to get people acclimated to these dilemmas now, because although no technology can eliminate traffic deaths, self-driving cars will probably greatly reduce them. Curious to hear other strategies if you have them. Jason's, for instance, were different than mine. Also, can I just say that I love the idea of pets "flouting the law by crossing on a red signal?"
Read more ⇗