A few years ago, I had a colleague that attended Google I/O. This was when Google Glass was still a Big Deal, before the creepy gadget joined the fitness tracker and bluetooth headset at the bottoms of junk drawers.
My colleague asked a panel of Google engineers and product managers to describe how they juggle ethical dilemmas encountered during product development. Their answer was that Google did not have a designated ethicist, but instead relies on each individual engineer to make ethical decisions.
I guess they figure the result of many small ethical decisions will naturally produce an ethical product. I guess they figured the overwhelming white dude Google employee demographic was qualified to answer any and all ethical questions.
My colleague, a huge fan of Google’s, was disappointed by this answer. If something is everyone’s job, then it becomes no one’s job. Like unloading the office dishwasher.
Turns out that Volkswagen has been cheating at vehicle emissions tests since at least 2009. Someone – almost certainly a high-up manager – decided to break the law. That manager needed engineers to actually carry out the dirty work.
During at least six years – when VW created a public health risk, lied to regulators, and lied to consumers – not a single engineer stood up. Not a single one went above their manager’s head to say “what we’ve done is wrong.” Or to a newspaper. Or Twitter.
So software is eating the world, huh? Well, software is built by engineers. Engineers at VW have hurt people. Engineers at Google didn’t foresee privacy concerns over Google Glass, and wrote software that called black people gorillas.
Software isn’t eating the world, Engineers are, and that fucking terrifies me.
I’m not sure what an ideal form of ethical oversight might look like, but anything would be an improvement over what we do now: nothing at all.