In the days since a self-driving Uber vehicle killed a pedestrian in Tempe, Ariz., local police and federal agencies have yet determine whether Uber’s technology was at fault.
Uber says a single metric isn’t a clear indication of an autonomous car’s safety
Drivers still had to take over once every 13 miles in Arizona, according to new documents the New York Times obtained.
Smith Collection / Gado / Getty Images
But new documents obtained by the New York Times show Uber’s technology had made little progress in the last year. The story details the series of setbacks the company faced in trying to get self-driving cars to market, including careless safety drivers who are supposed to take over test cars in case of emergencies.
The key stat underpinning the report is what’s known as “miles per intervention,” or the number of miles the car can drive on its own before the safety driver has to take over the car. The Times story cites internal documents showing Uber was unable to meet its goal of driving an average of 13 miles without a driver having to take back control as of March.
That’s not much better than its rate of intervention this time last year. In March 2017, documents Recode obtained showed Uber’s safety operators had to take back control of the cars an average of once every 0.8 miles.
Uber wanted to spell out what this metric means and doesn’t mean in the wake of the fatality and sent this statement:
MPI is not a measure of the overall safety of our testing operations, and shouldn’t be interpreted as such. Miles per intervention is one of many metrics that we use to track our system’s improvement, but without context it can be one of the least useful. For example, depending on where and how it’s tested, the same software could result in significantly different MPI. Additionally, companies may define interventions differently from each other.
In other words, miles per intervention is a broad metric that includes most of the times drivers have had to take back control from the system over the course of a week.
The reasons for these interventions can include navigating unclear lane markings, the system overshooting a turn or driving in inclement weather. The stat excludes accidental disengagements, end-of-route disengagements and early takeovers.
Have more information or any tips? Johana Bhuiyan is the senior transportation editor at Recode and can be reached at johana@recode.net or on Signal, Confide, WeChat or Telegram at 516-233-8877. You can also find her on Twitter at @JmBooyah.
Other metrics include the average number of miles between “critical” interventions — when a driver has to avoid causing harm, such as hitting pedestrians or causing material property damage, and the average number of autonomous miles between “bad experiences” — things like jerky motions or hard braking, which are more likely to cause discomfort than damage.
Then there’s the total number of miles driven autonomously. The documents the Times obtained indicated Uber had driven three million miles as of March.
While other companies may define interventions differently, Uber’s rate of intervention in Arizona is far more frequent than that of its competitors’. For example, Alphabet’s self-driving company, Waymo, had a rate of 5,600 miles per intervention in California.
Still, this is by no means an indication that the technology was at fault in the fatal crash. It is, however, an indication of Uber’s slow technological progress with its self-driving cars.
This article originally appeared on Recode.net.
