The promise of Amazon’s new futuristic convenience store is no line, no wait and, frankly, little to no human interaction. But for now, the machines still need some help from our species.
Amazon’s store of the future has no cashiers, but humans are watching from behind the scenes
Machines in the front, hominids in the back.
Jason Del Rey has been a business journalist for 15 years and has covered Amazon, Walmart, and the e-commerce industry for the last decade. He was a senior correspondent at Vox.
Recode has learned that Amazon has staff on call behind the scenes to assist the computer vision system that is supposed to detect which items a shopper pulls off a shelf and carries out of the store.
An Amazon spokesperson confirmed the setup and said that Amazon staff is asked to help out when the system used in the new Amazon Go store can’t make a determination.
“[W]hen the machine vision system isn’t sure, it asks a human for confirmation,” she said. “It happens a small fraction of the time.”
The spokesperson said that the review work is being done “in house” and not through Mechanical Turk, the Amazon-owned website that pays out pennies per item for tasks similar to this one. She did not immediately provide answers to several follow-up questions, including queries about the frequency of the human intervention, who exactly is doing the work and whether they are reviewing still images, video footage or something else.
Amazon announced its first Amazon Go store last month in Seattle, though it is currently only open to company employees. The store’s technology allows customers to take items like sandwiches and drinks from shelves and leave without checking out; items are charged to the payment card a customer has on file with Amazon. The company has dubbed its system “Just Walk Out Technology.”
While Amazon wouldn’t describe it this way, customers are essentially being tracked by cameras inside, which help identify which products are removed and by whom. An Amazon patent application unearthed by Recode in 2015 suggested that such a store may also include microphones to locate someone and track their movements. (Update: A spokeswoman said the store does not contain microphones.) The company has said that sensors also play a role.
It is not unusual for humans to help train computer vision algorithms, and the Amazon spokeswoman said fewer manual reviews will be necessary over time as the technology gets smarter.
But knowing there are people working behind the scenes does makes Amazon’s announcement video seem a little less picture-perfect.
This article originally appeared on Recode.net.
