Amazon’s Vesta no-show highlights the challenges of home robots

You’d think Amazon would be the one to shake up a product category that has chewed up and spit out startups and corporate giants alike, but the company’s next flagship robot seemingly remains a work in progress. At a press event in downtown Seattle, some expected Amazon to preview a home robot that’s reportedly like a roving Echo Show, replete with wheels, microphones, and a display. But the announcement never came, and Amazon’s reticence might speak to the many challenges inherent to home robots — and indeed, robots at large.

Amazon’s robot — code-named Vesta, after the Roman goddess of the hearth  — apparently packs far-field microphones and speakers that enable it to understand and respond to the thousands of commands Alexa recognizes. It’s said to be able to navigate through homes using computer vision and techniques like simultaneous localization and mapping, and select Amazon employees are reportedly piloting it ahead of a launch as soon as this year.

An unforgiving market

Home robotics — and robotics generally — has proven a tough nut to crack for even the best-funded ventures.

In April, Anki — the San Francisco startup behind AI-imbued robotic toys like Overdrive, Cozmo, and Vector — closed its doors after raising close to $200 million in venture capital from Index Ventures, Two Sigma Ventures, J.P. Morgan, Andreessen Horowitz, and others. Anki claimed to have sold 6.5 million devices in total and 1.5 million robots last August alone, with revenue close to $100 million as of year-end 2017.

Anki’s demise followed that of Bosch-backed startup Mayfield Robotics, which was developing a larger, pricier ($700) home robot dubbed Kuri. Robotics company Jibo, which engineered a social robot featuring a bespoke conversational assistant, shut down earlier in the year. Honda canceled its Asimo program. And in a somewhat related development, industrial robotics company Rethink Robotics was recently forced to cease operations after attempting unsuccessfully to find an acquirer.

But that hasn’t stopped others from forging ahead. Temi, a startup headquartered in New York that’s developing a $1,500 telepresence robot with voice assistant integration, recently raised $21 million, in part from former Alibaba chief technology officer John Wu. Separately, wellness robots like Mabu and Diligent Robotics’ Moxi have found their way into hospitals, homes, and nursing centers, where they’re doubling as orderlies and symptom trackers for chronically ill patients.

Perhaps the best-known home robot success story is that of iRobot, which has sold more than 25 million units to customers around the world. Chief technology officer Chris Jones attributes the company’s success to its singular focus on housecleaning — and to perseverance in the face of logistical challenges.

“You have electrical, mechanical, software … and all that has to come together in a practical package that actually does something valuable, and getting those to work together efficiently and effectively is a challenge,” said Jones. He described the industry as an art rather than an exact science. “Every home is different — people interact with robots differently. It’s a tall order, and that’s why staying focused on practicality really matters.”

More human

Amazon’s home robot will have to overcome formidable barriers to success, chief among them a lack of emotional intelligence and customers’ sky-high expectations.

On this first point, Alexa AI chief scientist Rohit Prasad recently revealed that teams at Amazon are experimenting with systems to detect happiness, sadness, and anger from voice alone. The initial fruit of that labor, frustration detection, emerged this week.

Facial and object recognition are poised to play key roles in this as well, and Amazon has all the technical resources necessary to build a robust system. Its AWS DeepLens camera can run pretrained or custom AI models to perform sentiment analysis and detect a variety of activities, such as tooth brushing or guitar playing. AWS’ controversial Rekognition service can suss out sentiment and more. Amazon’s Echo Look taps computer vision to recognize clothes. And just this week, Amazon deployed an AI model to the Echo show that’s able to make out common pantry goods.

In a home robot, facial recognition could be used to record photos or videos around the house or enable the robot to greet kids when they return home from school. As for object detection, it might help personalize product recommendations and spot signs of a break-in, like Amazon’s Alexa Guard feature. Or it could work in tandem with services like Amazon Key to follow strangers around the room, paving the way for the remote installation of home furniture or appliances that can’t be simply dropped off on a doorstep.

Beyond the convenience of recognition features, emotional intelligence and contextual awareness can lead to interactions that feel more natural. Studies have shown that people are predisposed to name and even ascribe motivations to robots, which indicates that they will need to be not only perceptive but communicative as well.

Teams like those behind Mayfield’s Kuri and Anki’s Vector have laid the cornerstones for paradigms of emotional expression. Kuri responded to nearly every turn in a conversation with animated expressions, including a confused “huh?” emoji if it didn’t understand something or a “got it” following a command. And Vector, a tiny handheld robot with dual treads and an articulated “head,” conveys feelings of nervousness, joy, panic, annoyance, excitement, and more with animations and sound effects.

“We explored putting third-party interfaces into robots and found that having to say a hotword [like ‘Alexa’ or ‘hey, Google’] felt awkward and mechanical,” Anki’s Mark Palatucci told VentureBeat in a previous interview. “We wanted [Vector] to feel more personal — more emotional.”

Amazon’s robot would do well to follow their leads, perhaps with expressions, animations, or sound — and with music and activity recommendations tailored to users’ habits and sentiment. It’s a future Amazon inched toward with Alexa Hunches, which proactively recommends actions based on data from connected devices and sensors, and with a feature that takes into account the proximity of devices when Alexa responds to commands like “Alexa, turn on the lights.”

Function over form

Whatever form Alexa’s robot takes, its size and appearance will be key to mitigating the preconception problem. As Palatucci explained, there’s an uncanny valley in robotics: People expect larger and more human-like robots.

Aeolus is a prime example. The janitorial robot can identify objects, clean the floor with any off-the-shelf vacuum, and grasp drinks and other objects. The only problem? It moves at a snail’s pace. In a demo at CES last January, Aeolus took a full minute to pick up a stuffed animal and put it in a nearby bin.

Optimists like Misty Robotics CEO Tim Enwall firmly believe every home will have a highly capable robot within 20 years, while more cautious observers like Carnegie Mellon University professor of robotics Henny Admoni expect it’ll be five to 10 years before mass-produced robots can pick up after kids, tidy furniture, prep meals, and complete other domestic chores. As for folks like Jones and iRobot CEO Colin Angle, they predict that a family of machines — rather than a single robot — will work together to perform chores like folding clothes, washing the dishes, and assisting older or disabled family members.

“The home can handle several different types of robot. You’re going to be able to buy them incrementally, each specialized to do a purpose really well, and there’s going to be some things where combining functionality into one robot makes sense,” explained Angle.

That’s to say Amazon’s first home robot probably won’t fulfill the promises of a Jetsons future, and it will likely be the first of many models and designs to come. Of course, Amazon is no stranger to playing the long game. Just this week, it announced Sidewalk, a wireless internet of things (IoT) protocol with which it hopes to supplant standards that have had more than a decade head start. And it’s an open secret that Amazon lost a good chunk of change on early Fire tablet sales, which the company has long since recouped through sales of ebooks, Audible subscriptions, Amazon Prime Video rentals, and more.

When Bosch announced Mayfield’s shuttering last year, the company said it couldn’t find a “fit” to “support and scale” Kuri, a sentiment that resonated with Admoni. “I think [these companies] didn’t find a compelling use case,” he candidly told the Financial Times last year. Jones is of a similar mind — in an interview with VentureBeat at Amazon’s re:MARS conference last summer, he said robots that fail to catch on are those that fall short of marketing promises.

If Amazon doesn’t play its cards right, it risks repeating history.