Researchers are developing better, faster ways of providing human guidance to autonomous robots.
Told to optimize for speed while racing down a track in a computer game, a car pushes the pedal to the metal … and proceeds to spin in a tight little circle. Nothing in the instructions told the car to drive straight, and so it improvised.
This example – funny in a computer game but not so much in life – is among those that motivated Stanford University researchers to build a better way to set goals for autonomous systems.
Dorsa Sadigh, assistant professor of computer science and of electrical engineering, and her lab have combined two different ways of setting goals for robots into a single process, which performed better than either of its parts alone in both simulations and real-world experiments. The researchers presented the work June 24 at the Robotics: Science and Systems conference.
“In the future, I fully expect there to be more autonomous systems in the world and they are going to need some concept of what is good and what is bad,” said Andy Palan, graduate student in computer science and co-lead author of the paper. “It’s crucial, if we want to deploy these autonomous systems in the future, that we get that right.”
The team’s new system for providing instruction to robots – known as reward functions – combines demonstrations, in which humans show the robot what to do, and user preference surveys, in which people answer questions about how they want the robot to behave.
“Demonstrations are informative but they can be noisy. On the other hand, preferences provide, at most, one bit of information, but are way more accurate,” said Sadigh. “Our goal is to get the best of both worlds, and combine data coming from both of these sources more intelligently to better learn about humans’ preferred reward function.”
Demonstrations and surveys
In previous work, Sadigh had focused on preference surveys alone. These ask people to compare scenarios, such as two trajectories for an autonomous car. This method is efficient, but could take as much as three minutes to generate the next question, which is still slow for creating instructions for complex systems like a car.
To speed that up, the group later developed a way of producing multiple questions at once, which could be answered in quick succession by one person or distributed among several people. This update sped the process 15 to 50 times compared to producing questions one-by-one.
The new combination system begins with a person demonstrating a behavior to the robot. That can give autonomous robots a lot of information, but the robot often struggles to determine what parts of the demonstration are important. People also don’t always want a robot to behave just like the human that trained it.
“We can’t always give demonstrations, and even when we can, we often can’t rely on the information people give,” said Erdem Biyik, a graduate student in electrical engineering who led the work developing the multiple-question surveys. “For example, previous studies have shown people want autonomous cars to drive less aggressively than they do themselves.”
That’s where the surveys come in, giving the robot a way of asking, for example, whether the user prefers it move its arm low to the ground or up toward the ceiling. For this study, the group used the slower single question method, but they plan to integrate multiple-question surveys in later work.
In tests, the team found that combining demonstrations and surveys was faster than just specifying preferences and, when compared with demonstrations alone, about 80 percent of people preferred how the robot behaved when trained with the combined system.
“This is a step in better understanding what people want or expect from a robot,” said Sadigh. “Our work is making it easier and more efficient for humans to interact and teach robots, and I am excited about taking this work further, particularly in studying how robots and humans might learn from each other.”
Better, faster, smarter
People who used the combined method reported difficulty understanding what the system was getting at with some of its questions, which sometimes asked them to select between two scenarios that seemed the same or seemed irrelevant to the task – a common problem in preference-based learning. The researchers are hoping to address this shortcoming with easier surveys that also work more quickly.
“Looking to the future, it’s not 100 percent obvious to me what the right way to make reward functions is, but realistically you’re going to have some sort of combination that can address complex situations with human input,” said Palan. “Being able to design reward functions for autonomous systems is a big, important problem that hasn’t received quite the attention in academia as it deserves.”
The team is also interested in a variation on their system, which would allow people to simultaneously create reward functions for different scenarios. For example, a person may want their car to drive more conservatively in slow traffic and more aggressively when traffic is light.
The Latest on: Autonomous systems
via Google News
The Latest on: Autonomous systems
- Visiongain Report Offers Transformative Insights on the USD 29 Billion Autonomous Weapons Marketon March 20, 2020 at 5:15 pm
by Geographical Region (North America, Europe, Asia-Pacific, and Rest of World). Plus, Analysis of the Leading Players in the Market Space • DoD awards BAE Systems $2.7bn APKWS contract • USAF tests ...
- FluSense system tracks sickness trends by autonomously monitoring public spaceson March 20, 2020 at 12:34 pm
FluSense is an autonomous, privacy-respecting system that counts the people and coughs in public spaces to keep health authorities informed. Every year has a flu and cold season, of course, though ...
- Volvo India recalls 1891 cars over faulty autonomous emergency braking systemon March 20, 2020 at 7:33 am
The faulty system in the affected cars will not brake automatically, though it will warn the driver of an impending collision.
- Why this Bay Area autonomous freight trucking company is closingon March 20, 2020 at 6:52 am
The San Francisco autonomous trucking company saw its funding dry up, co-founder and CEO Stefan Seltz-Axmachersaid Thursday in a blog post. He not only outlined the demise of his startup but also ...
- Autonomous driving still isn’t a reality in 2020 — and it’s getting people killedon March 20, 2020 at 5:54 am
Teslas aren’t even close to being fully autonomous vehicles. Self-driving cars do not exist Speaking about the incident, NTSB Chairman Robert Sumwalt said: “The car involved in this crash was not a ...
- 2020 Innovations in Robot Vacuums, 3D Printing, Autonomous Vehicles, and Exoskeletons - ResearchAndMarkets.comon March 20, 2020 at 3:52 am
Innovations include automated order fulfillment for E-commerce, large area thermoset additive manufacturing, robot vacuum cleaner, hovering autonomous underwater vehicle, exoskeleton for operator ...
- AXA XL Launches Insurance Policy for Autonomous Vehicleson March 20, 2020 at 3:51 am
a technology company with a focus on mobile robotics and autonomous systems, since 2016. In addition, AXA XL in 2017 joined the UK government-backed DRIVEN consortium to contribute to the deployment ...
- Global Automotive Autonomous Emergency Braking Systems Market 2019 Industry Segmentation, CAGR Status, Leading Trends, and Forecast To 2025on March 19, 2020 at 1:06 pm
The report provides rational insights along with historical and forecast data to said in better understanding of the Global Automotive Autonomous Emergency Braking Systems Market. The report provides ...
- Ford releases a data set to accelerate autonomous car developmenton March 19, 2020 at 8:36 am
Ford notes that each log in the Ford Autonomous Vehicle Dataset is time-stamped and contains raw data from the sensors, calibration values, pose trajectory, ground truth pose, and 3D maps. It’s ...
- Global Autonomous/Driverless Car Market Projections, 2020-2025: World Market Anticipating a CAGR of ~18%on March 18, 2020 at 5:26 pm
Dublin, March 18, 2020 -- The "Autonomous/Driverless Car Market- Growth, Trends, and Forecast (2020 - 2025)" report has been added to ResearchAndMarkets.com's.
via Bing News