Is it a computer program or a living being? At TU Wien (Vienna), the boundaries become blurred. The neural system of a nematode was translated into computer code – and then the virtual worm was taught amazing tricks.
It is not much to look at: the nematode C. elegans is about one millimetre in length and is a very simple organism. But for science, it is extremely interesting. C. elegans is the only living being whose neural system has been analysed completely. It can be drawn as a circuit diagram or reproduced by computer software, so that the neural activity of the worm is simulated by a computer program.
Such an artificial C. elegans has now been trained at TU Wien (Vienna) to perform a remarkable trick: The computer worm has learned to balance a pole at the tip of its tail.
The Worm’s Reflexive behaviour as Computer Code
C. elegans has to get by with only 300 neurons. But they are enough to make sure that the worm can find its way, eat bacteria and react to certain external stimuli. It can, for example, react to a touch on its body. A reflexive response is triggered and the worm squirms away.
This behaviour can be perfectly explained: it is determined by the worm’s nerve cells and the strength of the connections between them. When this simple reflex-network is recreated on a computer, then the simulated worm reacts in exactly the same way to a virtual stimulation – not because anybody programmed it to do so, but because this kind of behaviour is hard-wired in its neural network.
“This reflexive response of such a neural circuit, is very similar to the reaction of a control agent balancing a pole”, says Ramin Hasani (Institute of Computer Engineering, TU Wien). This is a typical control problem which can be solved quite well by standard controllers: a pole is fixed on its lower end on a moving object, and it is supposed to stay in a vertical position. Whenever it starts tilting, the lower end has to move slightly to keep the pole from tipping over. Much like the worm has to change its direction whenever it is stimulated by a touch, the pole must be moved whenever it tilts.
Mathias Lechner, Radu Grosu and Ramin Hasani wanted to find out, whether the neural system of C. elegans, uploaded to a computer, could solve this problem – without adding any nerve cells, just by tuning the strength of the synaptic connections. This basic idea (tuning the connections between nerve cells) is also the characteristic feature of any natural learning process.
A Program without a Programmer
“With the help of reinforcement learning, a method also known as ‘learning based on experiment and reward’, the artificial reflex network was trained and optimized on the computer”, Mathias Lechner explains. And indeed, the team succeeded in teaching the virtual nerve system to balance a pole. “The result is a controller, which can solve a standard technology problem – stabilizing a pole, balanced on its tip. But no human being has written even one line of code for this controller, it just emerged by training a biological nerve system”, says Radu Grosu.
The team is going to explore the capabilities of such control-circuits further. The project raises the question, whether there is a fundamental difference between living nerve systems and computer code. Is machine learning and the activity of our brain the same on a fundamental level? At least we can be pretty sure that the simple nematode C. elegans does not care whether it lives as a worm in the ground or as a virtual worm on a computer hard drive.
The Latest on: Reinforcement learning
- How NLP, ML and Deep Learning Can Transform Your CX Strategy on November 11, 2018 at 5:26 am
Reinforcement Learning: The algorithm (also called the agent) interacts with the environment. That is, it observes the state of the environment, performs an action and then receives a reward/penalty. ... […]
- RL— Introduction to Deep Reinforcement Learning on November 9, 2018 at 3:50 am
Deep reinforcement learning is about taking the best actions from what we see and hear. Unfortunately, reinforcement learning RL has a high barrier in learning the concepts and the lingos. […]
- OpenAI launches reinforcement learning training to prepare for artificial general intelligence on November 8, 2018 at 11:40 am
OpenAI today announced the launch of Spinning Up, a program designed to teach anyone deep reinforcement learning. OpenAI is well known for making funky-looking agents in virtual environments that ... […]
- Facebook open sources Reinforcement Learning (RL) software on November 5, 2018 at 2:15 am
The latest trends in software development from the Computer Weekly Application Developer Network. Everybody’s favourite social media platform company Facebook has — despite the US supreme court apolog... […]
- Why data is the new oil: What we mean when we talk about “deep learning” on November 4, 2018 at 2:00 am
The basal ganglia of the brain, which receive projections from the entire cerebral cortex and project back to it, solve this problem with a temporal difference algorithm and reinforcement learning. Al... […]
- Using machine learning to teach robots to get dressed on November 3, 2018 at 1:35 am
In the Siggraph 2018 paper Learning to Dress: Synthesizing Human Dressing Motion via Deep Reinforcement Learning, a Georgia Institute of Technology/Google Brain research team describe how they ... […]
- Facebook launches Horizon to bring reinforcement learning to production environments on November 2, 2018 at 10:27 am
Facebook is trying to bridge the gap between reinforcement learning’s impact in research and its narrow range of use cases in production. The company’s new open-source project Horizon uses ... […]
- NVIDIA Is Using Reinforcement Learning Algorithms to Train A.I. Robots Simultaneously on November 2, 2018 at 6:23 am
A team of researchers at NVIDIA have developed a GPU-accelerated reinforcement learning simulator using just one NVIDIA Tesla V100 GPU and a CPU core. The research team was able to train the AI to com... […]
- Deep reinforcement learning – trick or treat? on November 2, 2018 at 3:09 am
Deep Reinforcement Learning (Deep RL) is the new buzzword in the machine learning world. Deep RL is an approach which combines reinforcement learning and deep learning in order to achieve human-level ... […]
- Facebook open sources its reinforcement learning toolkit Horizon on November 1, 2018 at 10:34 pm
Facebook Inc. has open-sourced another important piece of artificial intelligence software designed to help developers build “reinforcement learning models” into their software applications. […]
via Google News and Bing News