The Defense Advanced Research Projects Agency (DARPA) announced on Friday the launch of Data-Driven Discovery of Models (D3M), which aim to help non-experts bridge what it calls the “data-science expertise gap” by allowing artificial assistants to help people with machine learning. DARPA calls it a “virtual data scientist” assistant.
This software is doubly important because there’s a lack of data scientists right now and a greater demand than ever for more data-driven solutions. DARPA says experts project 2016 deficits of 140,000 to 190,000 data scientists worldwide, and increasing shortfalls in coming years.
For example, in order to construct a model for how different weather, school, location, and crime factors affect congestion for ride-sharing services in downtown Manhattan, a team of NYU students spent the equivalent of more than 90 months of work hours to complete the model. DARPA sees problems just like this all the time and the D3M Program will strive to construct it to drastically reduce the time and expertise needed to make models like these in the future.
“The construction of empirical models today is largely a manual process, requiring data experts to translate stochastic elements, such as weather and traffic, into models that engineers and scientists can then ask questions of,” said Wade Shen, program manager in DARPA’s Information Innovation Office. “We believe it’s possible to automate certain aspects of data science, and specifically to have machines learn from prior example how to construct new models.”