2019 Trends: Interview on the Machine Learning and Artificial Intelligence Market

Date: February 27th, 2019

This article is from an interview I recently gave to Mike Barlow for an Oreilly report to be released on their website (One of Mike's previous similar reports here). Usually these answers are broken up into quotes in the course of the overall article Mike will produce, but we thought it fit well on our blog as a series of comments on current market trends as well. Re-posted with Mike's permission.

Question: "What’s the best way of explaining what AI is and what AI isn’t?"

What AI is not

AI is not alive, self-aware, nor near any level of “general artificial intelligence” that keeps getting marketed today. We’ve seen these cycles before (2 times, previously), where AI gets over-marketed and then crashes through results, while impressive and drive new industry sectors, always fall short of “magical”.

Why do we keep doing this to ourselves?

AI has long held a place in society’s collective imagination because it poses an existential threat to our fundamental role in society. People relate to their place in the society by the role they take on in the working world, whether that is clergy, engineer, or philosopher. If we are all automated into irrelevance, then those roles and relations disappear and this becomes an existential issues for most people. Fundamentally, we keep running through this hype cycle due to fear of this scenario.

What is AI?

Artificial Intelligence today in real terms is “applied machine learning”. This definition can be expanded further, from the automation viewpoint, to “artificial intelligence is a term for algorithms that increase user productivity”. We can make this designation because some techniques, such as game-state search, are not qualified as machine learning yet fall under the historical banner of AI.

Folks who over-market machine learning to be “general artificial intelligence” do the entire computer science industry a disservice. Machine learning is (largely, in practice) classification and regression and in no way matches up to ephemeral aspirations of an all-knowing, self-aware system that can help the reader with their marketing problem.

For an extended take on this topic, check out “Appendix A: What is Artificial Intelligence?” in our book from Oreilly “Deep Learning: A Practitioner’s Approach”

Question: "What’s the relationship between data science and AI?"

Data science involves the practice of applied machine learning.

The term “Artificial intelligence” is how we poorly market “data science practicing applied machine learning”.

Question: "Will AI eliminate all jobs or just some of them? Which ones will it eliminate?"

This is a poor narrative that has been propagated too many times.

If we agree that AI is not going to be “general artificial intelligence” any time soon, and that we’re really talking about “applied machine learning” in most cases, then the engineer in us thinks its simply an issue of how well we can build and apply models to real world problems.

The problem is, the world is not that simple.

Engineers do well when code does not have to enter the real world, because when its “just a webserver”, they can always just “reset the server” when things go bad. There are no regulations or oversight or major laws to consider in this case.

But in the case of everyone’s favorite automation topic, self-driving cars, we begin to hit a litany of real-world issues that engineers don’t hit in pure-software land. If our models are even slightly imperfect, “1% Edge cases” become “we only hit 3 mailboxes out of 300 this morning on the way to the office”. Other issues include regulatory, legal, and then just how fast society will accept the risks posed by a major automation change.

So change becomes slower, and productivity enhancers like “lane assist” slowly become integrated as “automation”. And new features take a lot longer to “get right” and get integrated in a way that federal law will bless.

Most automation will occur slowly over the coming decades and society will adapt with it, just like it has since the time of the automated loom. In many cases, given that the red queen demands we all “run as fast as we can to stay in the same place”, we will continue to work in many of the same jobs as they evolve, yet will welcome ways to be more efficient and continue to compete in our respective markets. Jobs will continue to be created and some jobs will become extinct, as is the natural cycle over time.

Question: "How does a software developer become an AI developer?"

Through the natural evolution of adding job skills. The computer science industry looks favorably on those candidates who continue to add new techniques to their toolbelt. Engineers do not have to get a phd in math or machine learning (a common fallacy) to do quality data science work. They just need to find good fundamentals in the areas of:

Data collection
Data cleaning and extract, transform, and load (ETL) methods
Applying modeling patterns from similar problems to new problems
Model deployment and integration
Model monitoring and evaluation

Question: "Please name five skills you need to become a data scientist"

The python ecosystem has signficant gravity these days, so I'd suggest:

Python, Pandas, Numpy, and Jupyter Notebooks
Spark and Extract, Transform, Load (ETL) skills
Learning basic data vectorization and feature engineering
Model deployment, integration, and management
Understanding how to triage use cases

I'd also add that data scientists need to focus on getting a result for their line of business, and less on their favorite langauge or toolkit.

Question: "What do you personally find most frustrating about AI?"

The propensity of technology companies to take any new technology, such as “AI”, and make marketing statements that would make Theranos executives blush.

To continue this discussion, reach out on twitter or contact us.