The Bait and Switch of Data Science

Every morning, I read almost a dozen articles. A few articles usually relate to data science (I hate the term “data science”, as it’s very overloaded. Here, I’ll confine data science to AI or machine learning). Lately, I notice more and more articles echoing something I’ve been saying for years – data science is hard, and a lot of companies will find this out very soon.

This MIT Technology Review article from yesterday – “This Is Why AI Has Yet To Reshape Most Businesses” – encapsulates a lot of the cold reality businesses are seeing when they delve into “doing data science”.

Many companies are finding out that data science is bait and switch. Yes – data science will potentially yield a ton of value for businesses. No – simply hiring data scientists and arming them with Python notebooks will do very little.

I feel bad for newly minted data scientists. A lot of fresh data scientists get a reality check the hard way. They show up on the job, expecting to geek out on awesome algorithms. Instead, many end up doing the the equivalent of manual labor – data engineering, backend coding, data wrangling/cleaning/etc. “Why am I not applying my new Masters Of Data Science degree?! You promised me I would be building Skynet, no doing lowly data work!” lament these fresh graduates.

Despite the insane hype around data science, I think people are finally starting to understand the reality of the challenges to making it work in the real world. As I mentioned, there’s the manual labor of data. Then there’s the bigger issue – some companies are simply not data driven, nor genetically capable of using data to any degree. My litmus test – if a company has intractable organizational silos, this is a good indication that it can’t succeed with data, let alone data science. Conway’s Law at work and all.

The hard truth is that data science still needs data upon which do the science. And right now, success with data still requires a lot of good old blue collar work.

Comments are closed, but trackbacks and pingbacks are open.