Many businesses have become over encumbered with data, leading to data silos and inaccurate data processing. This has changed the role of the data scientist over the last 10 years to a point where they now must focus on integration and improving data pipelines.
In 2012, Thomas H. Davenport and DJ Patil said that data science was the sexiest job field, due to the burgeoning industry growing around artificial intelligence, data analytics, and machine learning. A decade later, the pair returned in another Harvard Business Review post to discuss if data scientists are still on top, and what has happened to the industry in those ten years.
SEE ALSO: Supercharging the Data Flowing into Advanced Analytics
When data science was first posited as the “sexiest” job field, many large technology companies were only just coming to terms with what skills were necessary in the hiring process and what role data scientists would have in the company.
As Davenport and Patil confirm in the original article, many data scientists were pulled from roles outside of the normal triage of university degrees that tech employers seek. Those included PhD level experimental physicists, astronomers, and psychologists.
In today’s world, data science is an established field both in academia and in the workplace. Colleges and universities have many courses on data science, with specific programs available for AI, analytics, machine learning, and other sub-forms of data science.
Outside of traditional academic structures, there are hundreds of certificates, online courses, and even YouTube guides out there specifically for data science.
Even though it is established, data scientists still struggle inside businesses to make a difference with the data. Organizations by and large do not have data-driven cultures, and data scientists regularly spend more of their time “cleaning or wrangling data” instead of providing actionable insights to leadership.
“Being hired and paid well doesn’t mean that data scientists will be able to make a difference for their employers,” said Davenport and Patil. “As a result, many are frustrated, leading to high turnover.”
Speaking of pay, data science is one of the most in-demand fields and companies in Silicon Valley are regularly willing to pay above $200,000 starting salary.
The scope of what type of company would need a data scientist has changed massively over the past decade. What was once considered to be only for Bay Area startups and tech firms is now employed in almost every industry, reflective in the phrase “data is the new oil”.
The belief that a better data pipeline and analytics can enable businesses to have an edge against competitors has led hundreds of thousands of startups to try and compete with established players in almost every industry, while also forcing the established businesses to hire data scientists.
But as mentioned previously, just because data scientists can build the infrastructure doesn’t mean that all businesses take full advantage of it. In the past decade, many businesses have become over encumbered with data, leading to data silos and inaccurate data processing.
This has led to sub-sectors of data science, aimed at improving the pipeline for data and ensuring that analytics works for everyone in an organization. It has also led to business leaders and advisors calling for organizations to adopt a data-driven culture where justification comes from the data and not from gut decisions.
Will the data scientist still be on top in 2032? All indications say it will remain one of the most in-demand job titles for the next few years, and it doesn’t look like data will ever be less important to businesses.
