Product Development for Data Science - From Idea to Production in 7 Phases
Towards the end of 2020, Melissa Perotti and I held a talk on Product Development for Data Science Products at GoDataFest - the (this time virtual) festival of Data Technology. We elaborate on how we follow a 7-phase process to take Data Science projects from idea to prototype and from prototype to production.
About the talk
In every industry, a remarkably large share of Data Science (DS) projects fail. There are various reasons why DS projects may come to a halt:
- Data quality or size is insufficient,
- the model doesn’t perform as expected,
- or it performs excellently but its predictions don’t convert to business value.
It’s important to recognise that failure is a natural co-occurrence of doing Data Science. After all, it includes the term ‘science’. Science, by definition, describes the process of generating knowledge through research - i.e. exploring the unknown. Consequently, uncertainty and failure are inherent in doing (Data) Science. There is only one way to really find out if your endeavour will succeed: by putting work into it. Too bad if you have spent months developing a DS solution that eventually doesn’t deliver the expected results, right?
Adapting your development process is key to accounting for the uncertainty inherent in Data Science and minimising its risk. That’s why we at the Customer Happiness DataLab structure our Data Science projects in seven phases that guide you all the way from an initial idea to a production-ready product. It enables us to iterate quickly, fail fast, and spend our time as efficiently as possible.
Learn more about it in the recording of our talk on Product Development for Data Science Products at GoDataFest!