From research to applications

Having discussed how research is done in AI, it’s now time to focus on applications. Assuming you already have a data science team in place and preliminary research on a problem you want to solve done, the next step is to gather and clean data. This process can be short if most of your business is digital with easy access to data, or long and painful if you have many sources to look at and data is far from clean (say, surveys of customers done in various formats). If that’s the case, preprocessing is a task that would need a separate team to complete. It’s especially essential for all the later work, so don’t ignore cleaning data.

Applying research to business applications means using machine learning models on data coming from your business and measuring how well they behave compared to how you usually solve the problem at hand (e.g. time spent on a business process, marketing/sales, number of relevant leads). After receiving data, your team of machine learning engineers will put metrics in place to measure progress and start implementing (coding) machine learning algorithms, filling them with data you have provided. The end result will be either a metric of the accuracy of prediction, automated business process, or optimized computations.

What’s crucial in the implementation stage is having good metrics to compare models (architecture), machines (infrastructure), and data used in training and evaluation. Understanding why the results are what they are is as important as arriving at the model yielding the best results.

Scaling the solution to more data and using models in production often requires more engineering than machine learning talent and thus requires hiring a different talent pool than in the research phase. However, as implementation and research are tightly knit, the best is having on board both researchers and engineers to work closely together.

Summing up, practical AI in business is done through a cycle of:

1. researching possible machine learning models;
2. gathering data;
3. using models on collected data;
4. improving upon infrastructure, data, and scale.

After step 4, we come again to step 1, having feedback on how models work in practice and what can be improved, researchers can look for new algorithms and methods to tackle the problem at hand. The whole cycle is repeated until metrics defined at the start of the process are satisfied. That’s why to finish the project successfully it’s crucial to build a framework for machine learning experiments at the very beginning, even if it is to change later on with new experiments.