Over the years, software companies worldwide have worked hard to hone practices that smoothen up their software development. From Production to Deploying to Monitoring, These practices have become known as DevOps. Now you may ask, what is the need for MLOps when such software development methods exist?
In addition to the software team, machine learning ventures introduce a whole new breed of coders called data scientists. These developers tend to present a completely new big and messy piece of code, their own niche machine learning models which defy the previously set bottlenecks and DevOps practices.
MLOps is a field emerging to unify launch cycles of Data Scientists and Software Developers. We here at NimbleBox.ai would like to take readers on a journey to uncover and understand such practices involved in MLOps, that we think are ought to help while thinking from an ML perspective.
Data Pipeline Orchestration
Location, location, location is the game's name for real estate. On the other hand, machine learning is all about data, excellent data, and even better data! Machine learning algorithms rely on data for training and what you put is what you get (WYPIWYG).
A promising data pipeline ensures a unified source of ground truth, which can be viewed through dashboards, enabling authors to get insights into various parameters that test your model's efficacy in the wild.
Let us look at some steps involved in an ideal Data Pipeline:
For DevOps teams: Data lakes should be created beforehand that are easy-to-understand and handy so that no unwarranted time is wasted over the pre-existing bottlenecks introduced by limited processing.
For DevOps and data engineering teams: Building efficient ingestion pipelines that ideally unfasten any need of Data Engineer and DevOps to interact over every small iteration of the product.
For Data Engineering teams: An automated system for this acquired data to be transformed into something your ML Model can consume.
Let us now look at some of the critical tips to be followed while implementing these steps.
Data Lakes should always be created to be queried later, rather than storing data (or dumping JSON).
Having good Data Visualization Pipelines (you can use versions of Jupyter Notebook or Grafana Dashboards)
Data Engineers should have efficient knowledge about the pipelines to perform independently of the DevOps team.
Model Development
What is a premium location without a beautiful piece of architecture standing on top of it? Or, in the language of CSVs and JSONs, what is a well-annotated clean piece of data if a Model doesn’t adopt it and produce results?
However, the traditional methods of Perceptrons and Neocognitrons are not something that will make your Tesla work. Bigger data means more features to explore which in turn means a deeper understanding of the information, which brings us to Deep Learning. You may wonder, classical Machine Learning has been around for 70+ years, so why suddenly go for Deep Learning, let us look at why?
Let us continue this trend of easy to compare specifics for your ML Model Development and decide what way should the actual code be developed according to the requirements:
Model Deployment
Can you think of the number of projects and ideas that you came up with during your ML Journey, that you thought will take the world by storm? What didn’t allow you to do so, ever thought about that?
Deployment, the action of bringing resources into effective action, is a process where most ideas and concepts smother out. Stemming from the fact that the contrast between perfect lab conditions and the real raw world is far and wide.
Machine Learning models with their overwhelming amount of data and the astounding amount of computing demands some special types of deployment techniques than normal software, let us take a look at some of them:
Kubernetes Based: the process of deploying the project on a Kubernetes Platform which provides an open-source orchestration platform that enables easier rollbacks and coordinates clusters of nodes at an efficient rate.
Docker Swarm Based: the process of deploying the model through the docker swarm which serves as a container orchestration tool, meaning that it allows the user to manage multiple containers deployed across multiple host machines.
Monitoring
Deploying a machine learning model to the real world suddenly exposes it to technical, physical, and computational data and work. This, in turn, gives birth to the need to have a systematic pipeline in the bank to monitor and course correct the model parameters for smooth deployment.
In the era of microservices, all software asks for multiple API endpoints aided by intricate Kubernetes tasks that can communicate a deployed software's current state. This, however, takes a rather difficult turn when implemented on Machine Learning models. This is because ML Applications tend not only to want to serve requests but also to store and analyze them, which requires a large team to build and maintain.
Let us look at some of the elements that are essential for a good deployment:
Monitoring: Having the ability to watch and change the model's behavior on the go, with switches in place to revert to an older stable version.
A/B Testing: It refers to a randomized monitoring process wherein two or more different versions of the model are shown to various segments of the users. A/B testing is also known as split testing. A/B testing
CI/CD (With human-in-loop): The ability to test models on live samples before putting them into production, giving them an upper edge by their actions and reactions getting human validation.
You can extend the above list, but the core objective behind having such methods is to capture better data for the parameters of the ML Model and fix it over time. This is greatly aided by a speedy deployment which is essential for back-testing.
Back-Testing: a general method for seeing how well a strategy or model would have done post-deployment
Monitoring Tools
The previous section focused on the necessity of monitoring and the things to be kept in mind while monitoring a deployed machine learning model. In this section, let us look at some basic tools that you can use to monitor your deployment.
Let us first take a look at some of the parameters that you may want to keep an eye on while monitoring:
These parameters require intricate monitoring tools that have been iterated to meet the requirements over the years. So let us have a closer inspection at some of these tools and services.
1. Kubeflow:
It is an Open-Sourced ML Toolkit for Kubernetes (something touched upon in the last section). Working towards maintaining ML Applications by managing and packaging Docker Containers.
2. Neptune:
An ML metadata store to log, store, display, organize, compare and query all metadata generated during the ML model lifecycle.
3. Grafana:
An Open-Sourced multi-platform analytics and visualization web application, equipped with modern visualization techniques for efficient model monitoring.
4. mlflow:
An Open-Sourced platform to cater to the whole Machine Learning lifecycle that includes but isn't limited to experimentation, reproducibility, deployment, and a central model registry.
Written By
Aryan Kargwal
Data Evangelist