Deployment of Machine learning Model Demystified (Part 2)

source: https://pixers.us/wall-murals/rocket-in-space-63044962

This is the second part in a two-part series. you should read the first part first

The first goal here is to convert the notebook into production-ready code which is nothing but writing pythonic scripts and modules.

jupyter notebooks to scripts and modules

Notebooks can be written in jupyter lab, Jupyter notebook, Google Colab etc in which one can also write scripts as well but I have chosen VS-Code as my favourite IDE which is what I will be using for this purpose. Whichever IDE you are using does not really matter.

As part of the setup required when it comes to writing production code, it is a good standard practice to always have a different virtual environment for each project to avoid dependencies issues. This can be created and activated in the command line using the commands below haven navigated to your project root folder:

python -m venv name_of_your_virtual_environment 
#activate the environment
name_of_your_virtual_environment\Scripts\activate
#all project dependencies can now be installed

The project dependencies can be installed with a single command when listed in a requirement.txt file with the example below

#Begining of the requirement file example
numpy==1.17.2
scikit-learn==0.21.3
pip==19.2.3
pandas==0.25.1
packaging
setuptools==40.6.3
wheel==0.32.3
# testing requirements
pytest==5.3.1
# fetching datasets from kaggle
kaggle==1.5.1.1
#End of the requirement file example
pip install -r requirements.txt

The next question is which part of the machine learning pipeline goes into the production code?

Machine Learning Pipeline in Production [1]

Only the circled parts of the pipeline need to be converted into production code. There are 3 major ways to write deployment code for ML which are listed below.

• Third-Party Pipeline Code: This involves the use of OOP and instances are run using a third-party pipeline such as the sklearn pipeline.