Get your model into prod

Deploying your modelling code into production with Microsoft Azure

Dean Marchiori

Wave Data Labs

Go from this…

To this…

Part 1: Overview of Machine Learning Operations (MLOps)

What is MLOps?

Source: Kreuzberger (2023)

What is MLOps?

CI/CD Automation
Workflow Orchestration
Reproducibility
Versioning of data, model, code
ML metadata tracking
Monitoring & Feedback

Source: Kreuzberger (2023)

Who is MLOps?

Source: Kreuzberger (2023)

Who is MLOps?

Experimentation vs Productionisation (Inner Loop vs Outer Loop)

Business Understanding
Data Understanding
Data Preparation
Modelling
Evaluation
Deployment

Source: CRISP-DM

Kenneth Jensen, CC BY-SA 3.0, via Wikimedia Commons

Outer Loop
Infrastructure Deployment
- Inner Loop
  - Business Understanding
  - Data Understanding
  - Data Preparation
  - Modelling
  - Evaluation
Model Registration and Deployment
Monitoring

Reference Architecture

Deployment patterns

graph TD;
    MLArchitecture-->Classic;
    MLArchitecture-->ComputerVision;
    MLArchitecture-->NLP;
    Infra-Deployment-->Bicep;
    Infra-Deployment-->Terraform;
    Infra-Deployment-->CLI;

graph TD;
    Orchestration-->AzureDevOps;
    Orchestration-->Github;
    MlOpsCode-->Python-SDK;
    MlOpsCode-->Azure-CLI-v2;

Azure ML Studio Tour

Part 2: Converting development code into production quality code

Case Study: Predict NYC Taxi Fares

Not really important, just a quick example.

RandomForestRegressor(n_estimators = 500,
                      bootstrap = 1,
                      max_depth = 10,
                      max_features = 1.0,
                      min_samples_leaf = 4,
                      min_samples_split = 5,
                      random_state=0)

TARGET_COL = "cost"

NUMERIC_COLS = [
    "distance",
    "dropoff_latitude",
    "dropoff_longitude",
    "passengers",
    "pickup_latitude",
    "pickup_longitude",
    "pickup_weekday",
    "pickup_month",
    "pickup_monthday",
    "pickup_hour",
    "pickup_minute",
    "pickup_second",
    "dropoff_weekday",
    "dropoff_month",
    "dropoff_monthday",
    "dropoff_hour",
    "dropoff_minute",
    "dropoff_second",
]

CAT_NOM_COLS = [
    "store_forward",
    "vendor",
]

Review experimentation code

graph TB
    subgraph id1 [Monolithic Notebook]
    a1[Load Dependencies]-->a2[Data Prep]-->a3[Test/Train Split]-->a4[Train Model]-->a5[Evaluate Model]-->a6[Diagnostics]
    end
    a7[(Data)]-->a2
    style id1 fill:lightblue;

Split into scripts

graph LR
  a1[[Data Prep]]-->a2[[Model Training]]-->a3[[Evaluation]]-->a4[[Unit Tests]]

Read in required inputs
Export object for next step
Parameterise scripts with argparse
Add logging using mlflow
Bundling code as functions

MLOps additions

graph TB
  b1[[Infrastructure Deploy]]
  a1[[Data Prep]]-->a2[[Model Training]]-->a3[[Evaluation]]-->a4[[Unit Tests]]
  e1[[Deploy REST API Endpoint]]
  style b1 fill:red
  style e1 fill:red

Convert to ‘pipelines’

flowchart LR
  subgraph p1[Infra Deploy Pipeline]
  b1[[Infrastructure Deploy]]
  end
  subgraph p2[Model Training Pipeline]
  a1[[Data Prep]]-->a2[[Model Training]]-->a3[[Evaluation]]-->a4[[Unit Tests]]
  end
  subgraph p3[Endpoint Deployment Pipeline]
  e1[[Deploy REST API Endpoint]]
  end
  p1 --> p2
  p2-->p3
  style p1 fill:lightblue;
  style p2 fill:lightblue;
  style p3 fill:lightblue;

More detailed pipeline

flowchart LR
  s1[[config.yml]]-->b2
  subgraph p1[Infra Deploy Pipeline]
    direction TB
    b1[[install az cli]]-->b2[[create resource group]]-->b3[[create workspace]]-->b4[[create compute]]
  end
  a7[[environment.yaml]]-->a6
  a9[(Data)]-->a8
  subgraph p2[Model Training Pipeline]
    direction TB
    a6[[register environment]]-->a8[[register data]]-->a1[[Data Prep]]-->a2[[Model Training]]-->a3[[Evaluation]]-->a4[[Register Model]]
  end
  subgraph p3[Endpoint Deployment Pipeline]
    direction TB
    e1[[Create REST Online endpoint]]-->e2[[Deploy Model to enpoint]]-->e3[[test endpoint]]
  end
  p1 --> p2
  p2-->p3
  style p1 fill:lightblue;
  style p2 fill:lightblue;
  style p3 fill:lightblue;