Data Version Control

From version controlling code to version controlling your entire ML pipeline.

Git and why it fails in ML pipelines

Well, most of us are aware about Git and how it can be used as a version control system for our entire codebase.

however, its just worth to ensure that we are on the same page
every time we made a commit git stores that particular version of code to which we can rollback if required.

With each commit git generates a sha-id which can be essentially used to roll back by running the command

git checkout <sha-id-of-your-commit>

Pretty powerful right?

Well, For most of the part yes!
But if we talk about Machine Learning Systems , where we have data files whose size is generally larger than what essentially git can track . it becomes impossible for us to track those files using git

Lets have a look at the root directory of very basic ML system

credit-card-fraud-detection-system/
├── data/
│   ├── raw/fraud_dataset.csv          # 150MB
│   └── processed/train_scaled.npy     # 200MB
├── models/
│   └── xgboost_v3_recall_tuned.pkl    # 400MB
├── notebooks/
│   └── eda.ipynb
└── src/
    └── train.py

Now try running git add data/ on that 150MB CSV.

• GitHub will reject files over 100MB.

• Even if it didn’t do you want to store binary blobs in your Git history?

• Every commit would balloon in size.

• Cloning the repo would take minutes.

• Diffs would be meaningless (you can’t diff a .pkl file)

The core problem:

Git is built for code

ML projects are code + data + models + metrics , and these are interdependent on each other , a change or update in data would result in different metrics over time.

also, while training models our interest lies in experimenting with different set of parameters, models , and data splits to get the best possible model for a given problem.

Git is not built for tracking data , models and providing a modular way to track our experiments

---

Lets take a real scenario:

You are a data scientist intern working at a fintech startup building a credit card fraud detection system. After weeks of experimentation, your team lead walks up and asks:

"Hey, last Tuesday you ran that experiment where you used the 80-20 split with XGBoost and got a recall of 0.91, can you show me those exact results again? Also what were the feature importances? And what was the precision at that threshold?"

You open your laptop.

Your notebook is called ‘‘final_v3_ACTUAL_final_USE_THIS.ipynb’’

You scroll up. Then down. Then up again.

• The cell outputs are gone, you restarted the kernel yesterday.

• The model file got overwritten when you retrained.

• The dataset got updated when your colleague pushed new labeled samples.

• The threshold you used? You think it was 0.42. Or maybe 0.38. You tried both.

You remember the recall was good but you cannot reproduce it because:

• The data has changed : new samples were added, you have no record of what the dataset looked like last Tuesday

• The model file is gone : overwritten by the latest training run

• The parameters are buried somewhere in a cell you may have edited three times since

• The metrics were only in the cell output : which you cleared

• The preprocessing steps : did you use StandardScaler or MinMaxScaler that run? You switched between them twice

So you do the only thing you can do.

You retrain. From scratch. Hoping you remember every decision you made last Tuesday.

Your team lead is still waiting

This is exactly the problem DVC solves.

Data Version Control

DVC (Data Version Control) is an open source version control system specifically built for machine learning projects.

It works alongside Git , not as a replacement.

Git continues to track code. DVC tracks data, models, and metrics. Together they give you complete reproducibility of the entire ML pipeline.

But how does dvc solves this problem?

• dvc generates a basic pointer that points towards the specific version your data either remotely or locally , git just tracks the pointer and not the entire data or model

for example in this case our interim folder is begin tracked by dvc with the help of a md5 hash id, dvc.lock is tracked by git itself , giving us the version of data for a given version of code.

Getting Started with dvc

Let’s get back to our spam detection system, but this time take a real example of an email spam detection system, and to your surprise we have a github repo waiting for us to setup and understand dvc

1. Clone the github repository

git clone https://github.com/moksh-m9u/dvc-blog-tutorial.git
cd dvc-blog-tutorial

2. Install the requirements

pip install -r requirements.txt

---

Understanding the repository structure, (optional step just in case you wish to explore more)here in order to run the notebooks in experimentation, we must first install the data set from kaggle hub , email spam detecion dataset and place it in experiments folder

DVC-BLOG-TUTORIAL/
├── experiments/
│   ├── mynotebook.ipynb
│   └── notebook.ipynb
├── src/
│   ├── data_ingestion.py
│   ├── data_preprocessing.py
│   ├── feature_engineering.py
│   ├── model_building.py
│   └── model_evaluation.py
├── .gitignore
├── dvc.yaml
├── params.yaml
├── README.md
└── requirements.txt

• In this project you would notice that , the experiments are separated from actual modular pipelines, and that’s the standard one follows, this provides you modularity and allows you to monitor your pipeline over longer runs or in production

• params.yaml contains different sets of parameters your modular pipeline uses, think of it like a config file we can directly make any sort of changes in this file to implement it in our pipeline.

data_ingestion:
  test_size: 0.2
feature_engineering:
  max_features: 400
model_building:
  n_estimators: 900
  random_state: 2

---

3. Initializing dvc

   • as simple as initializing a git repo!

dvc init 

4. Running the entire pipeline (reproducing)

dvc repro

now this runs your entire pipeline in just one command!
else you would have to run this modules iteratively or maybe using a python script but the main power of dvc lies in its functionality to provide tracking over you data

Running this pipeline would create this folders in your directory, the thing to notice here is that these files will not be tracked by git,

but git tracks .dvc file which essentially contains information about version of our data and models

4. Tracking a untracked file (data) using dvc :

dvc add experiments/dataset.csv    #you can only perform this if you downloaded the dataset from kagglehub

link to dataset - https://www.kaggle.com/datasets/uciml/sms-spam-collection-dataset

you can download and add this data set into your experiments folder to follow along step 4

now dvc will keep a track of this file too!

Note that files already tracked by git can not be tracked by dvc

5. Tracking and committing this change

now , we need to track and push this one iteration so that it can be fetched if required in future

dvc repro
git add .
git commit -m "Experiment 1: max_features=400, n_estimators=900"
dvc push   

or we can simply run

dvc exp run

now this is a more cleaner way , in which we don’t have multiple commits for each experiment and dvc track everything for you

Real power of dvc

the real power of dvc lies in doing multiple experiments and choosing the best one out of them

• go to params.yaml and make some arbitrary changes to the numeric values of parameters

data_ingestion:
  test_size: 0.3
feature_engineering:
  max_features: 350
model_building:
  n_estimators: 750
  random_state: 42

now we will reproduce a pipeline with these new set of parameters

dvc exp run 

now just notice that your metrics in your reports folder will be changed.
But how to track all these?

just simply run or you could install dvc extension in your IDE to visualize it better

dvc exp show

Now every change you make , to the parameters of your model , you would simply run dvc exp run and that would just keep a track and do the heavy lifting for you!

Now a better way to visualize this is with help of dvc extension

1. Install this dvc extension you IDE

2. Locate dvc path and you are all set to get a better user experience of your experiments!

and you can rollback to any version of your parameters , data and model based on the requirement of the given problem ,

and btw you can answer your senior in less than 15 seconds by simply running

dvc exp apply b35f590

and you are on your previous model!
Each experiment has this unique id through which we can rollback to any previous version , logically to model which has more accuracy , precision or recall depending on our use case and dvc keeps track of all these changes locally

---

About me!

well, this is my first blog, and i hope this added a bit value or maybe……..

a lil bit about me:

I’m Moksh.

Digital wanderer stuck between Strava, Kaggle, and Claude. Half my tabs are ML research papers. The other half are Fight Club, Alice in Borderland, and well…… what not

I'm deep in the trenches of learning the craft.

linkedin - https://www.linkedin.com/in/mokshhh/

X- x.com/moksh1326

kaggle-https://www.kaggle.com/moksh067