Domain-driven Data Science

Pietro Peterlongo

PyData NYC, Nov 8 2024

github.com/pietroppeter/domain-driven-data-science

Agenda

Why domain is important? 🤹
Logistics and Supply Chain 📦
Stories and Ideas 💡

Agenda

Why domain is important? 🤹
Logistics and Supply Chain 📦

Forecasting 🔮

Stories and Ideas 💡

👋 Pietro (he/him) 👨‍👩‍👧🇮🇹🍝🏔️⛵️🎭

🧮👨‍🔬 (Applied) Math

👋 Pietro (he/him) 👨‍👩‍👧🇮🇹🍝🏔️⛵️🎭

🧮👨‍🔬 (Applied) Math

🏭🧑‍💻 DS@ToolsGroup toolsgroup.com

👋 Pietro (he/him) 👨‍👩‍👧🇮🇹🍝🏔️⛵️🎭

🧮👨‍🔬 (Applied) Math

🏭🧑‍💻 DS@ToolsGroup toolsgroup.com

🐙🗽 Recurse Center recurse.com

👋 Pietro (he/him) 👨‍👩‍👧🇮🇹🍝🏔️⛵️🎭

🧮👨‍🔬 (Applied) Math

🏭🧑‍💻 DS@ToolsGroup toolsgroup.com

🐙🗽 Recurse Center recurse.com

💙🧡 PyData Milan milano.python.it

👋 Pietro (he/him) 👨‍👩‍👧🇮🇹🍝🏔️⛵️🎭

🧮👨‍🔬 (Applied) Math

🏭🧑‍💻 DS@ToolsGroup toolsgroup.com

🐙🗽 Recurse Center recurse.com

💙🧡 PyData Milan milano.python.it

🏄🧑‍💻 DS@AgileLab handbook.agilelab.it

1. Domain 🤹

Data Science Venn Diagram, Drew Conway, 2010

📚 Content Production

🧮 Math

🧑‍💻 Code

🚚 Domain

📚 Content Production

🧮 Math

🧑‍💻 Code

🚚 Domain

📚 Content Production

🧮 Math

🧑‍💻 Code

🚚 Domain

📚 Content Production

🧮 Math

🧑‍💻 Code

🚚 Domain

via GIPHY

Success of DS/ML/AI Projects

Technology

data quality
model accuracy
implementation

Success of DS/ML/AI Projects

Technology

data quality
model accuracy
implementation

Business

stakeholders
valuable
used

Inspiration 💡

Domain Driven Design

⠀

from SWE
domain is important for devs
ubiquitous language

Data Mesh

⠀

DDD for Big Data
Data Product
agilelab.it/blog

Data Mesh

⠀

DDD for Big Data
Data Product
agilelab.it/blog

Our product: witboost 🤝

Data Mesh

Data Architecture

🎧 podcast

2. Logistics & Supply Chain 🚚

What is Logistics?

"an army without its baggage train is lost; without provisions it is lost; without bases of supply it is lost" Art of War, Sun Tzu, 5th BC

Storage

Transportation

Automation

Modern Supply Chains

extended networks of suppliers
demand-driven market
compete on both service and cost
turbulence and volatility

Planning

Demand

Service

Stock

Planning

Demand

Service

Stock

Planning

Demand

Service

Stock

Planning

Demand

Service

Stock

Uncertainity

Demand is uncertain, Supply is uncertain

Uncertainity

Demand is uncertain, Supply is uncertain

Lead Time

Uncertainity

Demand is uncertain, Supply is uncertain

Lead Time

Safety Stock

extra stock due to uncertainty to mitigate risk of stock outs
linked to demand variability

Constraints

capacity: production, budget, storage, time
integral: minimum lot, incremental lot, unit of measures
feasibility: schedules, opening hours

Constraints

capacity: production, budget, storage, time
integral: minimum lot, incremental lot, unit of measures
feasibility: schedules, opening hours

Optimization techniques (Operations Research)

2.1 Forecasting 🔮

Modelling

Dimensions

Product (SKUs, Family, Brands, Divisions)
Market (Warehouses, Regions, Customers, Channels)
Time (day, week, month)

Algorithms

Statistical

Exponential smoothing:

$$y_{t+1} = \alpha x_t + (1 - \alpha)y_t$$

Holt Winters (trend, seasonality)
Croston (intermitted demand)

Algorithms

Statistical

Exponential smoothing:

$$y_{t+1} = \alpha x_t + (1 - \alpha)y_t$$

Holt Winters (trend, seasonality)
Croston (intermitted demand)

boosting methods (LightGBM, ...)
neural methods (DeepAR, ...)
foundational models (TimeGPT)

Evaluation

Metrics

Bias
RMSE
MAE
❌ MAPE

vandeput's article

Evaluation

Metrics

Bias
RMSE
MAE
❌ MAPE

vandeput's article

M5 Competition

Kaggle competition
Walmart Sales data
Point Forecast and Probabilistic
12 product-market levels
WRMSSE

research article

Seasonality

Promotions

Initialization

Seasonality

Promotions

Initialization

Seasonality

Promotions

Initialization

Seasonality

Promotions

Initialization

Forecasting with Nixtla

github.com/pietroppeter/pymi-timeseries-forecasting-nixtla

3. Stories 💡

🔮 Trust in the model

Story: a model for forecast of New Product sales

model is better than baseline
users do not trust the numbers
they are used to a different process
does interpretability help?

🏗️ Data Generating Process

Story: Data Exploration of Warehouse data

order data: many more rows for outbound than inbound orders, why?
pick task data: missing a day in a specific week, why?
...

🏗️ Data Generating Process

Story: Data Exploration of Warehouse data

order data: many more rows for outbound than inbound orders, why?
pick task data: missing a day in a specific week, why?
...

Rules of ML

Eugene Yan on 1st rule

First, design and implement metrics
Choose machine learning over a complex heuristic. "

Google's rules of ML

Talk to experts

Interview

Try to uncover opportunties and risks for a data-driven tool you might want to build.

⠀

Interface

It helps to iterate with an interactive tool that shows data and visualization.

Document the domain

e.g. Business, Domain and Data essentials in README

Document the domain

e.g. Business, Domain and Data essentials in README

- Company X is investing strategically in 3rd party logistics
- Low costs are key to a successful operation

Document the domain

e.g. Business, Domain and Data essentials in README

- Company X is investing strategically in 3rd party logistics
- Low costs are key to a successful operation

- Main cost component in a Warehouse is picking (time)
- Using a ABC class based positioning of items could help

Document the domain

e.g. Business, Domain and Data essentials in README

- Company X is investing strategically in 3rd party logistics
- Low costs are key to a successful operation

- Main cost component in a Warehouse is picking (time)
- Using a ABC class based positioning of items could help

- Data comes from a WMS
- We have 2 years of data, 1 year of clean data