
Ensure compliance of production machine learning pipelines
Compliant AzureML
- Budget: undisclosed
- Team Size: managing 5-6 data scientists, working with two internal large organizations
- Context: Microsoft, US
🚀 Project Objectives
Modernize the ML training experience for internal data scientists by migrating legacy pipelines into a secure, scalable and compliant AzureML platform. The goal was to standardize workflows, improve reproducibility and lineage, and ensure traceability and governance across teams.
🛠️ What I Did
As the lead data scientist and data science experience architect, I designed SDKs and libraries to support internal data science teams in their daily workflows. I managed a team of 5–6 data scientists, each embedded with different product teams to ensure successful onboarding and adoption. We operated as the internal onboarding team, guiding early adopters through migration and shaping the platform based on their feedback.
We partnered with pilot teams to gather requirements and identify tooling gaps, then built a flexible framework for writing ML pipelines with full configurability, reproducibility, and CI/CD readiness. I led the effort to package this into a standard internal library and drove its adoption across teams within a year.
🎓 What I Learned
Managing platform migration at scale requires iteration and co-design with users. By deeply understanding the needs of data scientists and translating them into intuitive tooling, we reduced friction and built trust. Aligning technical capabilities with platform constraints helped scale adoption into sustained momentum. The standard library now powers more than 95% of ML pipelines and remains actively used in production.