New technologies have been advancing rapidly across the areas of frictionless data ingestion, customer data management, identity resolution, feature stores, MLOps and customer interaction orchestration. Over the same period many large enterprises …
Exploiting the Data Code: Duality Applying Modern Software Development Practices to Data with Dali
Most large software projects in existence today are the result of the collaborative efforts of hundreds or even thousands of developers. These projects consist of millions of lines of code and leverage a plethora of reusable libraries and services …
Real-time Personalization of QuickBooks using Clickstream Data
In this session, we will talk about Intuit’s real-time personalization ML pipeline. We will use a self-help use case to show how Intuit provides proactive self-help to millions of users by personalizing content based on user behavior to increase …
Reusability in Machine Learning
In this session we will explore modern techniques and tooling which empower reusability in data and analytics solutions. Creating and leveraging reusable machine-learning code has many similarities with traditional software engineering but is also …
Towards a Unified Real-Time ML Data Pipeline, from Training to Serving
On a global marketplace like Etsy where buyers come to buy unique, varied items from sellers from around the globe, the inventory of items is constantly changing. User preferences also change in real time as they discover the latest selection being …
The Only Truly Hard Problem in MLOps
MLOps solutions are often presented as addressing particularly challenging problems. This is mostly untrue. The majority of the problems solved by MLOps solutions have their origins in pre-ML data processing systems and are well addressed by the …
Financially Responsible Feature Engineering
Anyone who has tried doing machine learning at scale knows it can get expensive. The costs associated with training models using on-demand compute and storing features in low latency databases can quickly get out of hand, and we’re often forced to …
Programmatic Supervision for Software 2.0
One of major bottlenecks in the development and deployment of AI applications is the need for the massive labeled training datasets that drive modern ML approaches today. These training datasets traditionally are often labeled by hand at great time …
A Point in Time: Mutable Data in Online Inference
Most business applications mutate relational data. Online inference is often done on this mutable data, so training data should reflect the state at the prediction’s “point in time” for each object. There are a number of data architecture / domain …
Towards Reproducible Machine Learning
We live in a time of both feast and famine in machine learning. Large organizations are publishing state-of-the-art models at an ever-increasing rate but the average data scientist face daunting challenges to reproduce the results themselves. Even in …
Apache Kafka, Tiered Storage and TensorFlow for Streaming Machine Learning Without a Data Lake
Machine Learning (ML) is separated into model training and model inference. ML frameworks typically use a data lake like HDFS or S3 to process historical data and train analytic models. But it’s possible to completely avoid such a data store, using …
Third Generation Production ML Architectures: Lessons from History, Experiences with Ray
Production ML architectures (deployed at scale in production) are evolving at a rapid pace. We suggest there have been two generations so far: the first generation were very much fixed function pipelines with predetermined stages, the second …
