Feature stores are becoming ubiquitous in real-time model serving systems, however there has been limited work in understanding how features should be maintained over changing data. In this talk, we present ongoing research at the RISELab on streaming feature maintenance that optimizes both resource costs and downstream model accuracy. We introduce a notion of feature store regret to evaluate feature quality of different maintenance policies, and test various policies on real-world time-series data. … Read More
In this presentation, Clément will provide insights into the revolution taking place in the open-source community with machine learning. From the CEO who is on a mission to create the “Github of Machine Learning,” learn how the best-in-class companies and talent are using Hugging Face’s tools, and why the open source approach is particularly powerful in doing so. … Read More
Reinforcement Learning has historically not been as widely adopted in production as other learning approaches (particularly supervised learning), despite being capable of addressing a broader set of problems. But we are now seeing an exponential growth in production RL applications: so much so that it looks like production RL is about to reach a tipping point to the mainstream. In this talk, we’ll talk about why this is happening; detail concrete examples of the areas where RL is adding value; and share some practical tips on deploying RL in your organization. … Read More
This talk breaks down stage-by-stage requirements and challenges for online prediction and fully automated, on-demand continual learning. We’ll also discuss key design decisions a company might face when building or adopting a machine learning platform for online prediction and continual learning use cases. … Read More
Customers evaluating MLOps platforms as a service need to provide customer data during the evaluation phase. The data often needs to be moved to the MLOps companies’ warehouses. This is not a simple task and can become costly if the two partners are using different cloud service providers. Apart from the challenges of data transfer, there is also the matter of compliance and privacy. For sensitive data, a secure transfer is not enough, and masking and other anonymization measures need to be implemented. In this talk, we review the myriad roadblocks faced by companies evaluating MLOPs platforms in providing access to their data for evaluation purposes. Further, we discuss some potential solutions.
Machine learning systems are now easier to build than ever, but they still don’t perform as well as we would hope on real applications. I’ll explore a simple idea in this talk: if ML systems were more malleable and could be maintained like software, we might build better systems. I’ll discuss an immediate bottleneck towards building more malleable ML systems: the evaluation pipeline. I’ll describe the need for finer-grained performance measurement and monitoring, the opportunities paying attention to this area could open up in maintaining ML systems, and some of the tools that I’m building (with great collaborators) in the Robustness Gym and Meerkat projects to close this gap. … Read More