Today, we’re excited to announce that Tecton is becoming a core contributor to the Feast open source feature store, and that Willem Pienaar, creator of Feast, is joining the Tecton team. In other words, we’re strong believers in the Feast project and are investing meaningful resources into its success. We share a joint vision with Willem: enable every organization to operationalize ML at scale by allowing teams to build and deploy features quickly and reliably.
If you’ve been following Tecton, you know that we’re also building our own commercial feature store that’s currently in early access and soon-to-become Generally Available. And this naturally raises a few questions: why invest in two feature stores, and what does this mean for both Feast and the Tecton feature store?
Making Feature Stores Accessible to Every Organization
We believe that feature stores are an essential part of the modern ML stack because they solve the hardest problem of productionizing operational ML: getting fresh feature data reliably computed and served into production. As discussed in “What is a Feature Store?”, a feature store solves this problem by managing feature transformation pipelines, storing features, and serving features in production. The Tecton founding team saw the need for a feature store at Uber where we built the Michelangelo project. Similarly, other technology companies like Gojek also realized the importance of feature stores, leading to the development of Feast.
Unfortunately, feature stores have up to now been accessible only to the largest, most sophisticated technology companies that have the skills and resources to build their own infrastructure.
For the rest of the industry, this technology has remained mostly out of reach. Building your own feature store requires dedicated teams of highly skilled engineers and multiple years of development time. Our objective is to change this, and to put feature stores in reach of every organization, regardless of their ML maturity and available commercial resources.
Freedom to Choose Between Open Source and Commercial Software
We have seen strong demand for teams to enter the early access program we introduced earlier this year. We’ve partnered with customers ranging from small growth-stage startups to large Fortune 50 companies, across North America and Europe. We now have customers like Atlassian running at production scale on the Tecton enterprise feature store.
Along the way, we’ve learned that user needs vary significantly based on their level of maturity, size, and resources. Cloud services are preferred by larger teams that are operationalizing ML at scale and need to move fast, offload the burden of managing and updating software, and have guaranteed service levels with enterprise support. On the other hand, open source is highly valued by small ML teams that are just starting out and want to get their hands dirty, and by large organizations that need to customize the software to their specific needs and have the resources and skills to self-manage their ML infrastructure.
Our objective at Tecton is to solve the data problem for Operational ML for all organizations. The Tecton enterprise feature store is only available as a managed cloud service, leaving a gap for organizations that need open source software today. To close the gap, we decided to put significant resources behind Feast. Partnering with the Feast project, our plan is to leverage our existing development efforts in and learnings from Tecton to help make Feast the best open source feature store, both for small teams just starting out and for larger teams that need the flexibility of open source. As strong allies, we’re going to advance the state of the art for feature stores and enable their adoption across the industry.
Feast: The Leading Open Source Feature Store
Feast was developed jointly by Gojek and Google Cloud, and first announced about two years ago. Since its initial release in 2019, Feast has grown rapidly, with multiple companies, including Microsoft, Agoda, Farfetch, Postmates and Zulily adopting and/or contributing to the project. Today, the project has more than 1,100 GitHub stars and has been adopted as a component in Kubeflow.
Willem and the Feast contributors have a clear vision of where they want to take the project, and that vision is very well aligned with Tecton’s vision.
Feast will remain a completely independent project, managed by the LF AI & Data Foundation. Willem will continue to be fully committed to Feast, and an official maintainer of the project. And of course, Feast continues to welcome new users and contributors. If you want to learn more or are looking to use Feast: please check out their resources and get involved!
- Project website: feast.dev
- Slack channel: #Feast
- Documentation: docs.feast.dev
- GitHub repository: feast-dev/feast
Feast and Tecton: Deployment and Support
Tecton and Feast’s differences are most notable on how they’re deployed and managed: Feast is fully open source and self-managed. Tecton is a fully-managed cloud service and can be deployed into a company’s private VPC, or into a Tecton-provided single-tenant SaaS environment. While Feast got started on GCP and Tecton got started on AWS, both will support GCP, AWS and Azure in 2021.
|License||Open Source, Apache 2.0||Enterprise|
|Deployment model||Self-managed via Kubernetes Helm Charts||Hosted SaaS|
|Cloud platforms||Amazon Web Services|
Google Cloud Platform
Azure (Q1 2021)
|Amazon Web Services|
Feast and Tecton: Capabilities
Tecton and Feast have independent roots and were started by different companies. Tecton’s capability set is largely a superset of Feast’s, with a strong focus on fully-fledged transformation support and enterprise capabilities:
Beyond Feast’s capabilities, Tecton’s proprietary capabilities are going to focus on data governance, advanced collaboration and operational excellence.
Here’s a more detailed overview of the projects’ capabilities:
|Precomputed Feature Ingestion||Batch Ingest|
|Feature Transformations||Coming in 2021||Batch|
|Storage||Offline feature store|
Online feature store
|Offline feature store|
Online feature store
|Serving||Online Serving API|
Offline Serving with point-in-time time travel
|Online Serving API|
Offline Serving with point-in-time time travel
|Monitoring||Operational Health Monitoring||Operational Health Monitoring|
Feature Drift detection
Data Quality Monitoring
|Web-UI||Coming in 2021||Available|
|Data Governance & Security||OAuth2||Fine-grained access control|
SSO via SAML
|CI/CD Pipeline Integration||Coming in 2021||Supported via CLI|
|Feature Versioning and Reproducibility|
Offering a seamless migration between Tecton and Feast
Often, the preferences between using a managed enterprise offering and a self-managed open source project change over time. An enterprise using Tecton’s managed offering may eventually require greater customizability and prefer to switch to a custom fork of Feast. Similarly, a small team that got started with Feast may eventually desire the enterprise capability and managed service of Tecton. To simplify transitions, Tecton is committed to providing a simple migration path between both projects. This migration path will eventually be enabled via:
- Feature Serving API Parity: The feature serving APIs will be identical, allowing customers to switch between both feature stores without having to modify the feature store clients. Model training notebooks and prediction services running in production can remain largely unchanged as long as the client points at the new Feature Store backend.
- Feature Definition Parity: Tecton’s feature definition DSL will be a superset of Feast’s DSL. As a result, Feast feature definitions can seamlessly be applied to a Tecton deployment.
- Feature Store Parity: Tecton and Feast will support the same offline and online feature storage technologies (e.g. S3, Delta, DynamoDB, Redis etc.) and understand the same storage contract. As a result, no physical data will need to be migrated as customers choose to migrate between both projects.
Want to learn more?
If you’re now embarking on a journey to operationalize ML and wondering, first, whether you need a feature store and, second, whether Tecton or Feast is the right choice for you, we’d love to speak with you. We’re obviously big fans of both the Feast and Tecton feature stores, and can guide you to the best option based on your specific needs. Please get in touch at email@example.com or sign up for a demo and we’ll be in touch soon.