Why Tecton is Backing the Feast Open Source Feature Store

Today, we’re excited to announce that Tecton is becoming a core contributor to the Feast open source feature store, and that Willem Pienaar, creator of Feast, is joining the Tecton team. In other words, we’re strong believers in the Feast project and are investing meaningful resources into its success. We share a joint vision with Willem: enable every organization to operationalize ML at scale by allowing teams to build and deploy features quickly and reliably.

If you’ve been following Tecton, you know that we’re also building our own commercial feature store that’s currently in early access and soon-to-become Generally Available. And this naturally raises a few questions: why invest in two feature stores, and what does this mean for both Feast and the Tecton feature store?

Making Feature Stores Accessible to Every Organization

We believe that feature stores are an essential part of the modern ML stack because they solve the hardest problem of productionizing operational ML: getting fresh feature data reliably computed and served into production. As discussed in “What is a Feature Store?”, a feature store solves this problem by managing feature transformation pipelines, storing features, and serving features in production. The Tecton founding team saw the need for a feature store at Uber where we built the Michelangelo project. Similarly, other technology companies like Gojek also realized the importance of feature stores, leading to the development of Feast.

Unfortunately, feature stores have up to now been accessible only to the largest, most sophisticated technology companies that have the skills and resources to build their own infrastructure.

For the rest of the industry, this technology has remained mostly out of reach. Building your own feature store requires dedicated teams of highly skilled engineers and multiple years of development time. Our objective is to change this, and to put feature stores in reach of every organization, regardless of their ML maturity and available commercial resources.

Freedom to Choose Between Open Source and Commercial Software

We have seen strong demand for teams to enter the early access program we introduced earlier this year. We’ve partnered with customers ranging from small growth-stage startups to large Fortune 50 companies, across North America and Europe. We now have customers like Atlassian running at production scale on the Tecton enterprise feature store.

Along the way, we’ve learned that user needs vary significantly based on their level of maturity, size, and resources. Cloud services are preferred by larger teams that are operationalizing ML at scale and need to move fast, offload the burden of managing and updating software, and have guaranteed service levels with enterprise support. On the other hand, open source is highly valued by small ML teams that are just starting out and want to get their hands dirty, and by large organizations that need to customize the software to their specific needs and have the resources and skills to self-manage their ML infrastructure.

Our objective at Tecton is to solve the data problem for Operational ML for all organizations. The Tecton enterprise feature store is only available as a managed cloud service, leaving a gap for organizations that need open source software today. To close the gap, we decided to put significant resources behind Feast. Partnering with the Feast project, our plan is to leverage our existing development efforts in and learnings from Tecton to help make Feast the best open source feature store, both for small teams just starting out and for larger teams that need the flexibility of open source. As strong allies, we’re going to advance the state of the art for feature stores and enable their adoption across the industry.

Feast: The Leading Open Source Feature Store

Feast was developed jointly by Gojek and Google Cloud, and first announced about two years ago. Since its initial release in 2019, Feast has grown rapidly, with multiple companies, including Microsoft, Agoda, Farfetch, Postmates and Zulily adopting and/or contributing to the project. Today, the project has more than 1,100 GitHub stars and has been adopted as a component in Kubeflow.

Willem and the Feast contributors have a clear vision of where they want to take the project, and that vision is very well aligned with Tecton’s vision.

Feast will remain a completely independent project, managed by the LF AI & Data Foundation. Willem will continue to be fully committed to Feast, and an official maintainer of the project. And of course, Feast continues to welcome new users and contributors. If you want to learn more or are looking to use Feast: please check out their resources and get involved!

• Project website: feast.dev
• Slack channel: #Feast
• Documentation: docs.feast.dev
• GitHub repository: feast-dev/feast

Feast and Tecton: Deployment and Support

Tecton and Feast’s differences are most notable on how they’re deployed and managed: Feast is fully open source and self-managed. Tecton is a fully-managed cloud service and can be deployed into a company’s private VPC, or into a Tecton-provided single-tenant SaaS environment. While Feast got started on GCP and Tecton got started on AWS, both will support GCP, AWS and Azure in 2021.

Feast and Tecton: Capabilities

Tecton and Feast have independent roots and were started by different companies. Tecton’s capability set is largely a superset of Feast’s, with a strong focus on fully-fledged transformation support and enterprise capabilities:

Beyond Feast’s capabilities, Tecton’s proprietary capabilities are going to focus on data governance, advanced collaboration and operational excellence.

Here’s a more detailed overview of the projects’ capabilities:

	Feast	Tecton
Precomputed Feature Ingestion	Batch Ingest Stream Ingest	Batch Ingest Stream Ingest RPC Ingest
Feature Transformations	Coming in 2021	Batch Streaming On-Demand
Storage	Offline feature store Online feature store	Offline feature store Online feature store
Serving	Online Serving API Offline Serving with point-in-time time travel	Online Serving API Offline Serving with point-in-time time travel
Monitoring	Operational Health Monitoring	Operational Health Monitoring Feature Drift detection Data Quality Monitoring Prometheus integration
Web-UI	Coming in 2021	Available
Data Governance & Security	OAuth2	Fine-grained access control SSO via SAML Audit Logging
CI/CD Pipeline Integration	Coming in 2021	Supported via CLI
Feature Versioning and Reproducibility
Feature Testing
Cost Controls
Autoscaling
High Availability
Disaster Recovery

Offering a seamless migration between Tecton and Feast

Often, the preferences between using a managed enterprise offering and a self-managed open source project change over time. An enterprise using Tecton’s managed offering may eventually require greater customizability and prefer to switch to a custom fork of Feast. Similarly, a small team that got started with Feast may eventually desire the enterprise capability and managed service of Tecton. To simplify transitions, Tecton is committed to providing a simple migration path between both projects. This migration path will eventually be enabled via:

• Feature Serving API Parity: The feature serving APIs will be identical, allowing customers to switch between both feature stores without having to modify the feature store clients. Model training notebooks and prediction services running in production can remain largely unchanged as long as the client points at the new Feature Store backend.
• Feature Definition Parity: Tecton’s feature definition DSL will be a superset of Feast’s DSL. As a result, Feast feature definitions can seamlessly be applied to a Tecton deployment.
• Feature Store Parity: Tecton and Feast will support the same offline and online feature storage technologies (e.g. S3, Delta, DynamoDB, Redis etc.) and understand the same storage contract. As a result, no physical data will need to be migrated as customers choose to migrate between both projects.

Want to learn more?

If you’re now embarking on a journey to operationalize ML and wondering, first, whether you need a feature store and, second, whether Tecton or Feast is the right choice for you, we’d love to speak with you. We’re obviously big fans of both the Feast and Tecton feature stores, and can guide you to the best option based on your specific needs. Please get in touch at hello@tecton.ai or sign up for a demo and we’ll be in touch soon.