New FeatureView Run SDK Method, Databricks Runtime Update, and Support for Python 3.8

New FeatureView Run SDK Method for testing feature transformations

Tecton is releasing a new Run API to be used for dry-run executing a FeatureView’s transformation the same way Tecton will execute it during materialization or at feature retrieval time. One method introduced is run which can be called from all Feature Views, and also supports mock input data. Another method is run_stream available for streamable Feature Views.

Here’s an example of how this looks in practice.

import tecton
import spark
from datetime import datetime
from datetime import timedelta

ws = tecton.get_workspace('...')
fv = ws.get_feature_view('...') # BatchFeatureView or StreamFeatureView
mock_df_1 = spark.createDataFrame([ # Spark and Pandas DataFrame are supported
    {'field_1': 'value_1', ...}, # row 1
    {'field_1': 'value_1', ...}, # row 2
])
mock_df_2 = spark.createDataFrame([
    {'field_1': 'value_1', ...}, # row 1
])

# input_1 and input_2 correspond to FeatureView input names.
fv.run(<input_1>=mock_df_1, <input_2>=mock_df_1, feature_start_time=start_time=(datetime.now() - timedelta(days = 7)))

The API details can be found here.

Databricks Runtime 9.1 and EMR Runtime 6.4 Support

We’re excited to announce that Tecton is launching support for Spark 3 with the latest Databricks and EMR runtimes! These new versions deliver faster query performance, security updates, and better support for Delta tables.

The latest Tecton SDK can now be used with Databricks Runtime 9.1 LTS or EMR 6.4.0. See the documentation for instructions on how to update your notebook cluster.

Tecton will continue to support compatibility with the current runtimes (DBR 6.4 LTS, EMR 5.30) through Dec. 31, 2021. Of course, we recommend you update sooner to take advantage of the newest versions.

Support for Python 3.8

Previously, the Tecton SDK required a Python 3.7 environment due to incompatibility between PySpark 2.4.5 and newer Python versions. Starting with v0.0.58, you can install the Tecton CLI in a Python 3.8. Just run pip install tecton to get the latest version.