batch_cluster_config now available, Resolved Plan Hooks issue for Windows environments, Adds .tectonignore to ignore paths and files
batch_cluster_config now available for Feature Table declarations
You can now include batch_cluster_config configuration in Feature Table declarations. By specifying an EMRClusterConfig or DatabricksClusterConfig configuration or referencing an ExistingClusterConfig, the ingest materialization jobs will run on a cluster of workers with the specified configuration options like instance type, instance size, and spark configurations.
FeatureTable(
name="user_page_click_feature_table",
entities=[content],
schema=schema,
online=True,
offline=True,
owner="example@tecton.ai",
batch_cluster_config=DatabricksClusterConfig(
instance_type="m5.2xlarge",
number_of_workers=2,
spark_config={
"spark.executor.memory": "7000m",
},
),
)
Resolved Plan Hooks issue for Windows environments
Previously including a plan.py would cause problems for users in a Windows environment. This issue is resolved in SDK version 0.0.54.
Adds .tectonignore to ignore paths and files
You can now add .tectonignore to your feature repository to ignore specified paths and files, similar to .gitignore. See the usage guide for an example.