Tecton

Instance Type Configuration and Data Source Previews

  • Derek Salama
By
Published: November 21, 2021
Last updated: December 1, 2021
Share

Default Stream Cluster Configuration now includes On-demand instances for Driver nodes

Using an on-demand instance for the Spark driver node can make stream processing more reliable by preventing losing the entire cluster to spot termination. By using a mix of instance types, you can ensure reliability while taking advantage of cheaper spot instances for additional processing power.

The new first_on_demand parameter for DatabricksClusterConfig and EMRClusterConfig enables configuring a mix of on-demand and spot instances in a single cluster. When configured, the first first_on_demand nodes of the cluster will use on_demand instances. The rest will use the type specified by instance_availability.

If not specified, then Tecton will default to first_on_demand=1 for StreamFeatureView and StreamWindowAggregateFeatureView.

Spot with fallback instance availability for Databricks

Materialization jobs on Databricks can now be configured to use the spot with fallback availability option.

DatabricksClusterConfig.instance_availability now supports the spot_with_fallback option. See the Databricks documentation for more details.

Raw Data Source Preview

You can now use the Tecton SDK to view Data Source inputs before the translator function is applied. Viewing a sample of this raw data can help debug translator and data source issues.

To do so, set apply_translator=False when using the
StreamDataSource.start_stream_preview() or BatchDataSource.get_dataframe() methods.

Share

Request a Demo

Unfortunately, Tecton does not currently support these clouds. We’ll make sure to let you know when this changes!

However, we are currently looking to interview members of the machine learning community to learn more about current trends.

If you’d like to participate, please book a 30-min slot with us here and we’ll send you a $50 amazon gift card in appreciation for your time after the interview.

CTA link

or

CTA button

Contact Sales

Interested in trying Tecton? Leave us your information below and we’ll be in touch.​

Unfortunately, Tecton does not currently support these clouds. We’ll make sure to let you know when this changes!

However, we are currently looking to interview members of the machine learning community to learn more about current trends.

If you’d like to participate, please book a 30-min slot with us here and we’ll send you a $50 amazon gift card in appreciation for your time after the interview.

CTA link

or

CTA button

Request a free trial

Interested in trying Tecton? Leave us your information below and we’ll be in touch.​

Unfortunately, Tecton does not currently support these clouds. We’ll make sure to let you know when this changes!

However, we are currently looking to interview members of the machine learning community to learn more about current trends.

If you’d like to participate, please book a 30-min slot with us here and we’ll send you a $50 amazon gift card in appreciation for your time after the interview.

CTA link

or

CTA button