Model Calibration in the Etsy Ads Marketplace | Tecton


Model Calibration in the Etsy Ads Marketplace

apply(meetup) - Feb '22 - 30 minutes

When displaying relevant first-party ads to buyers in the Etsy marketplace, ads are ranked using a combination of outputs from ML models. The relevance of ads displayed to buyers and costs charged to sellers are highly sensitive to the output distributions of the models. Various factors contribute to model outputs which include the makeup of training data, model architecture, and input features. To make the system more robust and resilient to modeling changes, we have calibrated all ML models that power ranking and bidding. In this talk, we will first discuss the pain points and use cases that identified the need for calibration in our system. We will share the journey, learnings, and challenges of calibrating our machine learning models and the implications of calibrated outputs. Finally, we will explain how we are using the calibrated outputs in downstream applications and explore opportunities that calibration unlocks at Etsy.
Erica Greene

Senior Engineering Manager


Erica Greene is an engineering manager based in Brooklyn, NY. She loves working with teams to design and build ML-driven products.
Seoyoon Park

Senior Software Engineer


Seoyoon is a Senior Software Engineer on the Ad Ranking team at Etsy. He has worked on various projects which includes adding tooling to introspect data quality, assessing model performance, and most recently calibrating ML models used for ranking. Prior to Etsy, he was at Signafire, a data analytics startup, where he built pipelines for data visualization and analysis.

Let's keep in touch

Receive the latest content from Tecton!

© Tecton, Inc. All rights reserved. Various trademarks held by their respective owners.

The Gartner Cool Vendor badge is a trademark and service mark of Gartner, Inc., and/or its affiliates, and is used herein with permission. All rights reserved.
Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

Request a Demo

Request a free trial

Interested in trying Tecton? Leave us your information below and we’ll be in touch.​

Contact Sales

Interested in trying Tecton? Leave us your information below and we’ll be in touch.​