From silos to platform: Building Twitter’s Feature Marketplace
From silos to platform: Building Twitter’s Feature Marketplace

Abstract: 

ML product teams at Twitter have largely relied on their own feature engineering using their own data model and technology stacks. With the proliferation of ML applications, this approach no longer scales. Democratizing feature access was a key objective of the ML Platform build by Twitter Cortex to leverage feature investments across the company. In this talk we'll share our story from genesis of the idea, to overcoming the technological challenges of finding the right abstractions to replace highly optimized custom pipelines, to thinking about incentives of how to make the marketplace work.

Bio: 

Wolfram "Wolf" Arnold has a Ph.D. in computational physics and has been a Silicon Valley veteran since the late dot com boom. He joined Twitter in 2013, and Twitter's Cortex team in 2015. He was part of Cortex's pivot to build an ML Platform organization in 2017, and founded the Feature Management team within Cortex whose flagship product has been Twitter's Feature Store. The Feature Store lets any ML product team benefit from feature engineering investments across the company and has unlocked model improvements and top-line metrics gains in several product areas.