Loading...

Batch Data Processing at Spotify by Erik Bernhardsson

4,462 views

Loading...

Loading...

Transcript

The interactive transcript could not be loaded.

Loading...

Rating is available when the video has been rented.
This feature is not available right now. Please try again later.
Published on Apr 24, 2013

WANT TO EXPERIENCE A TALK LIKE THIS LIVE?
Barcelona: https://www.datacouncil.ai/barcelona
New York City: https://www.datacouncil.ai/new-york-city
San Francisco: https://www.datacouncil.ai/san-francisco
Singapore: https://www.datacouncil.ai/singapore


Video Intro Animation provided by http://animoto.com/

Visit http://www.g33ktalk.com for more videos and tech talks. Speaker: Erik Bernhardsson, Tech Lead, Spotify

Erik is the Tech Lead of the discovery team at Spotify. He is building a music recommender system using large scale machine learning algorithms, mainly matrix factorization of big matrices using Hadoop.

Previously head of the Business Intelligence team in the Stockholm office where he was responsible for collecting, aggregating and making sense out of all the data.

About the talk:

Erik will be talking about Luigi, a recently open-sourced Python framework that helps you build complex pipelines of batch jobs, handle dependency resolution, and create visualizations to help manage multiple workflows.

It also comes with Hadoop support built in (and that's where really where its strength becomes clear). Luigi provides an infrastructure that powers several Spotify features including recommendations, top lists, A/B test analysis, external reports, internal dashboards, and many more.

Read more about it on Github: https://github.com/spotify/luigi

More content like this at http://g33ktalk.com

FOLLOW DATA COUNCIL:
Twitter: https://twitter.com/DataCouncilAI
LinkedIn: https://www.linkedin.com/company/data...
Facebook: https://www.facebook.com/datacouncilai

Loading...

Advertisement
When autoplay is enabled, a suggested video will automatically play next.

Up next


to add this to Watch Later

Add to

Loading playlists...