Loading...

Strata Data 2018: Perverse incentives in metrics - Bonnie Barrilleaux

664 views

Loading...

Loading...

Transcript

The interactive transcript could not be loaded.

Loading...

Rating is available when the video has been rented.
This feature is not available right now. Please try again later.
Uploaded on Oct 16, 2018

Analytics data scientists at LinkedIn are responsible for creating metrics to evaluate product success. They’re very good at increasing any metric the company decides to optimize, but following a single metric blindly can lead to problems. For instance, as LinkedIn encouraged members to join conversations, it found itself in danger of creating a “rich get richer” economy in which a few creators got an increasing share of all feedback. Highly skewed distribution of feedback occurs naturally in any system that distributes content virally, but that doesn’t mean it’s good for creators. As a result, LinkedIn implemented changes to balance out this increasing skew and added creator-focused metrics to monitor how well the ecosystem is distributing feedback to creators, which gives the company a more balanced view of the ecosystem.

Drawing on this example, Bonnie Barrilleaux explains why you must regularly reevaluate metrics to avoid perverse incentives—situations where efforts to increase the metric cause unintended negative side effects. Along the way, Bonnie explores a few illustrative historical examples to demonstrate that this problem has been common throughout human endeavors.

From the talk by Bonnie Barrilleaux at Strata Data NY. Read more: https://conferences.oreilly.com/strat...

Loading...

to add this to Watch Later

Add to

Loading playlists...