Loading...

"Apache Kafka and the Next 700 Stream Processing Systems" by Jay Kreps

31,963 views

Loading...

Loading...

Transcript

The interactive transcript could not be loaded.

Loading...

Loading...

Rating is available when the video has been rented.
This feature is not available right now. Please try again later.
Published on Sep 27, 2015

Most software systems continuously transform streams of inputs into streams of outputs. Yet the idea of directly modeling stream processing in infrastructure systems is just coming into it's own after a few decades on the periphery.

This talk will cover the basic challenges of reliable, distributed, stateful stream processing. It will cover how Apache Kafka was designed to support capturing and processing distributed data streams by building up the basic primitives needed for a stream processing system.

Finally it will explore how these kind of infrastructure maps to practical problems based on our experience building and scaling Kafka to handle streams that captured hundreds of billions of records per day.

Jay Kreps
CONFLUENT
@jaykreps

Jay Kreps is the CEO of Confluent, Inc. He was formerly the lead architect for data infrastructure at LinkedIn. He is among the original authors of several open source projects including Project Voldemort, a key-value store, Apache Kafka, a distributed messaging system, and Apache Samza a stream processing system.

Loading...

When autoplay is enabled, a suggested video will automatically play next.

Up next


to add this to Watch Later

Add to

Loading playlists...