Loading...

Using Structured Streaming in Apache Spark: Insights Without Tradeoffs

6,964 views

Loading...

Loading...

Transcript

The interactive transcript could not be loaded.

Loading...

Loading...

Rating is available when the video has been rented.
This feature is not available right now. Please try again later.
Published on Feb 8, 2017

Get started with Structured Streaming on Databricks today. https://databricks.com/try-databricks

Apache Spark 2.0 introduced Structured Streaming which allows users to continually and incrementally update your view of the world as new data arrives while still using the same familiar Spark SQL abstractions. Michael Armbrust from Databricks talks about the progress made since the release of Spark 2.0 on robustness, latency, expressiveness and observability, using examples of production end-to-end continuous applications.

Overview:
Parallelism and Complexity
Developer Productivity and Efficiency
Throughput and Latency
Production Use Cases
- Viacom
- iPass
Streaming at Databricks
Engineer Office Hours

This talk was originally presented at Spark Summit East 2017.

You can view the slides on Slideshare:
http://www.slideshare.net/databricks/...

Related Articles:
Structured Streaming In Apache Spark
https://databricks.com/blog/2016/07/2...

Real-time Streaming ETL with Structured Streaming in Apache Spark 2.1
https://databricks.com/blog/2017/01/1...

Loading...

When autoplay is enabled, a suggested video will automatically play next.

Up next


to add this to Watch Later

Add to

Loading playlists...