Loading...

Taming Big Data with Spark Streaming and Scala - Hands On!

41 views

Loading...

Loading...

Transcript

The interactive transcript could not be loaded.

Loading...

Rating is available when the video has been rented.
This feature is not available right now. Please try again later.
Published on Jun 18, 2019

Get the course at https://www.udemy.com/taming-big-data...

New! Updated for Spark 2.3.

"Big Data" analysis is a hot and highly valuable skill. Thing is, "big data" never stops flowing! Spark Streaming is a new and quickly developing technology for processing massive data sets as they are created - why wait for some nightly analysis to run when you can constantly update your analysis in real time, all the time? Whether it's clickstream data from a big website, sensor data from a massive "Internet of Things" deployment, financial data, or something else - Spark Streaming is a powerful technology for transforming and analyzing that data right when it is created, all the time.

You'll be learning from an ex-engineer and senior manager from Amazon and IMDb.

This course gets your hands on to some real live Twitter data, simulated streams of Apache access logs, and even data used to train machine learning models! You'll write and run real Spark Streaming jobs right at home on your own PC, and toward the end of the course, we'll show you how to take those jobs to a real Hadoop cluster and run them in a production environment too.

Across over 30 lectures and almost 6 hours of video content, you'll:

Get a crash course in the Scala programming language

Learn how Apache Spark operates on a cluster

Set up discretized streams with Spark Streaming and transform them as data is received

Analyze streaming data over sliding windows of time

Maintain stateful information across streams of data

Connect Spark Streaming with highly scalable sources of data, including Kafka, Flume, and Kinesis

Dump streams of data in real-time to NoSQL databases such as Cassandra

Run SQL queries on streamed data in real time

Train machine learning models in real time with streaming data, and use them to make predictions that keep getting better over time

Package, deploy, and run self-contained Spark Streaming code to a real Hadoop cluser using Amazon Elastic MapReduce.

This course is very hands-on, filled with achievable activities and exercises to reinforce your learning. By the end of this course, you'll be confidently creating Spark Streaming scripts in Scala, and be prepared to tackle massive streams of data in a whole new way. You'll be surprised at how easy Spark Streaming makes it!

Loading...

When autoplay is enabled, a suggested video will automatically play next.

Up next


to add this to Watch Later

Add to

Loading playlists...