 Hello, my name is Paul Bredner. I'm the technology evangelist at Instacluster, and this talk is building zero code streaming data pipelines using open source technologies. So it's a lightning talk. It's an update from the talk I gave last year, and it involves two folks. The technologies include Kafka Connect, which is common elastic search and Kibana, which I talked about last year, and the second pipeline PostgreSQL and Supercept. Previously, I talked about building a Kafka Connect pipeline. This is a photo I took in Berlin at Apache Con a few years ago. I wondered at the time whether or perhaps it transferred water or beer. In fact, what I'm going to talk about is a water data source. The source data is the NOAA title data, which comes from a REST API and gives you access to hundreds of title data sensors around the world. The first pipeline consists of the NOAA REST API with data fed into the REST source connector, then Kafka, then the elastic search sync connector, and finally elastic search and Kibana. There's an overview of the architecture. This is an example of one of the visualizations in Kibana, just showing the data as sort of like a graph. You can see things like the lunar day and the title range. What can go wrong? Well, I left the system running for a few days and discovered that some of the connectors were failing, and it turned out that sometimes there were JSON error messages, and sometimes there were even non-JSON errors being fed into the system as well, and the connectors did not cope very well. So what I needed was improved error handling, and I discovered the Apache Camel Kafka Connectors project, which has a very large collection of open source Kafka connectors with robust error handling. So what's new? This is the second path. It's basically a different pipeline with new syncs added, the PostgreSQL sync connector, feeding data into PostgreSQL and then visualization using Apache SuperSet. Some of the challenges around this pipeline were finding open source PostgreSQL sync connectors that work with schema-less JSON data and which can insert that data into JSON-B table columns, and then finally configuring Apache SuperSet to read the JSON-B data and visualize it. SuperSet has lots of visualization options. Here's just one of many examples using a scatterplot. The size and color show the C-level trends and explain perhaps where you should and shouldn't buy property in the near future. I also investigate scalability, how well do both systems scale, and in the conclusions, which you can see in my blogs with the URL there, you'll see more details about the scalability and cost comparison between these two pipelines, a more detailed evaluation of Kafka Connect and a comparison of Elasticsearch versus PostgreSQL and Kibana versus Apache SuperSet. This is part of a 10-part pipeline blog series, and it's got all the details and more. Finally, there's a new service that we're offering at the moment for our cadence for workflows and a new series that I've started called Spinning Your Workflows and Drones with Cadence, which looks at an example drone delivery service. Well, thank you very much. I hope you enjoyed that lightning talk. Please check us out at the website. We have a free trial for two weeks. You can try any of these technologies and more, and I hope you enjoy the rest of the conference. Thank you for listening.