 This talk is about your tpy, your py with Python, Streams, HTML for Streaming applications. So I think you can just start sharing your screen and we should be ready to go. So for everyone, please remember, so this talk is 30 minutes and after the talk, at the end of the talk, we are going to have a few minutes for Q&A. So you can click in the Q&A bottom and write your question and I'm going to ask it later. And after the talk, we have a Discord channel where you can continue the discussion or if you have any questions. Okay, so thank you very much. So this is an invitation. It's an invitation to help build a free open source Python package so that everybody can monitor, analyze, and respond to streams of data. My name is Mani Chandi and my co-author is Deepak Narayana. Each sensor in the Internet of Things generates a data stream. An airplane generates a stream of its locations. And the microphone generates an audio stream. Social media generates streams, Twitter generates streams of tweets. Now we have technology that can help us analyze streams. Inexpensive onboard computers such as the Raspberry Pi loaded with Python are connected directly to sensors. Powerful libraries such as SciPy can be used to analyze streams. The cloud is another resource for this variety of Python API services. So would you help to build a free open source Python package that everybody can use using free or inexpensive technology to solve this important problem? So I started working on this many years ago and you can see some of the work I get out. And I started IoT Py because I had worked on many applications with similar structures. We were rewriting rather than reusing code. One of the applications is the Caltech community seismic network. CSN is a team of Caltech scientists and engineers, some of whom are listed here. CSN has deployed about 700 inexpensive accelerometers all over the Los Angeles Unified School District area. So each red dot represents a school and a sensor in that school. I want to emphasize the community aspect of the community seismic network. You'll see that school administrators, students, parents, indeed the entire community, can benefit from harnessing streams. I'm going to show you a video, but you may not be very clear. So this video was created by Professor Rob Clayton at Caltech, which shows accelerations in schools. So each dot represents a school and the color represents intensity. Now, this kind of video, this kind of information about how the earth is shaking in schools, is immensely valuable for everybody. I've had the opportunity to work with many groups working on streams, analyzing tweets, a home hazard station, detecting radiation, structural health monitoring, and all the stream applications have aspects in common. They have threads that interface with data sources or actuators. Streams are processed right where they're generated on onboard computers in shared memory and in the cloud. All these applications acquire and process streams. So IoT Pi creates a common abstraction across this variety of stream applications. Moreover, we hope that this abstraction blends with the way you've programmed, blends with Pythonistic programming. Now we have two problems to solve to get a package that everybody can use. Firstly, Python's libraries weren't written for streams, and streams that process using different types of concurrency, threads, shared memory, message processing, these for everybody should offer a unifying approach. Let's look at the first problem. A typical Python function takes input data such as an array, a function like an invert matrix, takes the data, and generates an output. A inverse and then the function stops. By contrast, what we want is something persistent, something that lives forever. We want an agent that acquires data continuously and produces data continuously. So the abstraction for an agent is very simple. It's a black box, actually pink in this example, with inputs A, B, and outputs X, Y, Z. And everything here is persistent. It lasts forever. So for example, we might be able to instantiate this agent by calling a Python function F, with this example in streams A, B, and out streams X, Y, Z. Now here's the tricky part. So the agent is persistent, and it's inputs X, Y, and this example is persistent. And associated with the agent is an action. In this example, the action is to make Y equal to G of X. So if the agent's goal is established, in other words, if Y is equal to X plus one, in this case, suffixes one and Y is two, the agent's goal is established, so the agent is quiescent or idle. It's got nothing to do. If however, the input changes, so for example, the input is two, then the condition is violated, because Y is no longer X plus one. So the agent becomes active. And when the agent becomes active, at some point in the future, an active agent reestablishes its goal. So in this case, it changes Y to become three. The problem is that we don't know exactly when an agent will take a step. So for example, if the input is zero, one, two, three, four, and the initial output Y is zero, you would expect the output to go zero, one, two, three, four, five. However, if the agent only sees inputs two and four, the output will be zero, three, five. It'll skip some steps. What it means is that if you have two identical agents, Z and W, both of which are computing Y plus four, they might produce different outputs. So the tricky part, and we'll talk about this a little later, is to use streams to obtain desired outcomes, whatever the desired outcome is. Sometimes it's non-deterministic, sometimes it's deterministic. You can use theories developed by J. De Mestre and also ideas from Kishap and Gali. Now, I'm going to slow down and talk about a very specific kind of shared object. The shared object could be anything. It could be a graph. It could be a simple stream. But let's look at just a single stream. So you can think of a stream as a tape. An agent can read the entire tape, but it writes only at the tail of the tape. It keeps writing to the tail of the tape. And as I said, when this input changes, an agent becomes active because output doesn't match its input. And then sometime later, the agent takes a step. Here is an acoustics example developed by Deepak, and it produces a shimmer in an input. So the sensors and actuators are shown in ellipses. So for example, the top right, we have a thread of interfaces with a sensor, a microphone. It produces a stream called spoken, and this is a stream of data values from the microphone. The agent, the echo plus spoken gets two streams, spoken and echo, and produces a scene called heard sound, which then feeds an agent called pitch shift, which outputs a stream called pitch change, which also feeds an actuator, in this case, the speaker. When you go from a diagram to code, the diagram is the code. In other words, just write a function, which exactly represents an agent. In the top, we have a function f, which takes inputs y and z, produces an output x, and we create that agent just by calling f. Now, f is this agent is now going to live forever, and we expect the inputs and outputs to live forever. So as long as the microphone is running, the agents exist. When the microphone stops, and we declare that there's no further input, the agents will step themselves down. Now, a network of agents could be an agent as well. So for example, we can create a network, an agent GH, by combining agents G and H, and the combined agent GH has input x and outputs y and z. Next, let's look at how we use Python to create agents, because the key idea in all of this is to harness the power of Python's libraries to create these persistent non-terminating agents. So we start with a function f that operates on a bounded data structure like an array, standard Python function, and terminates. What we're going to do is to wrap that function with another function G from the IoT by library, and wrapping f in G creates this persistent agent. So let's look at a simple example. So here, we're declaring streams, S and T, happens to be, in this case, a NumPy array. So think of the stream as a NumPy array of arbitrary length that goes on forever. Each element of this NumPy array is itself a two by two array, in this case an array of floats. So we've now just declared two things that go on forever off type array. Now here is an example of how we wrap my Python function with an IoT by function to create this persistent agent. In this example, the function we are wrapping is matrix inverter that we get from NumPy, that's the NP linear algebra invert. So it's wrapped. And now we've created a persistent agent with input stream S, output stream T that expects to get a sequence of matrices that produces a sequence of inverse matrices. And this will go on as long as data comes in. Now IoT Py has a library of these agents and library of these functions to create agents. And what I hope we would do together if we collaborate is to add to this library, make it richer. Here is another way in which you can use IoT Py. So instead of having explicit core, you can use a decorator. Here's a decorator called map W also from the IoT Py library, it's map window. So the undecorated function subtract mean is subtracting a mean from a window which is an array or a list. It's just a regular call, call the array, get the result. The decorated function by contrast is now going to operate on an endless stream. So if you look at sequences of windows that slide with a step size of one, and it's going to keep producing output for as long as the input remains alive. Once again, I hope you would help add to these decorators. This can have state. So in this example, here's an agent where function f on the top left takes an input element of the input stream, and the agent state. It returns the output element of the output stream, and the next state of the agent. So for example, the state of the agent is the last value read. So for example, if the input is 14916, and the state of the agent is going to be 14916 and so forth. So that the output in this example is going to be the difference between the current value of input. Because the state remembers what the previous value was. You can also have a function f that's wrapped and this function could be a method of a class. So you can have very rich functions with all kinds of state information and other information that gets wrapped by an IOT by function to create this persistent agent. You can also use streams in a functional form. In this example, w, x, y, z, D are all streams. You can create the by using this functional form. So exactly as written. These functions f, g, h of functions from streams to streams. And once you write this equation, we now have persistent agents and persistent inputs and persistent outputs that keep living as long as the inputs are alive. Now let's look at the second challenge. We've got so many different ways in which streams are processed, threads, shared memory, distributed memory. And what we would like is a uniform way of handling all of them. The way we do that is think of it in just one way. You always connect an output to an input. That's the only way we go to think about concurrency, whether it's a thread, shared memory, a message password, whatever. So for example, let's look at how we would deal with a shared memory process in a multi-core computer. Once again, a box, in this case the top box f, is just a black box with inputs and outputs, in this case inputs y, z, and output x. It happens to be running as a process as opposed to an agent in a process or a thread in a process. But as far as we're concerned, it's just another black box. And we define this agent in this example if we're using multi-core as a dictionary. Let's look at that dictionary. We'll rename the process, v0. It's got an agent function f, and remember f could itself be a network of agents. It has inputs y, z, and output x. We similarly create a process for f, for g, and h, and that's it. So we don't really have to think separately, oh, this is a multi-process computer in a multi-core system. What happens if we have a distributed system? We would use, for example, the advanced queuing message protocol, implementations, and pica. The point I want to bring back to you is, what you've got to think about is connecting an output to an input. In the background, we've got all these things going on. Here is snippet of code to publish a stream. So you publish a stream to a publisher. Here is a pica publisher, specified by routing key and exchange with a local host. But what do you have to do? What you've got to do is say, publisher.publishY, where Y is the stream, and the stream goes out. Similarly, for receiving data from a stream. It's important to realize that IoT Py adds to your programming style. It's not like you stop programming in procedural style or object-oriented or functional, or whatever style you like best. You mix and match agents with your style. So you use IoT libraries and agents only when you need to. IoT Py can be used for very different applications than looking at sensors. Here's an example where you're simulating temperatures in an object over time. So the temperature at a spot in the object depends on the temperature of its neighbors and is changing over time. This is a science simulation, not at all like getting sensors from a microphone. Yet, the key to using the same structure to solve this problem is to realize that the temperature at a given point is just a stream. The temperature changes over time, and that's a stream of values. IoT Py has a collection of Jupyter notebooks that I'd encourage you to look at. That's the best way to see what IoT can do and cannot do. We have extensive documentation in a website called assemblesoftware.com. It tells you what IoT-wise goals are and what its limitations are. Now, this very short presentation leaves out a lot. For example, it leaves out a lot about the previous work, and there's an immense amount of work that's been done. Moreover, there's been so much wonderful work done on streaming frameworks, most of which though has to do with streaming frameworks in the cloud. I haven't talked about timing issues, which are critical for many applications. But we'll have to discuss that at some later site, later time. Here are some books that I've worked on on streams and parallelism. I worked with Roy Schulte of Gartner, very lucky to be able to work with him, on event processing. And that book has lots of examples of streams and agents. There's also a website that I've worked with. There's Mr. on the Parallel Aspects of it. That's a parallel program design book. And the communications ACM picture on the right shows a paper by Michael Faulkner at Caltech using your phone as a quick detector. Finally, I'm going to wrap up with an invitation to you. My goal today is very specific. I want to help create a free open source Python package that everybody say a high school student, anybody anywhere in the world can use free technology or very inexpensive technology, a $2 sensor for this vitally important class of problems. Please look at these can put a simple software by a deep eye and look at read the docs for documentation, and you can download using. Bye bye. Thank you very much. And we can play. Glad to take questions. Thank you. Thank you. Okay, so we have time for a few questions. So if anyone has a question you just can raise your hand and I can enable the Microsoft and you can, your microphone sorry no Microsoft. So we have the first one. You see saying, a lot of what you presented reminds me to the reactive X party. For example, the library. Erics pie. And he's asking was it was it a source of inspiration. No, sorry, so there was a lot of work done on reactive programming and reactive programming is certainly the source of inspiration. Thank you for telling me but I like spy I didn't know about it. So this is valuable information to me and I'd be glad to look at it. The source of inspiration was more the theory, as well as all these applications, the seismic network, radiation detection on Twitter. So any other questions. It's also possible if I can open the microphone, the microphone for anyone that wants to ask questions live. Okay. We'll have one more. So Artemis asking. What are you pie can already do. And he's asking if, can it be applied to home sensors, like a door or a window, that kind of things. Yes, yes. You can download party pie and play around with it. You can play around with the tube to notebooks. And then it works with Python 2.7 and above. So it uses shared memory for multi core computers like we have Raspberry Pi with four processes for course. And 2.7 Python. You can use IOT pie for shared memory on your Raspberry Pi and hook your Raspberry Pi to sensors. So that's all ready and set up. And of course you've got to connect whatever sensor you want to use. Okay. So I think that that's all. Thank you very much for presenting speakers are the most important ingredient for a conference so thank you very much. If anyone wants to continue the discussion in discord is a channel the channel is talk IOT pie. So you can go there and ask more questions and have a chat. So thank you very much. Have a nice day. Thank you.