 I just want to start out with a question which I have, which is, although obviously the data and the particular algorithms that you're using have been chosen for a specific application, to what degree do you find that the techniques and the applications that you've been using, could be transferable to other social or business issues, and have you given thought to if you were to take your particular work and expand it into other areas where that might be? Yeah I mean I think this is sort of a key issue. What is the issue? So the big data revolution around, but at least we're about 12 years behind where Google was, let's say, in terms of using big data like large-scale computing farms to assist with search. The style of work, the work load there, was such that it was amenable to this system of computing they invented called MapReduce. It's a style of paddle computation and it's very very good for things like searching, sorting, and some basic things that show up all over and over again in Google, like environments. And that world of information networks and subsequently social networks was able to absorb that. What we find in this new world of the transportation of grid, there are new kinds of computation that are required and so some of them are not just directly mappable to MapReduce like computations. So inventing these new things could actually just work across a new category of applications which I'm sure is saying for, some of it is going to be saying for transportation as with the grid type problem or even water networks whereas other things. So I think that's an interesting development. It's a category that's different. Just to add on what Balaji said, so in our domain there are two different things that we are trying to bring together. One is this big data processing machine learning but then there is this whole understanding of grid physics which is basically high performance numerical computation. How do you solve these partial differential equations very effectively do numerical computation which has never been done in the same framework before. But that happens over and over in many different industries. In fact, I think that's the sense of what we think is going to be the industrial Internet that you want to be able to analyze a very complex system along with the sensor data that is streaming in and see how this is going to evolve in short term so that you can take some control actions to run it more effectively and use your resources better. So I think fundamentally the ability to predict very large streams of sensor data and then use that to run a control system in real time is definitely something that I can see move to other industries. Of course, there are lots of issues in terms of bridging the last mile in terms of getting the connectors and everything into your system and tweaking your algorithms. But it's a paradigm shift in some ways. Just to add to these, during my PhD I worked on transportation on the sensing and data analysis and I saw a lot of those ideas transferred over to the grid because many performance metrics that you care about when you're operating a grid or transportation system or water system are very similar. There is consumers who are these individuals and you're starting to get more high resolution information about them. And I think because of these two things, the type of measurements as well as the performance metrics, of course the system itself may behave differently, but the algorithms and principles you come up with, many times you can transfer and in transportation there are many plots on reliability and so on that are rotated versions or just analogous to what I see for the power system, for example. So I think on the computation side there's a bunch of new challenges that need to be solved that will help across all of these domains. But in the understanding, science, modeling side, the principles you learn from one can be easily transferred in some cases, in some cases you need some more work. Great, thank you. So questions from the audience and we have a microphone in the very back. So question right over here first, very front and we'll go in the back. So my question is about data access when you talk about smart grid and big data. So you have a lot of independent power producers who would like to make their systems more efficient, especially if they have a retail and a wholesale power portfolio. So when you have utilities having more access to data, like at every 15 minute intervals and these independent power producers only having access probably at the end of the day. And they want to make their systems efficient. They want to inculcate both industrial internet at the wholesale level and get more customer data at the retail level. Where do you think what can solve the problem of data access so that they can make their systems more efficient? I can take a stab at it. Not that I have a good answer to this, but generally I think there are lots of so there is one one aspect of this is whether independent power producers are at a disadvantage compared to a utility that that might have this data. And I think there is a lot of regulation around it and deregulated markets so that you cannot unfairly provide access to this data to the competitive retail side or competitive wholesale side of the market. The other question which is what can be done to make this data more easily accessible? That's a tougher one. We hope more utilities and more regulators will push for that open access. And there are certainly some some states where the access to data is a lot easier than others. For example, in Texas, the retailers are getting access. It's not ideal, but it's still better than California, let's say. So, so hopefully the movement is in that direction. Recently, I was in the CPUC. I think it's called panel, where they were discussing access of data from utilities. And they asked for recommendation from everybody in the panel. And my recommendation was this private data, which is ours, which the utility also owns it, but but it's our data should be accessible to one to everybody be authorized, but also to academic researchers, you know, cooperatives want to analyze, at least in an anonymized way. One question that was raised during that panel was how about privacy. And I came back from that and I put a challenge for students in my class, where I said, Here's what I will do. I gathered a bunch of data from the web. And we had a data set where we have about 600 or 700 questions, everything about your smoke or not, your single or you have so many kids. And the question that I asked was, Can you improve on your ability to predict the answers for these questions by adding the smart meter data and the zip code beyond what I have? And I think by now we have had at least 20 different algorithms applied, maybe five students. And nobody could really improve more than two 3% on that. Then I discovered that in Europe, there is a at ETH, there's a faculty who has done research on this, and he also was not able to improve. So my contention was, to an extent, your smart meter data is much more private than you think. And the benefits you have for public availability of this information far outweigh the issues around data access and so on. Of course, this is not convincing out there, but one of the conclusions that we are seeing is I think in California, at least, utilities are going to have to release data for research purposes. And the thing that they said is they were going to average over 100 or 200 customers. And if you do that fast, then there is no point because all of it looks the same. So it's kind of a, I don't know, I'm very much about open data, open access and really the challenges in global warming and all of these things. It's such a small price to pay to get this data. Okay, now the question in the back of the room. Thank you all for the great talks. My question was inspired by the last one and it has to do with transmission. So it's kind of twofold. One is, are there any beneficial effects from this reduction in peak loads that you're showing to have at the utility level at a larger transmission scale? And as I follow on other, are you seeing the same opportunities for using big data, smart grid analytics at the transmission scale to solve these like major challenges of if we have all these wind farms in one location, how can we provide this renewable energy to places where wind or solar or whatever else isn't very accessible? Yeah, so the answer is yes. If you look at the demand response programs that are happening in the US, by and large, they are driven by transmission and wholesale needs at this point. And I think Ram showed in his presentation how 20% of the demand really causes a very big spike in the prices. And that's exactly where having more generation, which is lower cost, helps bring these prices down. So if you look at the PJM market, most of the demand response is helping bring these prices down in the wholesale market and not at the retail level. Now with big data, actually you have, you can take that one step further where you can instead of just looking at these wholesale nodes, you can do very precise optimization of the grid and start looking at local congestions caused by EVs or other new type of things that are coming into the grid. And so I think the answer is that traditionally, this is actually applied to the wholesale level first anyway. And then the second thing around wind balancing is the same thing. I mean, there could be a local congestion at a particular sink like LA may have a lot of congestion, but the solar may happen in a desert nearby and you have to bring that back in. And so one of the big use cases for demand response is actually to be able to relieve transmission congestion and provide local generation, which is at the source of consumption itself. And there was a question right down here on the front row, Steve. Question for Ahmet on the actual instrumentation that you need in order to deploy the demand response technology, which I know you're doing great with and congratulations on that. But I'm still not clear on what you actually what household, for example, would have to have in terms of instrumentation for the control side of it. I assume you need a smart meter to collect. But what do you need to send that that control signal out to turn something off? So again, the answer to that is whatever the home owner wants, you can design a whole spectrum of programs starting from behavioral or community programs where you just send an email. And if they do something great, if they don't do something, it's fine. You will just measure it on the other end by using the smart meter data and give them appropriate incentives. On the other end of the spectrum, they may have a full home gateway with switches to control their pool pumps and nest to move their thermostat up and set point up and down. And if they do that, then they don't have to worry about what to do when the price signal comes, and they can potentially get a lot more reduction and more predictable reduction in their consumption. And so instrumentation set in Oklahoma data was really impressive. Yeah, in the Oklahoma case, the utility actually ended up putting smart meters, which was part of their rollout. And they also gave a full thermostat, which was pre programmed to a set of customers. They gave to us another set of customers, they only gave an in home display. A third set of customers, they gave only a web portal. And so they did a very scientific sort of rollout. And you could measure how much reduction you would get for each subset of these customers. And clearly the more automation you had, the more reduction you were able to see. So, Steve, there's a question right there. And then sort of question back there. Okay. Why don't we go with the back and then we'll go with the side. I would like to know, with respect to the Singapore train station, what was difficult about integrating data from the actual train? It seems like it would be incredibly useful to know that train number 265 was delayed because the door was sticking or a wheel was bearing was failing. And that was the root cause. Why didn't you have that data too to integrate with your other data? I think this is available, but not easy to get a hold of. And the reason is that the way the system is set up usually there's an operator who runs a concession. And it's a question of, did you get the, are they also the ones running the smart card system? So this is all running on just a smart card tap in tap out, right? And it's a question of who, you know, you're working with to be in with. So for example, in the Bay Area, the train operator BART and the clipper card entity are not the same. Okay. The one that runs the clipper card is a separate entity. Cubic is a system. And this is all very telling because what's happened in transportation is the ones that run the trains are not the same as the ones that run the buses. It's not trivial to know, you can't have a single company on both. The expertise required is different. Okay. And the electronic overlay that comes on top of this, usually there's some vendors and then there's system integrator and then there's a system. And then the card fare collection and stuff is just done by somebody else typically. This is true in Seoul. This is true in every pretty much most big cities that you can think of. Just a quick follow up observation. I think that's, you've captured the key opportunity that a group like this has that these systems are always going to be heterogeneous in the optimizations that we can benefit from or at the next level up above whatever you were working from. Yeah. And right now it's a difficult problem as you just elaborated. But I think it's a solvable one that would be worth exploring. So this is a good point. I'd like to elaborate on just one aspect of it. If you just think of how the computer networks world was in late 80s and early 90s. If you just open a random research paper you'll see seamless interoperability. Okay. This is a phrase that every paper had to have in it to be relevant to the big topic of that day. Because the problem was we didn't know at the time how to get the wireless and the Wi-Fi system guys to talk to the telephone guys to talk to the data networks guys to talk to the ATM forum. They're all different standards bodies. Okay. They couldn't even agree on packet formats. And the big revolution that has happened which is the sort of opening out of the internet is there's only one global internet and there's a pretty much only one local network which is Ethernet. All the other guys just like went away. Okay. And still the question of seamless interoperability is you know you have Wi-Fi system and then soon you're in a wired network in the building and then you go to this suite hall which is where our router is and you take the wired internet and then you may take a satellite somewhere else and as far as the internet as far as your session is concerned it doesn't even care. Right. Commuting is multimodal. A person takes a bus and then takes a train and then takes a bus. Okay. Either the first or the last mile is usually a bus trip in many metros. The what's happened at the level of information there's a uniformity. It's the same smart card that they use for for paying the bus fare as well as the train fare. Like in the barrier now you can do part and muni the same clipper card. So it's an information layer where the commonality is emerging. Okay. And which is why that's a good place to sort of build your system around. And this is a well-known idea in like distributor systems you just find the common interface that is common to lots of systems. Finally maybe transparent systems are emerging with one. Stay close to the money. You will always find a commonality there. So this will probably be the last question. So I have a quick question for everybody. Both in terms of the energy industry and the transportation industry weather comes into play and affects load patterns or transportation patterns. And we talked about accuracy being very important forecasting accuracy. Is weather forecasting accuracy good enough at this point or is it a road block for accurately predicting transportation and energy load patterns? I'm not the most recommended person to talk about weather forecasting accuracy and how it impacts forecasting but talking to this team from Sailor's Energy who does a weather engine. One of the things that they mentioned is in their day ahead if they can predict the temperature within one or two degrees that doesn't affect the wind forecast anymore. From the load forecasting perspective what I learned doing these things by myself just based on time series was even some simple time series predictions were enough because the part that's tied to the weather on these residential and commercial loads there's still another layer of variability coming from your behavior that you add to that. So it's hard to say but this person from Sailor's also mentioned that for example that going in these models there is a bunch of layers and being able to model better all of those layers is a challenge and if you can do those type of things you can do better. There's something interesting at a contrasting view from transportation. It's all outdoor weather for transportation pretty much. I mean not how you what you feel at home is actually what will you make a decision to drive or take the bus off. I stand for people bike except when it rains and they remember that such thing as a car and it takes them about two months to abandon the car and get back on the bike. Two months after the rains have stopped. But the thing is that it's slightly more severe weather that seems to have an effect on transportation except the funny thing we found about light rain causing people to do these things and it looks like it could be a phenomenon of central business district type work. You know people working there not wanting to get their business clothes looking you know wet or something. But something interesting as well as transportation is very heavily affected by severe weather events of course as is the grid. And what tends to happen is the proper gation effect which I'm sure is the same in the grid. And that is actually what learning from this emergency response systems and stuff rely on the very network which is now snout. So after Hurricane Sandy there was a really bad problem of just getting gas. You remember all this like people running out of gas. And that for example is a kind of dependence that's worth understanding because there are more of those happening now. I mean I have a slightly different perspective on this and that I think that's an opportunity for big data to also come in. So whether forecasts are obviously improving and they're not as bad as they were like 20 years ago but there's still a lot of inaccuracy especially unless you put a specialized sensor or something at your site. But that's where I think the power of big data comes in. The fact that you can update your forecast as you get new input at a very massive scale allows you to react much more accurately to these weather changes which are somewhat inherently unpredictable. And so I mean we in our systems we can we can basically take any weather input and as soon as it changes we will recalibrate and reforcast. And then obviously the closer you are in terms of time horizon the more accurate you are and over time you can get pretty I mean you can get very inaccurate. But typically between 24 hours to 8 hours you can cut your errors down by half to two-thirds and then within eight hours you start getting pretty good accuracy. Okay if you could join me in thanking colleagues. I'm wrong for their presentations.