 Hello, my name is Andreas Klostermann and in this talk I will talk about brainwaves and how we as Python hackers can use our tools to explore our own brains a bit. Last year at European Python I held the 1.0 version of this talk. This year I have only 30 minutes so I'm going to make extremely short thrift of the theory behind brainwaves but we have to cover a bit of it so that every one of you understands what we are doing here but the talks don't have a lot of overlap so this is really 2.0. Now first of all what are brainwaves actually? Brainwaves are an electro physiological potential that you measure on your scalp that basically is a voltage measurement which is measured over time with electrodes that are on the scalp and what you are measuring there is the summed potential of most or even all your brain cells which give off electrical signals to communicate with each other but also a bit of waste energy gets transmitted outwards and so you can measure it on the skin without having to go inside the scalp. Most of what the most useful analysis of brainwaves usually is the Fourier transform which basically only means that we assume that the signal is composed of several frequencies and we want to know which frequency is dominant and by doing that we can infer something about what the brain is doing because we know okay when a certain frequency range is over represented then the subject is for example concentrated or relaxed or that kind of information. Now to measure this current we need an electroencephalogram and these devices can be quite cumbersome and quite expensive but what I have here is basically the Neurosky Mindwave mobile which is a Bluetooth connected headset really low powered and very optimized for developers and hackers. It does all the amplification stuff inside this headset and digitalizes the data. It also does some preliminary analysis of the data so that you could even connect this to Arduino and the Arduino would be able to tell if you're concentrated or not. When Arduino's are not really sophisticated processors are software in any case so it's quite nice. I wrote a physiology framework and that is a framework for physiological experiments and analysis. It currently mostly only supports mindwave EEG but eventually I also have code for ECG from Vitalino but that's not activated currently. It's powered by PsyPyData and it's IPyzen Jupiter enabled so what I want to achieve is an experimentation platform that works for do-it-yourself people who don't have a lab fancy equipment or something but I want to use mainly the do-it-these quantified self type devices with Bluetooth connections or other near field stuff. It's currently Python 3.4 only and I'm making heavily use of Async IO. Now you've probably heard of the Internet of Things. The basic idea is that you have local computers connected to some kind of sensor and there's a pattern that I see emerging in these data acquisition applications so you have a local component where it very much matters whether the computer is like a Raspberry Pi or an Arduino or something because it has to be right next to the device because of the low range of the sensor then there's a cloud component which actually analyzes the data and that does not matter where it is just that it is connected over a network and the user interface is connected also to cloud and local and displays the data and user can interact with the analysis and in my case the local component is just an Async IO data server that I've written it communicates over Bluetooth with this thing here and it also babysits the connection so if I switch it off and switch it on again then it will reconnect and try to fix errors and that's quite nice to have that separated from the iPython kernel which is the cloud component because it's difficult to deal for the user to deal with all the exceptions that occur on Bluetooth and we have most of the data processing inside the kernel then and the kernel pushes data to the user interface which is a browser so in an iPython notebook like this one you have the server, the kernel and the user interface and they are somewhat separated but you need a server for Python anyway now I'm going to quickly show you a real-time demonstration so this is junk data because nothing was connected and now if we are lucky now this is real I will be a silent for a moment though that you can appreciate normal brain waves the problem with EEG also is that the brain waves themselves are so so weak that pretty much everything is going on inside or in your head all the muscles the facial muscles and eye muscles all contribute artifacts to the data and are stronger than the actual brain wave data I can show you that I can clench my teeth which I like the strongest muscle in the head I can also use my eyes to and when I blink and that makes EEG signals very difficult to analyze also this is a bokeh graph I'm using iPython widgets to push data from the kernel to the notebook and also the kernel it's running an icing IO loop internally so that it can communicate over web socket with my data server which handles the Bluetooth stuff and now I have stopped and then I can go to the next slide now let's do some data science data scientist is who data science does so we need some data for that and to to get good data we need experiments because discovery requires experimentation and this is a central idea in the physiology framework that is this experiment stuff the experiment is here initially initialized and then immediately used as a decorator much like the iPython notebook interact one and this decorator will when when I started it will wait for data from the server and whenever there's new data this handle message is called and it can then dust it can then do its thing to do whatever in this case it just display clear output means the output is cleared and then some HTML is written with the last attention data so attention is the value that is computed by the newest guy minefield itself inside the headset and I can show you how that works and that is really all it takes so this is a value between 0 and 100 currently it seems I am totally inattentive but I can push that higher well sometimes it doesn't work that nice but we don't have time to do that now experiment classes have these attributes like attention as a time series so every second or so the such a value is computed by the device and send over Bluetooth and I try to figure out when that was but it's a bit difficult to do that and in any case the pandas time series stuff is really good and so it's just accumulating in this experiment also the raw data the meditation data and different several other things are also come communicated and stored now that was one thing running experiments in real time but we also like to record data that is done by giving it a file name and that is a hdr file file in this case I think I have a bug because I have like one or two gigabytes of data and that's it's completely crazy I have to fix that later in any case this is the same as before but it displays the amount of raw data which is here Len experiment raw over 500 12 because the raw data is at 500 12 samples per second and now let's record some raw data I think it's not working currently in any case now to resurrect the data we need a so-called batch class and the batch class can resurrect the data then and here we have recorded data from another session which just is much the same as the experiment I forgot to tell you what actually a batch is every time I run the experiment I record a different batch so first time batch zero second time batch one and that way experiments don't overwrite their own data and you can also try to vary the experimental conditions if you want to do that now I'd like to show you an expert simple experiment I don't have time for a lot of them so I'd only do one so the new sky mind wave computes these essence meditation attention values they are sort of patented and sort of back box algorithms where they don't really tell us what they are doing there and here I'm transforming the time series data to a table of features so I don't have time to really explain what's going on there but I take windows of sort of windows of the time series data then I compute how strong different brain wave frequencies are and help in the same row I also have the raw the tension and the meditation values I wrote separate library which is table cleaner and we need that to clean some of the data up because some of the data is bad as I told you before we have lots of artifacts in the data and we don't want to analyze artifacts we want to analyze real e.g. data so this table cleaner library is inspired by jungle forms and cleans tabular data of nice thing is it outputs both a validated data set and a data set of the errors so on the bottom we see a table of the grouped validation errors and their reasons and the most rows 206 rows were deleted because there was too much variation in the e.g. data so in the e.g. is when there's an artifact then we try to remove that that that row of the table also poor signal is computed by the brain wave itself but that's not so important in this case so if you need a data validation thing maybe you should look into it now I'm doing the linear regression analysis so I said we have features of brain waves correlated with attentional meditation data and the linear regression all it does is it tries to explain the tension value or the meditation value in terms of the frequency strength so it multiplies each frequency with a coefficient and then they are all summed up and that way when you figure out the coefficients you know how important the different frequencies are for this value and you can't probably can't read the labels on the graph but I didn't have time to make that any better what what you may actually see is that we have here the the best or the highest and most significant coefficients are at the what was that are in the high betas or in the beta frequencies from 17 hertz upwards you can't see that but just just just believe me when I tell you that there's a certain range of frequencies which is called beta frequencies between 17 and I think 30 hertz depending on the definition and they are strongly associated with all sorts of attention and concentration stuff so if you ever heard about the prefrontal cortex and the tension circuits of the brain also ADHD that has a lot to do with with beta brain waves now I'd like to meditate a bit about meditation there's a similar graph but we have more going on here we have the alpha values which are between in this case are between like eight and ten around eight and ten hertz which are significant but also some beta values which are a bit undecided I think that's because most of the time I or others too need to to do some concentration to enhance alpha values and during waking state you need to have a somewhat calm mind to actually exhibit alpha waves or you close your eyes which is also easy but that's not what we want to do in any case these values are used in newer feedback so you can write an application that looks for attention or meditation and feed it back to you and then you can learn how to consciously manipulate these brain states that's what I was talking about last year at Europe, Pison in Berlin 2014 and there's also a YouTube recording of the talk which where I go into more detail about the psychology and biology behind all this newer feedback stuff so this all sort of implies so the linear regression implies that the strengths of the frequencies are not dependent on each other and I don't know why that I'm not going to get it to work any cases these are correlated and we see that the alpha and the beta values are correlated among each other but not between the alpha and the beta waves and well it has some biological meaning but I'm not going to go into that I'd like to make some technical remarks bluetooth is really bad for timing so if you have bluetooth data coming in they don't come often often don't come with time stamps and these time stamps if you have to give them time stamps and the local component in the server data server and figure out which are the right type stamps and if you do it wrong then you have like overlapping data and crossing your own timeline is strictly forbidden except for cheap tricks um yeah I think I was very good for waiting and problem resolution so you can do like yield from weight and or yield from sleep and so I do so I can write these troubleshooting algorithms with reconnecting and I can wait for a few seconds and then reconnect again without disturbing anything else um the combination of bokeh and notebook widget really rocks because this notebook has its own web socket push communication channel which I can use to stream data to the javascript site and update the bokeh charts and in general Jupyter notebook is a really great tool to tell computational narratives I rather skip that or rather went hastily through my little computational narrative about the brainwaves and about attention and meditation but I didn't really have time for more but you can imagine that you can do a real-time experiment or show how the computation works and also explain or narrate how the data really works also it's a lot is the the pathology framework is an example of a library for the notebook which really is bigger on the inside so I just had this experiment decorator and entitles hides all this web socket and async IO and other magic uh mostly from the user so I'd like to thank you for your attention and my Twitter handle is page in horse which was a few years ago supposed to be a pun of trojan horse or something but now it just sounds silly my library is by theology I just published it it's very raw and you probably won't get it to work especially unless you have this device by the way resistance is futile um this table cleaner library I try to make it a bit bigger because I have had a lot of trouble over the times validating CSV data and that is sort of my attempt to fix that problem in any case I wrote also this notebook assets library which I use for for turning coffee script into java script and serving it's from the actual library itself and not from the extension stuff which I find a bit troublesome because the iPython uh notebook profile directory has no clear one-on-one uh direct uh connection to the the Python library installation stuff and it's basically all thank you for your attention and we have time for some questions maybe but you when you want to go to the lightning talks okay I tried to do some stuff with uh with the mine wave years ago uh but I'm guessing that your framework didn't exist then because then I had to connect via some some proprietary bluetooth executable that I had to wrap stuff up around so so does your python code do the bluetooth stuff yes also uh the current documentation for mine wave points to my older libraries for minefave so um it's a bit difficult to do and I think what the new sky stuff is doing isn't really that developer friendly on the library side but the protocol is very friendly any other question so uh if I want to get in one of these nearest guy things is that I just had a brief look at the web page is the new sky mine wave mobile which is the one to get yeah uh yes I think that's the best device there is one that is a bit more expensive and more variable but um I think the new sky mine wave mobile is the best one for like 100 or 120 euros currently you can also get it on amazon and other shops and you only need the device itself you don't need any of their software or anything you just use not at all okay great some people even via Arduino directly to the um sync your ship so as we as we have some time um would you mind to show the code uh for the demo part maybe the demo itself um yeah by the way this I want to publish this the notes but I wasn't able to um to push it yet from this network and it contains a lot of notes um of what I was talking about which are not visible in the presentation itself um so here's a hidden uh code which is a real-time raw demo and but to to know what's going on here's this normal bokeh um let's make a plot stuff and then it's the raw source bokeh model source column data source which is um important to have a reference to it and then touch bed um now in the handle message one I I'm doing some resampling and then I push the data other than I convert the data to this raw source data on this Python side but the bokeh knows it's uh these bokeh objects know the id on the browser also on the Python side so I push this id here I have this nm replace bokeh data source nm is a widget uh which has a communication between the Python side and the notebook side and it I have just this function replace bokeh data source which pushes this um this data source with its id and the new data to the javascript side and then it deconstructs it and puts that into the uh data source into the bokeh data source on the browser and then magically it's through uh different callbacks the the graph is just redrawn that's just because okay one moment I have to reconnect again um we have nothing here in this talk thing which I sometimes problems on this computer with the connection but okay now the connection is made I'm asking to show the demo yes now it's a really yeah it's it's relatively primitive I'm so excited I work on bokeh too is it working now? think something that's okay what is it um what are the loans me? I can't speak because when I speak I know I have artifacts can you morse code me the answers to my first question? yes it's actually an easy abuse of this technology to do morse coding like I can also blink