 Ladies and gentlemen, we have here a talk by Sebastian Stax, do I pronounce this well? Yes, Stax, Stax, Stax, I have to pronounce it in German. And he's related to the University of Aachen, he did a PhD in physics and he was in a team that developed a fantastic application, as I mentioned earlier on. He developed the app Fifox, do I pronounce this well? I would say Fifox for physical phone experiments. Okay, yeah, yeah, of course, I'm sorry, I'm not in that kind of department. But this application actually gives you other possibilities of the usage of your smartphone, really extending certain borders to my opinion. So please give a warm, warm welcome here to Schleffa. Okay, yeah, thank you, thank you for the introduction and welcome everybody to my talk. Yeah, as you've just heard, I'm a physicist from the RWH Aachen University, where I developed the app Fifox. Fifox is an app for those of you who do not know it already, that uses the sensors in the smartphone for physics teaching. So the idea is that students can use their own phones to do experimentation in class, in the lecture hall, so for schools and universities. I should explain that in contrast to some other talks by me. This one will not be that much about education, because this is the Cars Communication Congress, and this is the hardware track here. So I try to tell you a little bit about the app, a little bit about the sensors that we have in our phones. And yeah, I would really love to get into touch with some, especially people from maker community and from open source communities, to find some connections, how we can get many open source projects together, because we've got so much feedback from teachers. And I think I could also use some feedback from other developers as well. So I would like to start with a short explanation of what we actually do. So yeah, so said I come from a university and there we have this introductory lecture for physics students, which is called Experimental Physics 1. And this typical lecture looks like this. We have a fence in your lecture hall by now, but the situation is the same. We've got 300, I think 270 students this year sitting in the lecture hall and doing no experimentation at all. There's only one guy experimenting and that's a professor. And the students are sitting there and enjoying the whole show, like they would enjoy a YouTube video, and maybe they're might even used if something goes wrong. And we thought we could change this by using the sensors in the smartphones. We're not the first ones with the idea to use the sensors there, but for some reason we decided to write our own app, which turned out to be quite successful then. So in contrast to the old version, where students just had a look, they now get assignments where they can do their own experiments with their own measurement devices. And to give you an idea of what this looks like, I would like to start with the first experiment, which is about centrifugal acceleration or centrifugal acceleration, depending on your frame of reference. So the idea is from a rotation movement, we want to measure the radial acceleration as a function of the angular velocity, so the rotation rate. To do this, we take a regular smartphone, this is an iPhone 8 in this case, and we put it into a salad spinner. Okay, so we get some rotation in there. And let me just place it in there. And so it's not important, but it sounds nice, I have been told. So here we get the live data from the phone. Already, acceleration on the y-axis and angular velocity on the x-axis, if the salad spinner is actually moving. And what you see is the faster I rotate the spinner, the farther on the right you get your data, because that's angular velocity. And also the radial acceleration increases, if I'm not going too fast, because then I do not get any data at all anymore. Let's slow it down again. And we can fill up the gaps there by going really slow and filling up this part. And in the end, so here has a physics background. Okay, more than expected, great. Because those of you who just raised their hands should not be surprised that we expect a square relationship between the radial acceleration and the angular velocity. Those of you who do not know will believe me from this plot, where on the x-axis we've got the angular velocity squared and on the y-axis the radial acceleration. We get a straight line and that's what we would expect. So besides the physics, because this is not that much about the physics, this is a simple experiment all our students could do. And actually, we gave them this assignment. We gave them also a bonus point if they created a video. Don't worry, their consent that we used the video was not related to the point. They first got the point and then we asked for their consent to use the video. And we learned two things from these videos. A, our students do not really have salad spinners. They've got bicycles and office chairs. But B, and that was the most important thing. It looks like, I mean, these are from this year where we've got almost 100 videos. We actually could trigger them to go out, search for something where they've got a rotationary movement, and they could repeat this experiment. Okay, another example, which actually changes the course of the lecture a little bit, is a situation where we first give the assignment before we actually discuss the theory about this. Which means, in this example, this is a little bit older because we did not get there yet this year. We assigned our students to build string pendulums. They looked very similar because we were very precise about how they should build them. And then we had an online form where they could submit the length of their pendulum and the frequency they received from it, they measured with their pendulum. They should do this for three different cases. And the idea was that we did this assignment long before we discussed the pendulum in the lecture, so that they have got a little bit of a research experience. And after we collected all the data from then, then in the lecture we discussed the pendulum. So the physicists were then now, we do a small angle approximation, solving differential equation, all this theory stuff, and in the end when we were done, we could tell our students, we do not have to do this experiment on stage now because all of you did this experiment and we simply can compare the theory that we just derived with your data, and it worked out quite well. So you see most of the white points, which is the data from the students, matches the theory, which is the orange line, except maybe for those three who should proceed on a career of theoretical physics. But yeah, so this is all something you've got nice feedback from, and this is in principle how we use the app and what it's designed for. There are also, of course, many applications in school, by now more teachers use this in school than we use it at the university, so we take this into consideration as well. But that's the reason that I'm standing here talking about the sensors on the smartphone. That's the reason that I am trying to access them. Okay, so let's have a look at what sensors we actually have on our phone. I think the first one that most of you will think of when talking about sensors, besides obvious stuff like the microphone, would be the accelerometer. So I think I first explained how the accelerometer works, okay? So the accelerometer in your phone is actually a so-called MEMS device. A MEMS is M-E-M-S, stands for Microelectromechanical System, and it looks roughly like this. It's a simplification. If you search for actual MEMS devices, simply search for M-E-M-S and accelerometer, and you find some pictures, they usually are a little bit more complicated, although the accelerometer is not that much more complicated. It consists of an orange case, so far so obvious, but also two contacts, the blue and the red one, and the important part is this silvery structure here, or the metallic structure, which is under-edged. It's a little bit hard to see on this picture, but it's actually floating. It's only attached to the sides of the light in between here a little bit. So if you move around the accelerometer, the inner part can actually move. So let's do this. So at each point where the device is accelerating one direction or the other direction, due to the inertia, the metallic part in here is distorted or moved into one direction, and we can measure the amount by which it is deflected by this movement with the two contacts by measuring the capacity between these structures. So that's the principle of the accelerometer in there. One thing to mention at this point is that in the sense of physics, it does not really only measure acceleration. It measures acceleration, you see it in this image. If the device is accelerating, we get some data, but if you imagine we take this device and rotate it like this, then of course you also get a deflection of the metallic part by gravity. So gravity is pulling it down as well, and that's the main reason that the accelerometer is in there, because the developers and manufacturers of the phones are not really interested in measuring acceleration. At least there are not that many use cases for it, but instead what they want to have is an indication on which direction is down or which direction is up. So when you rotate the screen of your phone, actually they can rotate the content of the phone as well. With this you can also control video games by tilting your phone and stuff like this, because gravity also deflects the accelerometer, so Earth's acceleration which I tried to avoid because from the dectics point of view this is a nightmare to distinguish these both, but the point is that we can detect rotations like this. And this is pretty much in every phone. I mean this is not really a statistic, this is just the first pie chart we have about the availability. I have never encountered a single phone or a tablet that does not have an accelerometer, so if anyone ever encountered some special device, some very unique device that doesn't have one, let me know because I would be interested in this. At least I do not know of any device on which Fox actually runs which doesn't have an accelerometer. A bit more interesting is which data rate we can achieve. So most accelerometers have several hundred samples per second. Actually the fastest ones go up to 500 hertz, but there are also many devices that only do 100 hertz or 100 values per second. These are mostly the cheaper Android devices and all the iPhones. So I think the internal accelerometer will do more on an iPhone, but I have to admit at some point I can understand why they might limit this, but on an iPhone you get 100 hertz, that's the limit from the API what you can get there. But this is actually quite a lot, I will later see what we can do with this. And one other point about calibration of this thing, actually for all the sensors you get reasonable units from the system, so the acceleration is given in meter per square second. I just realized that I forgot the units, that's something I would really tell my students. So on the x-axis it's in meter per square second and you see that there's a wide range of values that you get there, so this data is from our sensor database, I will mention it later as well, this is a contribution from our users, what data there is. This is only absolute value that we get from resting phones and we would expect 9.81 meter per square second for Earth's acceleration. There's some local variation, but not on that scale, so do not expect your sensors to be well calibrated. Also if you've got any app that tells you you can push a single button and then it calibrates your sensor, don't trust it, it's not that simple. These sensors may have different errors on each axis, they are all 3D sensors, so we've got an X, Y and Z axis. These errors can be linear errors, so you have to multiply a correction, it could be an offset, so you would have to add a correction, and on top of this the entire device could be tilted within your phone. So actually if you look into the data sheets of the accelerometer, they have some tolerance on how much they might already be shifted or rotated within the package, and when soldering it into the phone, I would assume there will be an additional error. I've seen so many different errors on different phones, it's not that easy to simply calibrate that. But let me give you an example of what you can do with it, or just a quick look first, so you see in our app, so this is VFox, I think you've got this, you have an entry acceleration with G, that's the actual raw data from the sensor, or as raw as we get it. If I start it, you see if I shake it, you get some readings there. This process is already great, you can apply it to pendulum and measure the acceleration of the pendulum like this. But something I want to demonstrate is that we can also get the frequencies from this data by doing a Fourier transform and calculating the frequency spectrum of this acceleration data. And to demonstrate this, I brought a little device, an old hard disk drive, it says it's broken, but it's still rotating and that's the important part for us. So if I place my phone on top of it, start the measurement, turn on the hard disk drive, and then you see a peak showing up in the spectrum and it settles at 120 Hz. If you don't believe me, unfortunately we don't have a camera here right now, you can later have a look, it's supposed to run at 7200 rpm, which is 120 Hz. We can even get the time resolution of this. So if I turn it off again, you see how the frequency drops down, and if I turn it on again, there it comes up again. So this is an example of what you can do. It's great for students that can check if the washing machine at home is working properly or they can check other things, but usually I do not like to bring washing machines to talk, so I use the hard disk drive here. One other thing you might have noticed before is that we've actually got acceleration with G and acceleration without G. The second one is actually a sensor that removes Earth's gravity. So if I start the one with G, you will notice that down here on the z-axis, you still have the 9.81 m per square second, which is great because if I rotate the phone, this contribution goes to other axes and we can determine the orientation of the phone, but this is bad actually for deductics because, yeah, actually the phone is resting, it's not moving at all, there's no velocity involved, there is no acceleration, so luckily there's also an acceleration without G, which gives us roughly zero on all axes unless I actually accelerate this thing. The problem with this is this is only a virtual sensor. This is a sensor that's fusing the data from the accelerometer with an additional sensor like the gyroscope so we can actually distinguish between rotating the phone or accelerating it in one or the other direction. Usually you only get acceleration without G if you also have a gyroscope in your phone. I've seen two or three devices that offer you acceleration without G even though they don't have a gyroscope. In this case, don't trust them. This is nearly guessing. Okay, so they probably have only low frequency filter on top of this, so they're averaging out your movement and this doesn't really work for anything. Yeah, so that's the accelerometer. One other thing I want to mention is if you look into the API to access the sensors yourself for some reason, you will notice acceleration without G is usually called linear acceleration in our app since it's made for teaching. With and without G, so if you find accelerometer, that's the one with G and linear acceleration is the one without G if you look at other apps or the API. Okay, next up, I already mentioned this one, is a gyroscope. If you have some physics background, then when you think of a gyroscope, you're thinking of a device that's spinning fast so it has some angular momentum and then usually you want it to be heavy and to have the weight at a large radius so we've got a strong moment of inertia so that you get, when it's spinning fast, a strong angular momentum and due to the conservation of angular momentum, these spinning devices can keep an axis regardless of rotating the frame around it. That's what I was thinking about a gyroscope, what I think is a gyroscope when you just give me the term out of context. Of course, a heavy, huge, fast spinning device is the last thing you want in your phone, so that's not what's meant with a gyroscope when people are talking about gyroscopes in your phone. Instead, there, again, you have a MEMS device. So, again, a microelectromechanical system. You'll notice this looks almost exactly like the accelerometer. If you look for wheel devices, those are actually much more complicated because they need some specific geometry to make sure that they do not act like an accelerometer, but the principle is easy to explain with the same geometry. So, we, again, have this floating metallic part and we've got two contacts. So, again, we've got a part that can wobble in this direction here. But on top of this, we've got a motion that's perpendicular to this. So, this is not depicting the motion of your phone, but this is depicting a vibration that the gyroscope does by itself all the time. So, there are different ways to build them. Some have a rotary motion, some have this linear motion. Also, the way to create this motion makes this device so much more complicated, but, in principle, it's a similar structure which is vibrating forth and back. And now, if you add rotation to it, it's a little bit hard to see as it's rotating. The inner part now suddenly gets deflected. Let's change the frame of reference. So, let's get the camera in sync with this device. What you now see is that the inner part is moving left and right, although the device itself is only moving up and down. And the reason is... I don't want to deduce it entirely here, but most of you probably have heard of it. This is the Coriolis effect. So, yes, in fact, your phone is determining the rotation rate of your phone, not the actual angle, but the rotation rate or angular velocity due to the Coriolis effect, which is just mind-blowing if you do some of the calculations. There are some manufacturers on the Internet that claim that they can detect a movement of the magnitude of a single atom. And I believe them because we use similar structures in solid-state physics, so that's possible. If you want to try it, just turn on the gyroscope on your phone and do a slight rotation like this, which is about the z-axis, perpendicular to the display, you can detect really slow rotations with this and think about the fact that this is done using the Coriolis effect. And, yeah, it's just mind-blowing, I think. So, this sensor is a bit more available. Actually, almost 80% of the phones have them. This has become significantly more since Pokémon Go. The reason is when this game came up, suddenly people noticed that there's a device called the gyroscope, and if it's not present, they did not have this AR mode where you can actually take pictures of the nice cute Pokémon and so on. So, when many people noticed it and the manufacturers decided, let's just do it in the gyroscope as well, because it's not that expensive. In fact, usually it's on the same chip as the accelerometer. Then they're sold as one thing. It's an IMU inertial measurement unit. Not important that term, but it's quite a common thing. And the sensor rates look pretty much the same. You mostly notice a dip in the 100 Hz regime because those are the real cheap phones and then also don't have a gyroscope, but most of the phones achieve higher rates. Again, since we were laughing before, the iPhones also are here again at the 100 Hz. It wouldn't make sense to have the gyroscope fast at this point. Yeah, but that's it about the gyroscope. You've seen it in action in the salad spinner. And that's one of the sensors. You do not really see that often directly, but we're just mostly there to assist other things that you do where you need to get smooth motion, like controlling games, AR, and actually removing the Earth's acceleration from the accelerometer. Next up is a magnetometer, which I think is a more obvious sensor because that's your compass in your device. So when you're doing navigation with GPS in your car, it's a simple thing. GPS gets a position, you get a sequence of position as you're going, and from the sequence of the positions, you get the direction you're moving in your car. Your phone is attached to the dashboard, at least I hope so, so it's pointing in the same direction you're moving. Everything's obvious. But if you're standing on an open space, looking for not-so-trained station or anything, and you're wondering which direction you want to go from a point of view of GPS, it's always the same position. It doesn't get an orientation. You need a compass, which is the magnetometer. How do you get a compass on your phone? A hot-sensor is, in principle, just a conductor with charge carriers, so these are the nice, shiny white boards here, drifting from one side to the other, so it's just an electric current, and if you apply a magnetic field to an electric current or to any electric charge, then there's an effect you might know from school, which is called the Rohrins effect. So there's a charge going one direction, you get a magnetic field perpendicular to this, and then the charge is deflected into a direction perpendicular to the flying direction. And that's the Rohrins effect. The older guys of you will know it from CRTs. If you bring a magnet close to a CRT, the entire image is messed up due to this effect, and that's what we're using in a Hall effect sensor, in a Hall sensor. You've got this electric current, and if you bring a magnetic field close to it, the charge carriers are deflected to one side or the other, and therefore, if you're measuring the voltage perpendicular to the flow of the current, you get an actual voltage that's proportional to the magnetic field. That's the Hall effect. That's how your phone is able to determine the magnetic field. This one is even more common than the gyroscope, simply because it's used for navigation, and so people start to notice if it's not present, and they do not get an orientation in their navigation software. But the actual rate of these sensors is much slower than for the accelerometer. Most of them are running at 100 hertz. This will be important in two more slides. Besides that, there's not that much strange about the availability of this, but it's extremely sensitive because it's supposed to measure Earth's magnetic field. Earth's magnetic field has a strength around 50 microtesla. If you ever carried a magnet with you, did you feel some force from Earth's magnetic field? Of course you didn't. You need to build some compass where the needle is floating or something like this to actually get a rotation. It's a very weak field. That's good news and bad news as well, because on one hand it's very sensitive, and on the other, it's very sensitive, which means it saturates very early. If you want to measure the magnetic field of an actual magnet, don't even try. It will saturate right away, and you do not get anything. To demonstrate how sensitive this actually is, I've brought a flashlight, so a very simple one. I switched to a motor where we've got an SOS signal. It's coming. I point in this direction, and I place it next to the magnetometer in my phone. You see right away, so I'm not sure if you're seeing the light. If I point in this direction, you see the SOS signal popping up in the magnetic field reading, simply because of the current going through the LED. That's what we call an EarthStat field. This is just a typical magnetic field you get from any current flowing. Let's stop it. We've got a nice SOS signal over there. Three short, three long, and three short signals. It's just coming from this simple flashlight. This is also a good indicator on how sensitive this thing is. If you place your phone in a case with some magnetic closing mechanism, compass would not work anymore. If you're not careful when paying your clothes and you place your phone on the big magnet that removes the theft protection from the clothes, something in your phone will get magnetized and will certainly be stronger than Earth's magnetic field, and for the rest of the day, your compass will be pointing in the wrong direction. Luckily, usually the phones are able to notice this and they recalibrate the phone to simply subtract any constant field. That, again, is bad if you want to do absolute measurements because you have not much control over the recalibration mechanism. You can access the raw data value. In VFox, there's a check mark where you can disable the calibration, but then you have to do everything by hand. You will certainly have some background that's annoying. And one other thing, you should also take care and notice where your actual magnetometer is because in most phones it's on the top left corner, top right corner, top center, and this Pixel 3 is a very strange one. It has it on the right-hand side, but it's never at that center, I think, because of all the currents in the phone. I mean, you're charging your battery with three amps. What is this, how much you charged it now? This would yield a stronger field than the flashlight and you would see the magnetometer again. Another example what you can do with this, so it's a little homework for all of you who came by train. Yesterday when I came here on the ICE, I turned on the magnetic spectrum, the same thing as the exploration spectrum we just seen, and when you're doing it on train, you will see a peak at 16.6 ha. It might depend on your actual seat. You might move it around a little bit, but so far I usually always saw this peak. This is the electrification frequency of the German railway, so you can simply check if it's working properly. You should see 16.7 ha. One other thing that some of you might get in your head right now that you could do this with simple electrical outlets, there you get a problem with the rates. That's what I've mentioned, that the rate of the sensor is quite important. I also got something via Twitter yesterday, just as a response to the other one, I thought, well, I was looking for an example like this for this talk, so I just put it in. This is a measurement of an American power outlet, which should run at 60 ha, but this guy is seeing 40 ha, and he was wondering about this. That's what's called anti... Sorry, that's what's called aliasing, so the alias effect. Some of you might know this term from computer games. They usually use it in slightly different contexts. The idea is if you're measuring a frequency that is higher than half of the data acquisition rate of your sensor, so this one is running at 100 ha, like most of the phones do, then half of this frequency is what's called the necros frequency, and you notice that the spectrum goes from zero to this necros frequency. This is simple math, but it's math that's the rules of the Fourier transform, you could say so, and if you try to detect the frequency that's higher than this, so an American power outlet with 60 ha actually the higher frequency is showing up on the other side of this upper limit at 40 ha. Even if you go to higher frequency, it would shift down further and further until reaching zero, and then it would shift up again. So if you're interested in this, check out some articles about aliasing. If you're not that interested in this, just keep in mind if you're measuring frequencies that are higher than half your data acquisition rate, you will not see the correct frequency. Then one of my favorite sensors, the pressure sensor. For this one I need, again, the phone that's not on a wire. Let me, before I show anything, let me demonstrate what it can do, because that's something I find quite surprising. Let's turn on the measurement. By the way, those are wondering how this works. There's a function in VFox that we call remote access. It's basically a web server running in the app, which provides the data. We can simply access the data on the phone to demonstrate or to control the measurement. Now here we see the pressure sensor. Right now this is mostly noise, but what I do now is I hold it up, and if we wait a few seconds, you will see the pressure is actually dropping. It has dropped fine enough. Then I place it on the ground, and the pressure is rising again. Actually, your phone, if it has a pressure sensor, it has a pressure sensor that's sensitive enough, sorry, I turned it off, to measure a change of pressure over a distance like this. Again, when I first tried this, I repeated this test several times before I believed it was just not by accident. How do they do this? You have got another device that actually has a cavity, so below the blueish-grey part there's a cavity in there, by a silicon membrane, which is the blueish part, and if you change the pressure, this simply moves like you would expect from a membrane, just in small. To detect this movement, there's some material on top of this which changes its resistance or resistivity depending on this strain created by morphing this membrane. Unfortunately, this sensor is not that much available, so about a third of the devices that we know of have this sensor. Of course, there's some buys in there from the users that submit data to us. This means that these are usually the more expensive devices. So my rule of thumb is, if it's an iPhone, they usually have the pressure sensor, except for the iPhone SE or some older models. If it's an Android, if you pay it half as much as you pay it for an iPhone, then you have a good chance that you also have the pressure sensor as well. Data rates vary a lot. So the iPhones, like you just saw, are the rate of about 1 Hz. Most Android phones are around 5, 10 or 25 Hz. I've never had a device like this in my hand that does 100 Hz. I don't really believe that this makes sense because I already noticed on my phone that I think it does 25 Hz. Just handling it because of the sealed casing introduces more noise than you can actually use, at least for these small distances that I use it for. But you can do other funny things with this. So this is something I received by Diana Cowan. She might know her as a YouTuber called the Physics Girl. She used the pressure measurement on the flight. It's something you should do anyways because that's the way you can figure out how much air you get to breathe up there. It's much lower than you might expect. So at some point she saw this drop in the pressure and an increase again and she asked her followers, what could this be? I'm not asking the audience right now. I just give you the solution. She was on the laboratory and she flushed the toilet. So when water and air get sucked out, you can actually measure this. And then about a month ago, I found someone else who allowed me to use his measurement. So this guy, Philip Smith, was on an airplane again, and he did not actually go to the laboratory. He stayed in his seat and he just checked when people were flushing the toilet. So as he said, there were turbulence so they couldn't go for a while and then there was a rush for the toilet and he was plotting it. So just for those of you who came here by plane, Justin Hint is a conversation starter next time. When the guy next to you goes to the toilet and he comes back, tell him exactly how often he had to flush the toilet and you will enjoy the rest of the flight. Some other example that we actually use is measuring the movement of an elevator. So this is a lift in Aachen. We have the accelerometer, which measures the acceleration of this thing. We get the total height difference of the elevator from the pressure sensor, a barometer, that's a pressure sensor, and the velocity of the elevator as well from the change in height. So next time you enter an elevator, I want to see you all to take out your phones and measure the distance that the elevator is travelling and the velocity at which it does so. Okay, so these are, in my opinion, most important sensors, some honorable mentions. Almost all phones have a light sensor as well, which controls the display brightness depending on the ambient light. Unfortunately, there is no API on iOS to access this, so if there are apps that seem to access a sensor like this, they usually use the camera instead, which also works quite well, but it's slightly different. It's a difference between luminance and luminance, which I do not want to go into detail here. And on most Android phones, they are badly calibrated or there's so much difference in the quality of these sensors. So you have to check it on your own phone if it's worth anything, but it's a bit difficult. There's a proximity sensor, which is the one that turns off the screen when you hold the phone to your ear, so that's all. Sounds interesting, but unfortunately, it only distinguishes between a near and a far value, which is a difference between five centimeters, so I do not have that much use for it. There's a temperature sensor, maybe. If they are officially there, then they usually come along with a humidity sensor, but there are still sensors in your phone, so you should be a little bit skeptical about this. They're mostly measuring the heat from your battery or from your device. They try to compensate for this, but that's a difficult thing to do, so if you actually need a thermometer, take a thermometer. They're not that expensive. You might see some temperature sensors that are not official, which PFOX can pick up. Those are usually temperature sensors that are part of the pressure sensor to compensate for temperature effects, so they are not even designed I wanted to mention as well where we got the information about the sensors from, so in our app, at the very bottom, there's this entry, Submit to Sensor Database, which tells you to leave the phone resting on the table. It also checks if you're actually doing this. It doesn't let you submit it before. It's happy about the error rate or the standard deviation of the accelerometer. If you submit it, we collect the data on tfox.org-slash-sensor-db, and that's where I got the statistics from so far. If you're interested in what a new phone that you're about to buy can actually do, of course, we don't give you any guarantee, but you can check out all the data about all the phones, at least those that are already in our database. Of course, I'm happy if you contribute statistics about the sensors in your phone as well. I might want to play with this later. And then, finally, last thing to finally conclude is some information on how you can access the sensors. Of course, you can write your own app. I think here are quite a few who can do this. If you can write an app, have a look at the API. They're not too complicated. It's easy to access the sensor data. If you're not interested in designing your own app, but you want to include sensor data in some other projects, there are three ways you can use Pfox for this, which I want to introduce, because that's something that's one of the reasons I wanted to connect here. Don't hesitate. Pfox is free. You can get it for free on Google Play and on the App Store. And when I say it's free, I mean it's really free. So it's open source on the GPR, and you can also get it on afterwards so you're sure the code running on your phone is the code that you see. And we have three versions, how you can, at least categorize it into three, versions, how you can access the sensor data. First thing is you can implement something in Pfox yourself. So we've got this editor, a visual editor for our file format, which allows you to take a sensor, apply some mathematics. So this is just adding stuff, but you can apply a free transform or anything, and then assign it to a graph. Alternatively, and of course a bit more powerful, you can have a look at our XML format, which defines all the experiments. So actually, all experiments you see in Pfox are not hard-coded, but defined in our own file format. You can edit any of them to your needs. And when you're done, you can transfer the data with the QR code. Do not try to scan this QR code just from your QR code app. You have to scan it from within Pfox. And if you do, you'll find a nice little experiment which uses our file format to implement a Turing machine that's counting binary up to 256. So this is the proof that our file format actually is Turing complete, so you can do a lot with it. I'm not suggesting that you're trying to implement DOOM on it or something like this, because you won't be able to. It's not efficient that way. It's not designed to be Turing complete. It just happens to be Turing complete. So if you want to do something more, you can connect to Pfox via network. You've seen one example with the seller spinner when I said there's a web server running on the app. You can use this to access the data directly from your preferred programming language. This is an example where I'm using Python to read out the sensor data and control a synthesizer. So what's running on the web server is basically a REST API. So just visit our website and learn how to do this so you can read out the sensor data of a network and control your project with it. An alternative to this is a new network interface that we have which is more on this XML side or the design of our experiment configurations which is meant to collect data from many users and not live data. So we had this lecture. So this is the new lecture hall, by the way. So we had a lecture where every student got a spring from us and they were supposed to build a spring pendulum and we collected the data from all the students in the lecture hall in real time on the big screen to determine the dependency of the frequency from the mass of the pendulum. Okay? And another example, just a few days ago, during the winter solstice, we asked our international users to point their phone at the sun. So we get an angle for the elevation of the sun and the etymoth from the magnetometer, so the compass. And this way we could trace the path of the sun across the earth from all the users for each black point where the line is a contribution from a user. So, yeah, from this we could, for example, determine the tilt angle of the earth's axis. So just an example of what you can do with this network interface is you're able to set up some server to receive the data. You can use this network interface. We're still working on this network interface, so far it can only do HTTP requests as get or post. But we are also planning on implementing MQTT and other protocols like this. And the third option is Bluetooth connection, which is mostly designed for sensors. So if you want, if you have some Bluetooth or energy sensor that you want to read out, you can use FIFOX. So this is an example of a Texas Instruments Sensor Tech, which is a software which is not designed for FIFOX. But our file format is flexible enough to simply tell FIFOX how to read out the data, and suddenly we've got a sensor that can run independently from the phone. And, of course, you can include your own projects like this. So this is an example from actually my institute because, yeah, originally I'm a solid state physicist, so we're working a lot with Graphene. And this is a demonstrator we created with an ESP32. So this is, yeah, another version of an Arduino or Arduino compatible. And what we're doing here, we're reading out a Graphene Hall Sensor. So similar to the Hall Sensor phone, but based on Graphene. And we can get live measurement in FIFOX with this. And so if you have an Arduino project from which you want to send data that is plotted in FIFOX, you can do it with the Bluetooth or energy interface. But if you have some patience and maybe wait two more months, we are working on an Arduino library to make this simple. So this is the entire code you would need to read out the analog input from Arduino and send it to FIFOX to be plotted. So this is working right now. If you cannot wait, you can check it out on our website. So this is already available, although it's work in progress. The interface will change a bit still. I would prefer if you want to start right now, if you contact me so we can get some feedback and maybe even design the library also to your needs so that we get an idea. So with this, I'm about to finish. So just a short summary of what I'm hoping I can trigger here. So if you were mightly amused, or mightly entertained by this talk, check out our website or check out our YouTube channel or Twitter. We can get some more examples of what we do with the sensors in the phone. If you are a teacher, are teachers here? Quite a few. That's great. And if you want to use this in class or in a lecture, check out our website. We've got a database of experiments that you can do. That's then actually about physics and less about the hardware, where we also demonstrate the experiments and how they work. If you are a teacher that has a specific project in mind, check out our editor to design your own setup with which you can do something specific for a very specific experiment. Then if you are working on an Arduino project and want to plot something, you can visit vfox.org. There you can already access our library, although it's not complete as I said. So maybe wait a little bit or contact me first. If you have a Bluetooth or energy device that you want to use or integrate, you can visit vfox.org. If it's about a device that you did not design yourself, you probably need some background information about Bluetooth or energy. You should know what a GET server is and how characteristics and services, new IDs and all this stuff in Bluetooth or energy works. And it's good to get some documentation or to be good at reverse engineering. But in principle, I haven't seen many devices so far which could not work with vfox easily. Then if you want to read the values for another project via network, visit our website, the wiki on our website, where you can get information about the REST API and our new network interface. And finally, something I would really love if you want to contribute, if you can write some apps. I mean, we can use a lot of things. The iOS app is written in Swift. The Android version is written in Java. Our web server, of course, has web development and HTML in JavaScript. So if you want to contribute there, visit our website at vfox.org. We would love to see some help and development. And with this, I finish my talk and I'm looking forward to any exchange we will have later and any questions. And I'm just thankful that I was allowed to talk here and get so much attention. Thank you. Oh, by the way, since it's up there, one bad news, unfortunately, I can only be here today. So if you want to talk to me, try to catch me today. You can also call me. I actually brought the deck phone, but sorry, only today. Oh my God. So quickly then we have questions now, 15 minutes. Then 15 minutes, I think. And then afterwards you have to find him and catch him. Thank you, Sebastian. Questions? Shoot. There is one. You mentioned the aliasing effect during the... Is it possible to change or modulate the sampling frequency to actually find out... or if frequency is above the sampling frequency? Yeah, that's a good question, not only because of the aliasing effect, but also because in some projects you also want to reduce the sampling frequency. It's a little bit tricky because on both APIs and both iOS and Android, you cannot specify a target frequency. You can only specify a frequency that's specific for a certain use case. So for example, you say I need the accelerometer data, at a rate that's reasonable for UI changes, or at a rate that's reasonable for games, or at a rate that's as fast as possible. So if you do it for UI, you get something like, let's say 2, 3 hertz, something like this, so the user isn't waiting ages before the screen rotates. For games, it's 25, 50 hertz, something like this, so we can control the game. And fastest is the data I've just plotted. And FIFOX always requests the fastest frequency. And FIFOX, you will have a setting where you can limit the frequency. Actually, the frequency is not simply a multiple, no, the other way around. If the frequency given by the device is not multiple of the frequency that you gave, it's not easy to break it down to the target frequency. So you usually see some odd cases where FIFOX starts to group the sensor events along this to get near this frequency. So it might not work that well, and especially if you're looking for the alias effect, this might really mess up the alias effect. So you might need to try a little bit which frequency works good to do this. But, of course, in principle, you can average about multiple values, and this way we simply pick only every nth value, and this way we reduce the frequency. And, yeah, this can be done to our editor or the main screen. There's a plus button with which you can add a simple experiment which already allows you to set the sample frequency. Just keep in mind that you cannot, we always get this target sample frequency. Right. There is another question. No? Yes. Hi. Thanks for the cool task. It's a great app. I love using it in school. Thank you. I was wondering if those cool animations, how the sensor types are working, are available? Sorry. The animations... Yeah, I think... I wonder how to do this best. Before, I was already thinking about sharing the slides. Actually, this talk is just handwritten HTML and JavaScript. It's not easy to control for everyone. That's why I did not simply upload it. I will check later if I can upload the entire talk in some way if it makes sense, either on our website or if I'm not sure if it makes sense to upload it to the system of the conference still after the talk, I will check it. But I'm not... I want to share the slides, but I probably need to add some documentation on how to use them because they're not a PowerPoint PDF or some latest generated PDF. It's handmade. You can always cut them out of the video, get streamed and all that. Right. Not a question. Just a quick question of the axis of the phone. They're like that and that is towards us. Yeah, so for most phones, the X-axis is in reading direction. The Y-axis is upwards along the screen and the Z-axis is perpendicular to the screen. I say in most cases because officially the X-axis, at least I think I've written this in documentation for Android, is along the natural reading direction of the device. So if you've got a huge tablet which you naturally would hold in horizontal alignment, not portrait mode, it might be that the X-axis is the long axis. I have never seen this myself, but I'm a little bit careful to say that all these devices have the same axis, but Z is definitely always perpendicular to the screen and X and Y are then the other ones and they're fixed and usually the short side is the X-axis. Okay, there is one more question there. Please, sir, take the microphone. It's next to you. We got it off the internet so they're not in the video. You mentioned the necessity of the magnetic sensor to determine the current orientation. Can you not use past GPS data and then integrate over the gyroscope data to get the current orientation? Mathematically you're correct. The problem is integrating sensor data is not as simple. I'm often surprised on what some software can actually do. If you do it naively, right now I only have an example in mind for the accelerometer because you could also say you can integrate the accelerometer data to get velocity. You can integrate the velocity to get the displacement of the phone or the location. If you do this, we've got a very simple example in our wiki, but very naive one without any filtering. Then just the noise means that if there's a little error, you sum it up, integration is nothing else, but summing in small steps. You get an offset error in the velocity. If you integrate this again, you get an error in the location which is growing with the square of the time. If you do this for a location and try it out with our naive approach, your phone is supposed to be 100 meters upwards after about 10 seconds. If you do this for the gyroscope, it's a little easier because you only need one integration, but still there will be some drift. I'm not sure about all the techniques the manufacturers employ to filter out any errors. Obviously the gyroscope is self-caliberating, otherwise it would be pointing in a different direction all the time. On some phones, I've seen it jumping when it recalibrates, but if you simply integrate this, you will certainly get drift. There's no way that you can get a fixed position. What I think what they probably do for most cases is they use the gyroscope to immediately direct to rotation and then try to fuse it in some way with the magnetometer information to keep it fixed so that at the end you're not pointing in the wrong direction. But the gyroscope itself, only on its own, is unfortunately only giving you the rotation rate, not the absolute rotation, in contrast to an actual gyroscope, the big one that's rotating, so it's at least not that easy. That's all I can say. Wow, what a bunch of information, Sebastian. I really love the... There is someone else with a question. I really love your replication, actually. We immediately found about it. Go. Thank you so much for a great application and my question is just very short. Can you also integrate external sensors through Wi-Fi or is it only through BLE? No, no. That's what I meant with the network connection. Network usually is Wi-Fi in this case, except when you... I'm not sure if it would work on a conference like this until you're at the cable. So now you can get the data through our REST API. It might not be the fastest thing. Maybe we will add to our network, our new network functionality, something that will keep open apart and push the data in there so far the best thing to go is with our REST API. I was just thinking about the external sensor connection. Sorry, I was thinking in a different direction. Actually, that's a good question. That reminds me that there's something I wanted to add. You can use the REST API in theory to push data in there, but that's only a parameter in the URL, so a simple get push of a single value which doesn't get you far and which is quite inefficient. However, with the new network interface, you can do requests to other devices. So you can do get request and already is able to respond to JSON packet as a response, to interpret the JSON packet as a response. And as we're adding MQTT and stuff like this, this is supposed to go in both directions. But this is really new. So if you've got something specific, try if it works or contact me if it's not working, if you need some help, if you find a bug. But it's supposed to work with our new network stuff that's still in the configuration. So the idea, the workflow of all this connection with specific devices, if you have something set up like this, you create a configuration for PFOX, which then in the end is supplied with a QR code, for example. The user scans the QR code and all the information how to communicate with the device is already supplied. You can also do this for Bluetooth that the device itself provides it to PFOX. But in the end, it's these configurations and with the new network interface, it can also receive data from the network, but so far only via HTTP. Okay, thank you. I have maybe a last question if no one else has. What's the next step? What is your next goal? Because this is a tremendous successful thing and used in educational purposes. So that's fantastic, actually, isn't it? So it's not only on university level that you're using it, but all around in Germany, that's what happens. Not in Germany. It's another thing you could contribute. If you're speaking a language that's not PFOX is translated by volunteers and it's already available, I think, in 12, 13, 14 languages, something around this. So, yeah. But next step, I think we'll be using the camera because that's another sensor, broadly speaking, which we are not using at all, which can do a lot. But we haven't yet started on this, so there's a lot to do in this project. Super, I'm looking forward to see you next year then. Hey. Sebastian Stax. Thank you very much. It's an honor and a pleasure to have you.