 Hello, thanks for the invitation to FOSS Asia. I am happy to speak about the Laid-to-Hear project for self-adjustment of an open-source mobile hearing aid prototype. My name is Peggy Zulob. I am a computer scientist with master of public policy and media artist. We developed Laid-to-Hear books as a prototype for the citizen science project Here How You Like To Hear at Fraunhofer IDMT in Oldenburg, Germany. Having a moderate hearing loss myself, I initiated and lead this project from 2017 to 2020. If you are interested in the story of the project, take a look at the website or the social media channels. Meanwhile, I am not employed at Fraunhofer anymore, so this lecture is on my own personal account. I want to create a follow-up project to move the self-adjustment approach forward because I believe in the future of individualized and self-determined hearing. I also think a hearing support should be accessible to everyone who needs it. To make this possible, open source is the way to go. But let's start from the beginning, namely with the problem we are addressing. Globally, 1.5 billion people live with hearing loss. That's 1 in 5 people who are living with hearing impairment and the consequences. The majority of these people with hearing loss do not have access to intervention to prevent, identify and address the impairment. Hearing loss is often associated with elderly persons, but in fact 50% of young people aged 12 to 35 years are at risk of hearing loss due exposure to unsafe levels of sound in recreational settings like listening to music or going on parties. Even children get already hearing loss and in 60% this is preventable such as they get autism with media and there are many, many other reasons of hearing loss. If you have a look at the absolute numbers of hearing loss in the world, you see that about 215 million persons in the Asian region suffer moderate or higher grades of hearing loss. That's a huge number. An address hearing loss is expensive. It costs 980 billion dollars annually worldwide. The Asian region bears almost half the cost, about 414 billion. Europe too loses 225 billion dollars annually to untreated hearing loss. So from a general look at the lack of hearing loss treatment in the world, coming back to our project, which started in Germany. We have a very good healthcare system. Most people have access to hearing aids. Nevertheless, 75% of hearing loss in Germany is untreated. In Germany, a high-developed country with excellent healthcare where everybody can get a hearing support. Hearing support can be any kind of so-called hearing aids. This means not only medical hearing aids but also intelligent headphones, earbuds and headsets who provide amplifying of sound. Obviously, the existing earables, medical or non-medical, the existing earable market does not meet people's individual needs for attractive hearing support. Why don't people get the hearing loss treated? One reason is a stigma of hearing loss. Hearing loss is associated with being old. Everybody wants to get old but nobody wants to be old. Second point, people think hearing aids are too expensive, although the cost of untreated hearing loss is much higher in the long run. But even when these barriers are taken and people got hearing aids, it doesn't work out. What happens? Sound adjustments are done by professionals in audio laboratories, which cannot fully simulate real-world auditory scenes. This leads to many iterations of the fitting process, which is unsatisfactory for the patients. So, even when people try it, 45% of hearing aids are returned after a first fit. This is also due to the fact that the sound preferences of individual listeners are not well understood by science. The listening intention and therefore the amplification preferences in the same auditory scene may be also different. For example, if you're listening actively to music or you have background music while reading or working at the laptop, you might have preferences of different sort of amplification. For me, being a maker myself and having a hearing aid, the fact that I could not adjust my hearing aid myself motivated me to start the research for hear how you like to hear at Fraunhofer. And it was the first open source project of the Institute, by the way. The main focus of the research was on the self-adaption of sound amplification everyday situations. For this, we went out for 57 sound walks, got about 100 hours adjustment data records and also auditory scene records analysis. We asked for the wants and needs of the hearable of the future and got 550 submissions for our online questionnaire. We organized two hackatons, also the first hackatons of the Institute and worked together with makers, hackers, designers and hearable manufacturers. What do these sound walks look like? As I mentioned, for this field research, we left the audio lab and went to the streets, the park and cafes. People used our like to hear web app on their smartphone to modify the sound on the like to hear box, our prototype. We did not do any hearing test beforehand. People adjusted the sound in the way they preferred. The 2D touch interface was easy to use and intuitive, as the participants said. So let's listen to the experience of a participant with severe hearing loss why he uses the like to hear boxes and also till now in everyday situations. Too bad communication with my partners. My hearing devices, acoustics, gave themselves a lot of effort and showed a lot of patience in the search for a suitable hearing device and in the right settings to understand sound and speech. Unfortunately, it was all without success. We did not find a suitable hearing device with which I had a somewhat good understanding of speech. With the like to hear box, I had very good experiences. The first was that a good understanding of speech was present and especially impressive was the natural speech of the conversation partner. So I have a communication opportunity, as I have not experienced for a long time. Thank you very much to the Frauenhofer Institute and the participating institutions. Thank you Dr. Spiegel. Dr. Spiegel is just using the like to hear box as an early adopter but it's also possible to build the like to hearing aid prototype on your own by yourself. The framework is on top of the Raspberry Pi Implementation developed by Marc René Schädler at Karl von Esjärzky University in Oldenburg. The open source hearing aid algorithms are provided by the OpenMHA, the Open Community Platform for Hearing Aid Algorithm Research developed by HerTech in Oldenburg. This photo shows the hardware required for the prototype. All of this hardware can be ordered online. We have the full list with exact descriptions on GitHub. You need a battery with at least 3A. Then you need a Raspberry Pi with the audio injector sound card which allows sound processing with the delay less than the required 10ms for real time audio processing. Pre-amplifier for the mic with separate power supply. You need two. This hardware is with SMD components and must be soldered by yourself and we are looking for alternatives. Then you need headphones with integrated mix. You find also instructions for the installation of the software on GitHub. We are also planning to launch a forum soon. We have announced also on GitHub. So let's take a look now how the sound self-adjustment works from the user's side. The user connects with any device to the Raspberry Pi which works as a Wi-Fi access point. Once the web app is loaded in a browser, the user can log in and the screen for touch control is loaded. When a user is moving the circle on the screen, new parameter values are passed to the hearing aid algorithm implemented on the Raspberry Pi. The audio input of the auditory scene is processed and then the user gets the processed sound, amplified sound on the headphones. Here is the scheme of select to hear software architecture. Top left you see select to hear framework for user sound control with the web app. There is also the user interface which is quite intuitive and the smart self-fitting approach we realized with that. Bottom left you see the hearing aid prototype providing the hardware setup with hardware calibration and a basic open-image hearing aid algorithm configuration with dynamic sound compression and feedback reduction. Top right you see the open source hearing aid algorithm of Soil Open MHA with TCP interface which makes it easy to get with the web app through it and it's quite easy to configure. Bottom right you see the check audio connection kit for Linux being responsible for the transfer of audio between the applications. But to make a prototype yourself you can also just install the SD card image for Raspberry Pi we provide on GitHub. When you are using the web app interface you modify the sound in the following ways. Moving the circle in horizontal direction you change the gain presets. These presets change the amount of loudness of specific frequency bands using parameter values of a lookup table. Putting the circle on the left side like the dark dot you see in the graphic in the middle you have the same gain for all frequencies then gives the impression of more emphasis to the lower sounds. Putting the circle on the right side like the bright dock then you have linear increase to the higher frequencies. Sounds like more emphasis on the brighter sounds. With the move along the vertical axis you change the gain offsets let's say master volumes across all bands. Soft sounds are more amplified than loud sounds so we have a broken stick non-linearity as you see on the graphic on the right. This is how the prototype has worked so far. It's a great development but not perfectly operational yet. So let's listen to the suggestions of George Kuriki a participant of the research with moderate hearing loss. Hello my name is George Kuriki I came from Brazil and I have a medium hearing loss and also living in Berlin without public health insurance only private ones so I cannot afford a hearing aid kit. I took a part of like-to-hear project testing the prototype for about like 10 days and I just experimented some situations for example giving a class for one person in a closet place pretty quiet it was really really helpful I could understand even like the person was not like the good distance but normally I didn't hear but for example in place for for example like a bar with a lot of people talking a lot of noises and different distance it was quite difficult to understand when the person was talking to me directly and I think a smaller prototype like a smaller version will be really I mean like physically speaking it would be really helpful for me because like to carry like a big box and also like with the wires of the headphones it's not like so practical so that's okay thank you bye Thanks to George too so this feedback from George is actually amazing as a prototype was originally designed for research purposes only we want to take up these ideas we are happy about everybody who wants to support a follow-up project there are many possibilities you can contribute as early adopter with tech affinity or organize a web developer communicator or audio programmer feel free to get in touch with me aim is to turn the prototype into an open source hearable for everyday use which is accessible and affordable for everyone usable, independent of excess of professional hearing care without the need of audio labs with the advantage of individualization of sound by personal control to use it everyday the hearable has to be ease to use and robust so the hardware should be smaller and hansome and the software should keep the intuitive control in their face for everyday use to make this possible Enactus takes action for entrepreneurial follow-up project of like to hear so first milestone is a production of 20 prototypes for demonstration purposes Enactus students are excited about contributing to the 17 goals of sustainable development through innovation and entrepreneurship together we defined a small donation goal of $10,000 for first milestone with that money we want to produce 20 prototypes and use them to show more people and get more collaborators and funding here are some interesting references one of the WHO about the world report of hearing then the other one of the Fraunhofer IDMT of hearing speech and audiology in Altenburg, Germany this is where we developed the like to hear box and this is a sort of benchmarking of hearables the third one is speaking about self-fitting of hearing aids without professional help and the last compares some interfaces for controlling sound if you want to contribute to the project in any way or if you're just interested in knowing more feel free to contact me by email you'll find me also on LinkedIn thanks a lot for listening to this talk and I hope to hear from you bye