 Thank you. Thanks Tim. Hi, I'm Paula. This is where I work at the State Library of New South Wales in Sydney and I run an innovation lab called the DX Lab. This is actually a point cloud version of a work that we've done by Mike Daley who is one of our digital droppings and I'm going to talk a little bit about that programming, grants program and why we do it and then I'm going to pass on to Tom to talk about his project, 80 Hertz. This is kind of cool. This is probably a whole other talk. Yes, so I run the innovation lab and it's very unique for a cultural organisation to be able to do this and we feel very lucky that we have the support and the funding and it's important for the sector. You heard the last speaker talking about taking some risks in the cultural sector and we've been inspired by the British Library and formally the New York Public Library. So we're kind of like a little creative agency within a library and that's very, very unique. This is our website and we really are here to kind of look at new ways of design thinking, experimentation and research with technology and I kind of use that sort of analogy of punk and glam because that was kind of too good to give up but I sort of googled you know how do you describe punk and I'm sure there's many, many ways but there was this bit that stuck and stood out for me which was punk and turn pop music and its attendant culture on its head and I think in a way that's a little bit about what we're trying to do. You still need the structure, you still need that sort of long-standing, formal way of doing things but there is room and Mike Edson touched on yesterday about working outside of our comfort zones and that's not always comfortable but we've been given permission to do that and we're very lucky. We're actually quite a small team, the DX Lab, there's three of us, I'm the DX Lab leader, I have Kah Ho Chung the technical lead at the top and Luke Durnley who's the web developer down the bottom. We're a small team but we're a library-wide team because everything we do we work with different teams across the library. That's Luke finding some amazing code. Okay so okay we focus on user lead design thinking, we bring a new perspective to what libraries are really about and what they can do. We challenge some of the traditional ways of doing things as I mentioned before and expose things that maybe some of the normal digital teams don't have the capacity all the time to do because they've got such a busy job on their hands running everything else. Everyone is welcome at our library but we need to work at bringing in the audiences that we may not be reaching and sometimes this is not through our traditional channels. So this led me to when I was setting up the lab I really wanted to make a difference not just to the State Library of New South Wales but also to the sector and to creative technologists. So it was really important that I felt we should give back to the people who don't have the opportunities. So we developed a DX lab fellowship which is a $30,000 fellowship and that is funded through the library's foundation and we would not be able to do this work if it wasn't through their support. We also have a smaller scale program which is called the digital drop in. This is a $5,000 grant and it's a much shorter kind of quicker sprint and that's what that first point cloud data experience was under a drop in. So we've had quite a number of people come and work with us because we believe that the best sort of innovation comes through collaboration. That's why we do it. We need to provide these opportunities for creative technologists who just don't have some of those funding opportunities. They may not have some of the sort of connections that we have but we have the collections. So this is a really exciting thing for them to be working with. They may not necessarily know what to look for, how to use it and get exposed to things in new ways. So we are funded. We're lucky. We have permission to experiment and that comes from the top of our organization down. It is very much something that we are supported to do and we feel very lucky. So we give that back to other people through these opportunities. It's all about collaboration and using our collections and inspiring people to find and share our collections in new ways. So our first fellowship, sorry, resulted in unstacked. These fellows went on to have sort of commercial arrangements with other libraries in Australia. So it was fantastic for them to start an idea within a supportive environment, within the DX lab and then go on to be commissioned outside of their work with us. So due to the success of that first fellowship, we were lucky to get funding for the second fellowship and that was awarded to Thomas Wing Evans and his project was completely different to the first fellowship. We really wanted to make sure that we have kind of diversity in the projects that we choose. It's a five panel member judging process, two external people and Thomas decided that he would propose to experience our collection through sound, which was something that we'd never done. We'd never seen. We've seen some commercial things around but not cultural heritage. So and that lovely diagram of Thomas there on the right was, you know, the kind of exposing the collection to the city through sound. So one of the really exciting things about this project was that halfway through his $30,000 fellowship, which was to produce something for online, we had this opportunity to think about the physical space and the digital space, which is really an exciting space for us to be working in. We don't see these as binary in our organization, but the intersection of the two is really exciting space to be working in and that's sometimes where the magic can happen. You know, getting the freedom to work outside of the web but using web technologies and combining the two. So I pitched to Tom that we try to get some funding, some additional funding through the council to do a physical sculpture outside of the library that used his research to connect the outside with the inside. So we went through a big gallery's redevelopment program recently and we wanted to make sure that the passerby traffic was also being invited to come into our library. So we chose this space. It's a very, a sort of visual space on Macquarie Street, very underutilized, beautiful space next to Matthew Flinders statue, hardly ever used. This was Thomas' original concept. We did pitch it to the council for some funding, but we failed due to a timeout on a server in the application process. Anyway, that's a long story. But we didn't give up. We thought, okay, we've done all this work. Let's keep going. So we were lucky that our foundation took up this idea based on this drawing and some initial conversations. They were willing to take the risk with us. We had a very short turnaround time. We had a lot of challenges. Never assume the weather when you're making an electronic artwork that is slightly porous. We had very heavy rain. The budget was set. We couldn't go outside of the budget, but we were making something that we really had no idea we'd never made before. We risked not getting permission from the council in time for our launch on the 4th of October, but we proceeded. We just really wanted to make this happen. We got permission. So tight turnaround time. We had risks on multiple kind of collaborators while Tom was doing his research and design, but we took them and the project has been extremely successful for the library. Now I'm actually going to hand you over to Tom, the creator, and he's going to go through the making of this with you. So yeah, I'm Thomas. I'm obviously the most recent Dx Club fellow. Paul calls me a creative technologist, which I still don't fully understand. I guess I'm just more of an enthusiast. So I'm just going to show you a snippet of video which summarizes the project, and then I'm going to unpack how we did it in three parts. So number one, extraction. So pulling the data out of the paintings. Number two, translation, turning that data into sound, and then fabrication. So how we built this physical experience around it. 80 Hertz is the new installation by British designer and architect Thomas Wing Evans. Made in collaboration with the Dx Club, the project takes paintings from the collection of the State Library of New South Wales and translates them into music. Contrasting with the heritage sandstone of the library, the experience is housed in an acoustically resonant black timber and anodized aluminium pavilion. Inside visitors can select a painting from a reel of images that emerges from the floor. Bringing an image to the top generates a soundscape which vibrates through the floor and from speakers mounted in all corners. Okay, so for those of you who are unaware, the State Library has just opened a brand new gallery, actually a couple of galleries, you should definitely go and check them out. There's 300 paintings in one of the opening exhibitions which detail all facets of Australian life and history, and of those 300 paintings I chose around 40 to use as a basis for this image processing project based on a number of attributes. So things like historic moments portrayed in the in the paintings, well-known subjects, familiar landmarks, or simply images that are just visually striking, perhaps they're just monochrome or really really complex and colourful. So what data can you actually pull out of a painting? Well there are loosely two types of data. There's the metadata and there's the visual data. So when an image is digitised, as I'm sure you'll know, a host of information is attached to it which enables it to be archived and accessed much more easily. Things like the library category, the date created, method of creation and the painting dimensions. And then the visual data that we usually detect in person with our eyes can now also be extracted using image processing techniques. So things like the image time, the face detection, image contrast, accent and dominant colours, and then the overall complexity of the image. So these are the two parameters that I worked with. And what you end up with is a list of values for each image. Each value will often fall into one of a range of groups. So for example this painting is a size medium based on its dimensions relative to the other paintings. The image is simple in complexity as the main colour palette is only made up of a handful of colours. So it's given a value of one. And things like the visual data, the way I actually extracted this was much more complex in the sense that the metadata you kind of can just collate into spreadsheets and it's reasonably easy to kind of define rules for that. But the visual data was extracted using image processing, using APIs from big bag corporations like Google and Microsoft. And then a few existing online experiments that I basically built upon. So the next big challenge was translating this data set into sound values. As Paula said, it's a small team. There's only three people and me. And I think Luke was probably the most enthusiastic in helping, so really two man team. Meanwhile, the project was increasing vastly in scope and ambition. So the project was meant to be just a web page as we said and it's really snowballed from there. So we collaborated with Sonar Sound to develop the audio generation framework. We took the complete data set and developed a set of rules together. And then Sonar Sound worked in MaxMSP to basically build a custom patch, a miniature program that can actually pass this information that I've given them using the new rule set that we've created. So we'll just bring back the original values. This is how they translate as a compositional rule framework. So for example, the method of creation dictates a type of base tone, so whether it's painted oil on canvas or oil on board or watercolor, it's a completely different base tone. The accent and dominant colors determine specific chords in the scale. And the date of the painting affects how worn out and distorted the instrument sound. So the older paintings almost feels like you can hear the pianist moving around in his chair. If a face is detected, intelligent harmony is activated, which generates chord progressions based on Western principles of composition. So they sound more familiar. They sound more human. And if there is a face detected, the chord progressions are much more randomized and feel more jarring. So we had in our minds that the audio would be recorded fresh from live instruments, which was probably really naive, given our fairly conservative budget. So the initial cost with Sonar for doing this original composition was just way too high for us. So we basically went back to them and we decided together that, well, they've got this huge existing library of sound files, they've used another project, things they've already recorded. So basically we sampled from all these different sources that fitted the concept of each value, kind of like a weird form of DJing. The advantages were actually huge. So it gave the computer much more agency over how it composed the audio and gave us less opportunity to impart our own biases and tastes about what the music should sound like. And it highlighted the glitchiness of the frame that we created. So it celebrated the limitations of the rules and the software. And rather than kind of getting stuck in this uncanny valley of trying to sound too human, it just kind of sounds like a weird hybrid of us trying to talk to a computer. So here's a sample of some of the soundscape. So I'm just going to play through all of them just to give you an idea of the difference. The final and largest challenge was how to wrap everything up in a physical form. We would just create a digital experience without screens and remove the digital and make it feel like magic. So while researching the library archives, I noticed the artifact stored in these huge moving bookshelves that I'm sure some of you are familiar with. You can crank along with huge handles. And a similar method was used for scrubbing through old video reels. So this worked as a really simple way to choose images and why not use it. It was a nice reference to the archives. And so the images will be located on a central reel that could be wound by visitors to choose the painting they want. The next challenge was actually how to get something mechanical to work as a digital input, which I'd advise you never try and do. I came up with the idea to use RFID stickers mounted along the reel next to the images, which you see in that right clip there. And it sort of works like a linear GPS. Each sticker has a unique number. There's a reader in the base of the structure and the program knows exactly where you are on an image, what image you're on. So it's kind of super accurate in that way. So in the physical space, it was driven by some really key design factors apart from the kind of more conceptual ideas. So acoustics, we wanted the sound inside to be unique. We clad the structure, the pavilion in metal so it became this kind of reverberation chamber. We wanted it to be a bit of a beacon. It ran from 8 a.m. to 8.39 p.m. at night. So it works in daytime and nighttime. So the structure is open, allows light to come in during the day, and it becomes this beacon at night. And again, feeling open adds to the inviting and accessible nature of it. People can kind of peek in, decide if they want to come in and explore. Had to provide shelter. So it had to shed water, at least the majority of water, and protect the delicate electronics and mechanics. And then the final two had to be modular. It was assembled in an archive room in the sub basement of the library. Had to be quick to build on site, and it had to be portable because there was a legacy plan for it either to tour or to be sold or donated. So those last few points were a real challenge, making it modular and portable. So I had to develop a visual program in our algorithm that defines the entire design for the 80 hertz structure. On the left you have the input, and on the right is the output. So the script itself is basically the size of the finished building. And this is actually only one half, so you times that script by two, and you've got the main large structure. So just run you really quickly through a couple of the parts of the script. So this one pre-defines puzzle joints and bolt holes, so you can fix these panels together before getting to site. There are complex pre-cut joints cut by a CNC machine. So again, it requires just a little bit of intuition to understand how it comes together. And then the shingles that cover the entire structure with geometry was decided by the script, and the screw holes were pre-cut. So you end up with this flat-pack kit of parts, something that is intuitive to assemble and quick to construct. Basically like Ikea does cultural heritage. And then you literally just plonk it outside and hope for the best. Well, not really, there's obviously a lot of stuff that goes into that. So press. This is an example of some of the exposure that the project has received. It's worth noting that a lot of these websites and magazines are actually design, architecture, and innovation focused. They aren't cultural heritage or glam focused. So these are audiences the library doesn't ordinarily reach. And it's only by stepping out of the kind of collective comfort zone that we're able to achieve that. So just taking a handful of these, Design Boom, which is a design and innovation magazine, Arc Daily, Architecture, and then the APC Australia, each have a monthly readership of, you know, in the millions. And combining just those three is 27 million. The project seems to have reached the far corners of the earth through social media. In the central image, you can see it's been translated into Farsi. I can't actually read Farsi, but I know that the proper nouns are actually still in English. So I know it's a project that I did in New South Wales for the State Library, and I'm assuming everything else is compliments. So this clip was taken with a GoPro over five minutes in the structure. It's fascinating to see how people use the installation. Some of them don't actually even use it. They just hang out in there like it's McDonald's or an Apple store or something. But like Adam Moriarty said in his brilliant talk yesterday, it's not all about the numbers and the stats. Engagement with the public was the most satisfying part of this project, hearing what they had to say about it, especially if you didn't tell them, you designed it, and you just stood in there and listened. But ultimately, it's obviously because it's an experience designed for everyone to enjoy. This is the slide that Paul hasn't actually seen. So first and foremost, this project is a celebration of the physical artifacts in the Library's collection. The working process, the analysis, the sound generation are all digital. And then the finished result is a hybrid of the two, so it's digital and physical. So I thought it was only fitting to take it one step further and appeal to the types, the hypsotypes and the pretentious millennials like myself and turn the physical paintings that became digital soundscapes that became a hybrid experience back into pretentious vinyl. So yeah, we've got a few of those. Thank you very much. How does it handle the heat? How does it handle heat? It's actually fine in there. It's, I think, well it's under these two huge amazing fig trees that kind of create this like natural arch over it. So it's not so much the heat we have to worry about, it's the bat and bird guano. So it's pretty good at keeping that off. The only problem we have with heat was actually the some of the LED lights inside the centropodium had been on literally. It was originally run, the run was for three weeks and then it got extended another month, so it was basically two months of this light being continuously on. It never got switched off, so that suffered a bit when we had to replace that. But other than that, it's quite nice, light and airy. There's a possibility that it could be housed in another cultural heritage organisation. We would rather sort of see, from a sort of sustainability point of view, this structure be kind of enjoyed more in other places because we actually don't have enough storage space to keep it. It is transportable. It is actually still up, but it's coming down this end of this week. Yeah. Yeah, two days. What's that, sorry? There's been many talks about it. We're thinking about, yeah, different options. It depends on who comes in. If they're under like three foot tall, they love just what, winding around. But if they're over four foot tall, you get a range of people. You get some people who come in, they just want to experience the soundscape, feel the vibration in the floor and see some pictures and that's it. Then you get some people who are honestly in there for like 30 minutes and they close the door and they're not letting anyone sit in. They're just sitting there trying to play it like an instrument. I think you can kind of get a bit more of a nuance understanding of it that way. One thing we did notice, which actually partly goes back to what I was saying about letting the computer do the work of the composition for us, is that people want to find patterns in the images and they want to think that the image that's of someone who's looking sad makes sense because it's sad music and if it's not, they're like, well, why not? It's because it's just raw data. It's not trying to say that this is a sad image and sometimes even two images that are next to each other look very similar can sound very different because one is painted with a different medium, it's from a different era, or it's from a completely different location. So it's a theory, but it's also if you sat there long enough, I think you could probably work it out. We also had a lot of people who'd been into our galleries to see the actual paintings and then as they were walking out would come and then play and then they'd tweet that tweet about that. So there was a connection and we tried to put that on the label outside, the kind of sign. Hi there. Yeah, David Reeves and I had a good visit there. That's interesting. Another question? Yeah, it was really beautiful. Earlier you said that you were standing in there listening to the visitors comments and things like that and I just wondered what were the highlights and possibly lowlights of the things that you've heard people say? One of my favorite things was when lots of little people about three or four would come in and they'd be immersed in this sound, but they would see this kind of glowing pod in the middle and they'd run up and they'd put their ear on top of the structure because they thought the sound was emanating from the middle. So that was one of my highlights. I love that. There's also an elderly lady doing tai chi there in the mornings to the music, which is quite nice. I can't think of specific comments. It'll come to me and I'll tell you later on. People really enjoyed the physicality of the music as well, like the subs coming from underneath the floor. So it wasn't just hearing, but it was also kind of feeling the sounds. Yeah, that was nice. There's also multi-channel audio, so depending on where you stood, there was one, two, three, four speakers and then two subs, so you could actually move around and feel the vibration or walk to the other side and you'd hear a completely different sound. So it became quite a spatial experience as well. One quick question. Did you, were you ever tempted to add an environmental factor into the max MSP to add another level of composition? Yeah, there was a lot of things that we would like to have done. We were very conscious of feature creep and we were kind of running out of time. Yeah, I mean we talked about potentially having the music actually changing throughout the day, so depending on the time of day, depending on the weather, the audio would change, but maybe that's some ideas for another project later on down the line. And also the blending of the paintings too. We're going to do a little bit more in that space as well, but yeah, we didn't. Yeah, so the RFIDs that line next to the images, there's eight per image, but Luke Dernley from the DX lab did this amazing thing where he averaged between them, so basically doubled the amount of data points he had, so it was really accurate, which meant that you could play, get to the center of an image and then play just off of it and it would warp and sound different and then if you went completely off of it, it started melding between the two images. You ended up with this weird hybrid of the two images, soundscapes, which is quite cool. Didn't mention the other little structure that was next to it. There's another structure as well, but I felt like we weren't going to have enough time to talk about both of them, but it's a smaller one that's all black and you're sitting it, and then it projects on the floor in front of you, and that one is much more passive, you're basically sitting at the images scroll past and the soundscape changes, there's two embedded speakers above you and you're sitting there with friends and enjoying it that way. That's where the lady was doing her tai chi in front of that little structure, which is also available to tour. It might be auctioned and it's available for sale, so. Thanks very much.