 Ia p Dancing in this streamed session. It is from 3 film conservators at Naltonga Sound Envision talking to us about two case studies of their film scanning in conservation work. I would like to welcome Leslie, Richard and Gareth to this stage of the session. Thank you. Kia ora koutou. My name is Leslie Lewis. I am a moving image conservator at Naltonga Sound Envision. Thanks for giving us the opportunity to speak to you here today. I'm here with Richard Faulkner and Gareth Evans, representing the five strong film preservation team at Natanga Sound and Vision, so we're not getting a lot done over there today since we're all in front of you. Natanga is an organisation committed to collecting, preserving and making New Zealand's audio-visual cultural heritage available to both local and worldwide audiences. Our collection has film, video, audio, born digital works, as well as documentation and props. With regards to film, we hold everything from New Zealand's oldest surviving nitrate film, which is 120 years old this month, to digital intermediates that were just produced a few years ago. There are titles that were made by well-known production companies, government, regional organisations, as well as independent filmmakers and artists. We also have a large portion of our film collection is made up of what we call personal records, which are largely amateur-produced titles and home movies documenting the experiences and everyday lives of New Zealanders over the past century. Our team focuses on material that originates on film, specifically its conservation, so caring for the original material in order to safeguard it for as far into the future as possible, so we always have something original to go back to. And preservation, which is creating new copies on stable media to ensure a long-term survival and to create accessibility for as wide an audience as possible. The past couple of decades have been an explosion in the amount of audience members that we can reach and we need better copies that we can give them. And one of the ways that we are now able to do that is through digital preservation. In the past, these preservations were done photochemically, which created new copies on film stock through traditional printing and developing methods. Over the last 15 years or so, the film has seen a dramatic shift towards digital preservation. This involves scanning rolls of film frame by frame at extremely high resolution. For about 10 minutes' worth of content in a film, there are 16,000 frames, which each have to be dealt with individually. We do this in order to create preservation elements that are of comparable quality and detail to that which is achievable using film stock. And that's been something that really just in the past couple of years has become something where we're happy with the result that we're able to get and are much more comfortable calling it preservation. All of the post-production work is conducted digitally and the workflow adopted by Natanga and most other film archives at this point. The end-results and products remain entirely within the digital realm. We have no analogue output, which for anyone who's a traditional conservator is very nerve-racking, but maybe that's not the group here. Given the wide range of materials that we deal with on an everyday basis, accomplishing these goals requires highly specialized knowledge of both the original characteristics of the artifacts as well as the technology used to preserve them. Today, we'll be presenting two case studies that are particularly useful in highlighting different aspects of the digital preservation process. While the original format and content of the films in these two projects are quite different, you'll see how we use the same overall process, whatever the title, adapting it to meet the requirements of the particular source material. First, Richard will introduce you to the works of 1960s amateur filmmaker Hilda Brodie-Smith, and then Gareth is going to talk about the landmark 1980 documentary Bastion Point Day 507. Richard. Thanks Leslie and Kia ora koutou. First up, I'd like to make a very quick disclaimer that we actually don't have time to show the works of Hilda, so anybody who's just come along for the free movie, I'm very sorry but I'm not going to be able to show you very much. So, one difference between Ngataonga Sound & Vision and many other archives in New Zealand and around the world is that we accept deposits of domestic personal records and basically home movies in various formats, particularly small-gauge film collections. Our definition of small-gauge film includes regular 8mm film and suprate and 9.5mm, which is a bit more obscure. It's kind of interesting by virtue of having the perforations in between the frames of the film, which can spell disaster when it goes off the sprockets because they go straight through your picture. So, these films offer a really interesting personal side to our history, often spanning really long periods. In one deposit you can see maybe a baby growing to a toddler and then from there go to school and then even into teenage life and adulthood going off to work and getting married and sometimes even beyond. And they're always shot through the lens, dictated and told through the lens of what the filmmaker deemed worthy of shooting and actually taking the time to edit and sort of also through the lens of the context of their time as well. So that can make for some really interesting material, but of course for every nicely shot, carefully edited film there's a dozen scratchy looking, badly filmed kittens playing in a dark room that's scratchy and impossible to look at. Many dozens, unfortunately. By the 1960s though, many skilled amateur filmmakers were coming together in sort of amateur movie groups and cine clubs and comparing their skills and sharing notes and celebrating each other's works and groups like the Otago Cine Photographic Club. One such avid filmmaker was a Scottish immigrant named Hilda Brodie-Smith, who, with her husband Alan, made several films, many of which have been deposited with Ngataonga, Sound of Vision. This pair went well beyond the realms of scratchy cat films and recording glimpses of family holidays. They wrote, shot and edited fiction and non-fiction films, complete with well-lit interior shots, as you can see here. Complex and camera optical effects and even produced synchronised soundtracks with sound effects, music and voice-overs and even synchronised post sort of voice-overs and yeah, full nine yards, really impressive. Which is very rare for personal records from any period, really. And so with that collection having been last transferred in the 1990s using the technology of that period, we decided that given the interest of the material it was worth completely retransferring the entirety of the 15 home movies that were in the collection. So the first stage of preservation is to acquire the source materials from the vaults and ensure they're appropriately housed in good cans and whatnot, assess their physical condition and prepare them for digitisation. In the case of this collection the film's a regular 8mm colour reversal with mag stripe on the film each. So Adam briefly mentioned mag stripe. In the case of 8mm it's a ferrous oxide type which is about a millimetre, just slightly less than a millimetre wide, which was actually adhered to the finished cut film after all of the editing was done and then you would dub your sound onto that. So that can make it pretty fragile, especially 50 years later. Also retrieved from another vault is the master quarter inch reel-to-reel sound tape from which that mag stripe was dubbed onto. The film is expected by conservators at Ngataonga and who check the method and the integrity of each handmade splice and look for damage that could affect the integrity of the film and the digitisation process. Due to the fragility of the mag stripe on these particular films the normal cleaning processes of particle transfer rolling and cloth cleaning the entire film were deemed inappropriate and so they were spot cleaned using cotton buds and isopropyl alcohol and percolar ethylene when necessary. Even if the motion is in good condition as you can see here the image looks quite good when it's properly illuminated. When you actually look at the base of the same material it's very, very worn because of course where the films were popular with the family they would be yanked out at every holiday and when some poor sucker came over to the house and they'd run it through the projector again which would wear it out. So our conservators would record their findings in the materials metadata records of the database and with that pass the physical items on to us for digitisation. Until recently we'd only been able to digitise small gauge films using ALMOs which are a brand of adapted projector which basically projects the film and records video of the projection using a camera. Hilda's film collection had been transferred using this method recording to DV cam in the 1990s which was an adequate format at that point 15 years ago but with a resolution of 720x576 pixels barely a quarter of today's sort of broadcast standards of 1080x1920 is kind of an entry level for broadcast. So this time around the films were digitised on the MWA flash scan which Adam also showed a brief slide of which is driven by Mac Pro. We purchased that in 2011 and this is able to concurrently record a 24 bit 48 kilohertz wave audio file and create a 10 bit DPX image sequence or a quick time of sort of similar sort of size but still only maxing out at a resolution of 1280x720 pixels. So these modern scanners have a few big advantages over their older counterparts. They no longer use sprockets to advance the film at all instead using a rubber cap stand as you can see with a little arrow there and that just pulls the film along under its own tension so there's no risk or massively reduced risk of it sort of being damaged by sort of any kind of malfunction with the transport or a film splice breaking or something like that. So using a laser shown through the perforations the scanner is able to rack the film correctly and to frame it at transports and the film is illuminated using LEDs through a diffuse sort of light source which doesn't heat up the film really anything like the risk damaging the film through heat well like melting it as you've seen old films melting in front of the lamp and to record the mag stripe the film has run through a very small specialised magnetic reader which digitises the sound to 24 bit 48 kilohertz wave audio. So after loading the film the operator moves to the software interface which where this scanner is calibrated to the film and first the camera is focused on the emulsion and then the frame is cropped to over scan the image a little so here's an example of a frame over scanned we tend to scan just outside the image and a little bit into the perfs so that if the film does still juggle a little bit in the frame then you know that it's not going to juggle out of rack and you're always going to get the entire image later on it's cropped to look something more like that or sort of post-production purposes Once it's cropped and focused each frame is digitised by illuminating the film with a diffuse light source and capturing the resulting image with a 3 CCD sensor and saving it to disk in real time as a DPX image in a designated sequence folder the software is able to automatically detect the film's density and colour and can adjust the illumination of the film in order to correct it on the fly which it can do but it happens over about two or three frames and if you just set it at the start of the film and let it roll through all the shots you do see a little ramping over the first few frames of each shot so in terms of a digital preservation it's pretty much compromised so in order to obtain an image as close as possible to the original each film was scanned over several passes the first pass is done at the film's intended playback speed 16.6 frames per second which was a sort of a standard of small gauge and the entire film is digitised just in one go as well as the magstripe sound this creates a complete and final wave file of the magstripe sound as per the original in one piece and a matching video which we can use as a reference with which to compare subsequent shots for conforming the final DPX from there the films are scanned again shot by shot to reproduce the film's original colour as closely as possible while avoiding the introduced transitions that I described earlier due to die-fade in the emulsion or shifts in exposure each shot has to be checked to ensure as much information as possible is obtained without losing information in the shadows or the highlights one drawback of this particular machine is that it can add digital noise in areas of low light if you push the gamma or the mids to try and bring out detail in the shadows which it's kind of hard to explain but I've zoomed in an image, a frame here and so this is one frame and hopefully we can see it here but if I just jiggle between these two frames it's exactly the same frame but you can see two screenshots of the same frame you can sort of see some noise jumping around and that's caused by a gamma shift the sensor just creates noise in that sort of range when you push it too hard so in order to work around that we try and keep the gamma at a pretty low level and then make those gamma corrections as a post-production correction let's see so when we're satisfied that the colour's as good as possible the scanner is launched and it can scan at basically real time or up to 25 frames a second which is significantly faster than real time for these films the given colour setting might be fine to actually cover the entire film just locked off on the one setting or it might need changes for every single shot in the film so when you do hit a spot where it obviously goes out of range you stop it and then you go back and make a tweak to the colour settings and go back a few frames before and start again and you basically repeat that process until you've got every shot in the film in a separate little DPX sequence so upon completing the digitisation of the raw scan we can format using non-linear editing software but for this purpose we use DaVinci Resolve and first we stick to our reference scan that we got in one pass in the WAV file on a timeline and at that point we can compare the mag stripe sound which we got in that scan to the real to real sound which we've sent out of the building to be digitised in another place because that's not something that we can do at this point and we can compare which one we prefer and unfortunately for me we decided in most cases that we preferred the real to real sound because it's a generation earlier and it's just sort of superior for various reasons and sound quality because we also discovered at that point that mag tape can stretch anybody who's dealt with the real to real audio might have noticed that if you record it twice you'll get two different length recordings so when you're trying to sync that to a DPX sequence which doesn't stretch then you're going to have issues so what we had to do there was we had to lay down the sound next to that mag stripe the digitised mag stripe recording and just carefully adjust the speed of playback to the real version to match the mag stripe which with that sorted we could really focus on conforming the picture so each re-scan shot was placed onto the track above it and moved into sync with the original reference scan and the excess frames were removed and by checking the start and end frames of each shot as it's placed in trim you can confirm each frame is present and the shot is complete and double check the colour before exporting the whole thing is a new DPX sequence to be archived as a digital preservation master the master sequence is cropped and then you've removed that over scan and you can export a broadcast friendly sort of frame rate version say 24 or 25 frames per second with frames repeated to emulate the original playback speed the videos that we export are a ProRes 42HQ and that's for use in editing and broadcast and a highly compressed H.264 for quick reference and internet streaming the final preservation DPX sequence Wave and ProRes files are accessioned into our database and are archived to duplicate LTO tapes and stored in separate locations while the H.264 viewing file is saved to a local server for quick reference by staff and researchers yeah, I don't have time unfortunately to show you an example of your work very fully but I can show you a quick comparison of the old ELMO version compared to the new version so this is coming from a DV cam transfer and you can see the actual lines of the video sensor in there and this is a flash cam version and you can actually see the grain and the emulsion which is a nice thing to see after seeing a sort of blurry soft version with lines you can also see the cropping is much improved with the later version the DV cam ELMO version is really zoomed in quite probably too far this is a really cool little 18 minute film about the development of Porirua in the 1960s which is probably one of the most fulsome records of that development which is really cool so the workflow I just described is specific to the way that the MWA flash cam works and every machine as you probably realise requires a different approach and yields different results Adam was just telling you about the scan station that we've recently obtained and that's a fantastic machine because we've discovered now that we can actually transfer 8mm and superate on that as well as well as 16 and 35mm and that can output up to 5k and down res even with 8mm you can do a 5k scan of 8mm down sample that to 1080 or whatever other resolution is useful to your workflow and it can also export in the one pass you can colour grade it over all of your files exported in one pass which is really useful so yeah at this point I'll hand over to Gareth to talk about Bastion Point cheers Hi my name is Gareth Evans I work with Richard in the scanning department I'm going to talk about Bastion Point day 507 hopefully a few of you guys have seen it it taking place almost exclusively over a single day the film is a documentary about the occupation and the struggle for Māori land rights in 1977 New Zealand directed by Merita Mita and co-directed by Leon Nabi the film is a form of observational cinema with a minimum of narration to convey a compelling story with its use of different cameras and film stocks it presented a particular challenge for digital restoration one that highlights a symbiotic relationship between archival work and film post production like post production we use a workflow which consists of four distinct phases ingest, conform caligrating and mastering at the end of each phase is quality control or QC each and every phase is followed by at least one QC preferably involving two people we sit down and watch a project from start to finish looking for any technical faults or anything that could be improved to maintain the integrity of a project this must occur at the end of each phase before the film moves on to the next ingest is when you take the physical version of the film and create a new digital master or raw scan with bastion we use the original camera negative 16mm AMB roll and scan it on our ARRI scanner the ARRI is the flashgands big brother powerful but ponderous our ARRI scanner works at about 5-3 frames per second or for an average about 2 frames per second compare that to a feature film which plays at 24 frames per second powerful because each frame can be scanned at up to 6k in resolution what makes the ARRI unique as a scanner is that it can expose each frame twice this is to gain as much information from the picture as possible it exposes for the shadow detail and the highlights before combining the two to create a single logarithmic or log dpx file so I'll just play a video this has been exposed for the shadow details and then it will show you what it looks like when it exposes for the highlights and then the final one is the combined image each frame of the film gets one dpx file bastion point is about 37,000 frames long resulting in a folder containing 37,000 dpx files this forms what we refer to as the raw scan the raw scan is one of the most important elements of digital restoration it becomes the new master copy from which all subsequent versions are derived it is incredibly important that the raw scan be as perfect as possible this is why QC is so important the raw scan of bastion point actually failed its first QC the large variety of stocks meant some shots fell outside of what was technically acceptable some had blacks that were crushed others had highlights that were blown out to correct this issue the entire film was scanned again this time we calibrated the ARRI for every single shot all 366 of them wasn't actually extended the ingest time but resulted in a raw scan that passed QC and met our technical standards after the ingest phase we conformed the film Richard talked about this in relation to Hilda Brodie the process remains basically the same for every film you take a raw scan and edit it to match the original film in this case the original film was that top line that large clip everything else underneath that is every single shot with a project like bastion we used an old viewing print as reference we copied this exactly without a print to refer to it's easy to miss a cross dissolve between shots or opticals that might have been added later by using a print as a reference we could guarantee that the new version was close to the original as possible once again the conformed film is QC and then moves on to the next phase colour grading this phase encompasses a large portion of the restoration process it will often include things like stabilization cropping and the removal of dust and dirt it is where the line between archiving and post-production completely blurs as I've said earlier we archived the raw scan as our master we are now moving towards creating a new version of the film one that can be viewed and appreciated by a modern audience to do this we used the same tools available to a modern filmmaker that includes colour grading colour grading is performed by someone who is often referred to as a colourist it is the colourist who goes through the film and gives it its final look as you have seen the ingested log image is often flat and boring it is the colourist who goes through every shot and manipulates it to look aesthetically pleasing or still maintaining a technical standard I'll just give you a quick clip that shows the comparison of the raw to the graded the role grew out of what was previously called colour timing the look of a film was manipulated by dipping it into a developer bath the longer a film was in the bath, the darker the image a short amount of time resulted in a lighter image through this basic form of colour timing a film was manipulated to correct technical problems or through creative decisions from there colour grading evolved into the manipulation of light using filters in the printing process colour graders were able to manipulate RGB and YCM values on a shot by shot basis colour grading is performed using digital tools like DaVinci Resolve we often refer to them as colourists as their role now encompasses far more than just the manipulation of RGB values with Bastion Point the colour grading process was incredibly important we had no reference for how the colour should look the prints had faded and gone magenta the only other digital version had been transferred in the 90s and the colour was washed out to help mitigate this the colour grading process was a new hobby the co-director to attend the grading sessions he was surprised how much we were able to get out of this new raw scan with the notes he provided we were able to complete the film without a colour reference were still remaining true to the original intent during the grading process some cigarette burns were removed it's the round mark on the left these are circular marks that denote the beginning and end of a reel or edit they were often used in news in the case of Bastion Point they occurred at the beginning and the end of a few shots Modern audiences can find these quite distracting using the tools available now it is easy to completely remove them when we did something didn't feel quite right as an archive it is our mission to preserve an artefact exactly how it was intended although we could remove them seamlessly we ended up leaving the cigarette burns exactly how we found them we even now even though this could be distracting to a modern audience once the film was graded and cued we began the final stage of mastering a new DPX sequence was created with the graded footage from this a mezzanine quick time and viewing copy were created all new versions were cued before they were accessioned and became part of the archive we had the honour of presenting the finished film to the family of Merita Mita at the end her daughter stood up and gave us some background on what went into the making of the film all the different stocks and cameras were the result of them big, borrowing and creatively commandeering any gear they could some of the footage was taken from news coverage and that had been gained by very creative means these sections of the film were marked by the use of cigarette burns with this revelation the cigarette burns now take on a whole new meaning one that would have been lost if we had used all the techniques available for post-production the line between archiving and post-production must be, I mean, post-production might be blurring but the aim remains the same it was a poignant reminder that as an archive it is important we find a balance between modern post-production tools and remaining true to the original intent of a film this last clip I have is a comparison of the previous version that was available in our new restored version thank you