 Good afternoon, everyone, and welcome to our bridge meeting for October 2023. My name is Ariane Robin Mach, and I will be serving as the moderator for today's meeting. As a reminder, the Office of Agency Services at the National Archives and Records Administration, host these bimonthly records and information. Or bridge meetings. Bridge is co-produced by the Office of the Chief Records Officer for the United States and the Federal Records Center program and is live streamed to the audience over our YouTube channel. Generally, bridge meetings consist of a scheduled program of presentations with an open forum at the end of each meeting to ask questions of the presenters or of any related federal records management topic of interest. Viewers are encouraged to ask questions by sending an email to rm.communications at narra.gov. Our staff will be monitoring this email box during the meeting. You are also welcome to ask questions and make comments during this meeting in the YouTube chat. However, please keep in mind that all chat messages are subject to moderation, so we ask that you keep them relevant to the topics being discussed. Copies of the presentation slides used today are posted on the bridge page of the archives website. That web page is also where you'll find the links and information about previous and upcoming bridge meetings. If you have general comments about bridge or suggestions for future topics, you can use that same email address rm.communications at narra.gov to pass these along to us. We welcome your feedback. With that, I would like to start by introducing Lawrence Brewer, the Chief Records Officer for the U.S. government. Good afternoon, Lawrence. Hi, Aran. Good afternoon to you and good afternoon to everybody here. Happy New Year. Welcome to FY 2024. October's always an exciting meeting for us. We get to talk about the things that we finished up at the end of the previous fiscal year and we can really look ahead to what's coming. So, you know, congratulations to all of us. We survived the shutdown and we know we're good until November 17th, so we have a lot of work we want to fit in. Plans that we have for the year and I'm sure you do as well and looking forward to working with all of you on the records management projects that you have planned for this year. So, before I get into the agenda, which is on the next slide, you've got a couple of updates, just some reminders, things I wanted to highlight before we get into the program. The first is related to an AC memo that we sent out earlier this morning announcing the annual reporting dates for calendar year 2023. So, very, very similar to what we've done in previous years. The reporting window will open on January 8th and it will close on March 8th. So, at this point, we would like you all to evaluate where you are in terms of your contacts. We want to make sure when we get ready to send out the templates for reporting later this calendar year that we have accurate information from all of you. So, we would appreciate if you would let us know if any of your contact information has changed within your agency, perhaps you have new records officers, new liaisons, new people who are helping you with your annual reporting, please let us know by email to rmselfassessmentatnara.gov. And we will really appreciate it. And of course, when we get closer at the next bridge meeting in December, we'll have much more to share in terms of what to expect for annual reporting topics we're emphasizing for calendar year 2023. And any procedural or logistic information you need as we get closer to that January 8th date for the opening of the window. Second thing I wanted to highlight is we had a blog post go up last month on Records Express. And if you're not following Records Express, please take a look, subscribe, follow it, you'll find the latest information coming out on our Records Express blog. But I wanted to highlight a big milestone that we reached last month, where we publish the service measures as part of our Federal Electronic Records Modernization Initiative, or otherwise known as Fermi. So this is the final piece of what we call the Federal Electronic Records Management, Federal Integrated Business Framework, and the service measures reflect the performance measures that define how government measures, successful management of agency records based on timeliness, efficiency, and accuracy. So it, it was the final piece of the puzzle that we were able to include along with the requirements, the business cases, the scenarios that are already there and available to you on the Fermi website. So I encourage you to take a look to see what we've come up with in terms of the service measures and the other resources that are available on our Fermi page on our website. So on to the agenda. Next slide please. So we do have a lot to cover today. We're going to first hear from our colleagues in the Federal Records Centers program. Then we have a number of pieces related to recently issued guidance. And we're also going to talk about a records management self, a records management assessment of agencies managing social media records and we felt like it was a good topic to share. Social media and management of social media records is a hot topic and it's something that we're all interested in. And this was our attempt to really work with some agencies, gather some information and see where we should go next in terms of guidance related to social media. And then finally we will close the program with an update on where we are with the annual booth. So with that let's get to it and I'll introduce our speakers from the FRCP. We have Jeff Lunsford who is the chief financial analyst in the FRCP, along with Andrew Shearer, who is an account manager in the customer relationship management branch for the FRCP. So I will stop and turn it over to Jeff and Andrea for their presentation. Over to you. Thanks Lawrence. Good afternoon and good morning depending on where you're dialed in from. My name is Jeff Lunsford. I'm the chief financial analyst for the Federal Records Centers program. I just have a few quick updates on the FY 2024 rate schedule. Then I'll turn this section over to my colleague Andrea Shearer, who will go into a few more detailed updates with some wider ranging subjects. And then we'll do a quick Q&A on the FRCP section of this bridge meeting. So, exciting news of the FY 2024 rate schedule has been approved by the archivist. If you have not already received it, you will very likely be receiving it in a few days along with the interagency agreement for FY24. Andrea will have a better sense of the timing of all that. And the rate update was a collaborative project between myself, my colleagues in the Federal Records Centers program, and the NARA CFO. We worked on some changes over the over the summer and finally proposed a final rate package to the archivist. So there were three changes that we've made to the rates compared to FY23. One most exciting one is that we have changed the storage rate, the sort of S1, if you're familiar with the service codes, the cost for storing a box of temporary records with the record sharing program. And it's a rare occasion and I take great pride and joy in saying that we were able to reduce the price of storage by one penny per box per month. Hopefully that's good news. I'm sure there's some like virtual cheering all across the country. So that affects nearly every customer. It's a bread and butter rate for us. And the cost will go down again by a penny per box per month. We're pleased to be able to forward that cost savings on to you as we strive to maintain balance in our fund. The second, what sort of less exciting rate change is that should you need to purchase an FRC carton or an FRC box from us due to the increase in supply costs, we've had to change the rate from $3 per box to $3.50 per box. I believe those boxes are also available through GSA Advantage and other other sources. But should you need to purchase a box from us, again, that rate went up by 50 cents. The third change you'll see in the rate schedule is a new service that we are in the process of rolling out this year. So it's not fully available today on day one. But throughout the course of the fiscal year, we will be rolling out an additional service that will focus on digital delivery of your standard reference requests. So if you request a file, potentially several folders, we will be rolling out a service where we will be able to scan that and send that to you electronically. This is still very much under development, but I believe it's an exciting change, not just for us as we rethink the way that we interact with you, but also hopefully drive some efficiencies on your end too so that you can deal with electronic records instead of paper records. There's limits and procedures and definitions and all sorts of things that are still being developed and coming your way. But we do look forward to interacting you electronically. So those are the three changes that we'll see in the unit rates and that's the entirety of my update for you. I will turn it now over to Andrea and she'll be able to educate us more. Thank you. Thanks, Jeff. So hi, my name is Andrea Shearer. I'm an account manager with the record center program. And I'm going to continue with some of the things we were just talking about I'm actually going to skip over the 24 agreement agreements and go right to the digital imaging since we were just talking about that. So yes, new in 20 fiscal year 24 we have the record center program will no longer engage in any new imaging projects will honor any existing commitments, but no new imaging digital imaging projects are going to be started from 24 forward. But with that, we have and will be introducing in this fiscal year, rolling out incrementally by record center, the digital delivery of reference material which Jeff just talked about. The different maybe wondering what's the difference between this service and smart scan. Well, for the digital delivery service there will be no additional cost to customers. Also, there will be an upper limit of 500 pages, and it is possible that the for each request of the service that the images might not be returned as quickly as if it were a smart scan smart scan is often done for low volume images, where we have this data that 500 it, depending on the, the request, the, the records may require some document prep before imaging. So, that's the distinction between smart scan and digital delivery. And that smart scan is still available at all the records centers, and we're still fleshing out the details. Just if you can keep your ear to the ground, the account managers, or the acting director of the federal record center program someone was going to be once we have rolled out the, this service at the first record center. So now I'll talk, come back and talk about the interagency agreements with that service changes Jeff talked about, we had some, we had some work that needed to be done on the back and to make notifications to staff as well as updating the narrow terms and conditions. I'm happy to report that late yesterday afternoon, both of those remaining tasks, tasks have been completed. The account managers now have, as of this morning, the, the updated and approved narrow terms conditions so in the next day or so if you will be receiving the interagency agreements. The last thing I want to talk to you about is the continuing resolution. So, prior to the new fiscal year account managers reached out to each of their customers to confirm the level of service that we should provide in the event of a funding lapse. Fortunately, we were able to avert that until November 17. So, if you have responded to the account manager with the information. Thank you. If you haven't, the account manager will be reaching out to you again to make sure we know what to what level of service we can provide to your agency, should there be a funding lapse. And that pretty much covers all that I had for the FRCP updates. Are there any questions? Thank you both for that. Those updates. We do have a few questions that have trickled in. First, I guess it's for Jeff. What is the cost of the digital delivery of reference requests? That's a great question. We ran the numbers and we did our very best and we are including digital delivery as a baseline service. That means that the cost of digital delivery is included in the storage of your records. And there is no additional fee that you would have to pay to receive your records electronically versus receiving them. The standard, you know, analog way through the mail. Again, subject to all the caveats that we are rolling out the service. It is not available widely yet, but we do hope to incorporate it as sort of a quote free baseline service as soon as we possibly can. Thank you. Will the digital, I guess this is a follow-up, a natural follow-up. Will the digital delivery service adhere to the technical specifications for digitizing permanent federal records under subpart E of 36 CFR part 12 36? No. No. It will not. This is just going to be, it's just like a smart scan, 300 DPI and, but it will not. I'll throw this one out there too. Can you address whether or not there is permanent space at NARA FRC specifically in the NCR for permanent TSSCI records? We are getting conflicting information from various levels of NARA as well as from DHS headquarters records officers. I'm going to say I can't address that. I'm afraid I can't. I don't know. If you're getting conflicting information, I'll apologize for that. But if we can follow up on that and I don't know if we can somehow return that information. If you're willing to share who that asset question will be happy to follow up. We can do that. We can do that offline. Thank you. And here's one that's come in is Andrea saying smart scan will continue until the digitization service is operational. That is what I'm saying. Yes. So yes, smart scan stays online until such time as a record center has the digital delivery available. And then again, I'm short on answers because we haven't fleshed it all out, but it's my understanding that we would transition that record center to digital delivery. Are you planning to offer core or coordinate services? For example, remediation freezing for overseas or foreign locations. We've had hurricane typhoon impacts that affect our ability to ship damaged records stateside. Um, I'm going to have to table that one as well. Not really. We've had a lot of preservation people on the line. We have done that in the past for sort of domestic natural disasters and hurricanes. So I'm certain there's a process, but that's well outside of my wheelhouse and I don't want to make any promises or declare the statements regarding that. So all of these questions are being emailed in so we will definitely we can definitely have a paper trail so you guys can reach back out if need be. Jefferson or Andrea regarding the digital delivery of reference requests. Once this is fully deployed, you anticipate a similar turnaround time as we currently experience with traditional physical shipping. For example, I usually receive a requested file within a few business days. Can we expect a similar timeframe for digital delivery? That is, that is absolutely our goal. When we envision this in the in the early days, we acknowledge that we do have a 24 hour turnaround from the time that we receive the request to the time that we ship a record. And then that that shipping may take two or three days depending on, you know, the carrier and the distance and that sort of thing. Our goal is to do a two to three day call it three day turnaround on the scanning process and then either like an FTP site or secure encrypted email or that's something like that. The goal is to absolutely maintain the same response rate. It's just that there's different to sort of non comparable different processes that are involved in getting that that record to you in three days. And we really need to nail down that that business process in a timeline so that we can make a firm commitment. Okay, thank you. Thank you both. Please stick around those are the questions we have already in the queue, and any that come in will address at the end of the meeting so thank you both. We'll move on to the next presentation, which is on our bulletin 2024 04 managing records on collaboration platforms, and I'll turn bridge over to my colleague Marcus most Marcus. Thanks, Arion. Hello bridge. Next slide please. Wanted to take a moment to give you an update on a bulletin that we were able to publish right at the end of the fiscal year. In case you missed it amongst all of the other activity that was happening at your agency. So here's the bulletin about collaboration platforms and also the link. Next slide please. So quick snapshot of how we got to the bulletin. A lot of a lot of our bulletin start off with an assessment. And so that's what happened here. Nara and GSA got together and did a interagency collaboration application assessment back in f by 22. We also have been working with GSA hosted group called the interagency collaboration community of practice, which is a federal group of experts focused on the technical and policy mechanisms for agencies to work together through tools like collaboration platforms. We've briefed them a few times about the bulletin and the community of experts did provide some feedback on the bulletin. If you're interested, the same group is coming to this community of practice group is coming to brief our Microsoft 365 user group meeting in November. So we're excited to be working with them because they're helping us build better records management understanding in their community. And then we put out the bulletin for general agency comment back in July. So here over 100 comments from 20 agencies. And so that's really what I'm going to be talking about in the few remaining slides that I have. Next slide please. Perhaps the least surprising part of this bulletin is that, yes, agencies are creating federal records when they use collaboration platforms for their business. Perhaps simple, but a declarative statement is sometimes important. It's important to bring clarity to to the work that we're doing. Next slide please. So as part of the feedback that we received, we received four specific questions. And so we wanted to make sure that we answered them in the bulletin. So the first of these was do we need to record all of our video conferences, and the answer is no. If you do, then you likely have a record. It may not be permanent, but you do have a record that you need to manage. Kind of questions two and three here go together because they're kind of two sides of the same coin. Responsibility is if I provide a collaboration platform and what if I use one. Ideally, the agency providing the collaboration platform would be responsible for managing records for all the participants on the platform. If that's not possible, then at least allow other agencies to copy their records for their proper retention. And oftentimes this kind of agreement is handled through an MOU between the collaborating agencies and that can be helpful to clarify roles and responsibilities. Perhaps more importantly, bring your records officer into the conversation because there are implications here that need to be thought about. Next slide please. And perhaps the question or the question we did get back most in the in the comments is this what is the talk a little bit about this distinction between chats and MS team channels. So what we did for the first time in this bulletin is kind of pull together the three pieces that we thought were most relevant for agencies to understand. The first of these is the bulletin from earlier this year in which we expanded the use of capstone or role based records management to electronic record to electronic records. Second, there was an expansion of GRS 6.1 to include other types of electronic messages like chats. Third, there are also provided additional clarification in the GRS 6.1 FAQ. And specifically I wanted to point out question 11 in which we've got a pretty clear table that contains a non exhaustive list of examples of things that could be included like chats, but also an additional portion which would include channels. And just to drill down on that for just a minute. We've brought chats in because like messages or like email, they're filled or they're often affiliated with an email system, and they don't lose context when they're separated from other records. We decided to exclude because they're associated with projects generally and GRS 6.1 excludes any messages that would lose context when separated from other records within a larger collaboration platform or environment. So that's, let's go to the next slide. So that kind of gives you a flavor for what's in the bulletin. Obviously I would encourage you to take a look at it and email us if you have any questions. And I guess I'm ready to take questions now. Thank you Marcus for that update. We do have a couple questions. And first, could you please discuss chat a little bit more. Yeah, so we had to take a deep dive into teams to really understand how agencies had implemented and are using teams. That involved us talking with experts and agencies and with Microsoft to tease out the distinction in the chats function where chats are much more like email, which is managed at the user level, whereas channels are much more closely tied to projects, and are managed at the project level, rather than at the individual level. So that that was kind of the technical level that we ended up having you get to fully tease out the differences. What legal and or procedural guidance would you give to federal agencies with areas or offices collaborating with other federal agencies without involving records officers. Yeah, I mean, I feel like if the federal agencies are using collaboration platforms to create records then it would make sense that they need to follow the, you know, the narrow as guidance on managing those records. And I would just say from a, putting on my other hat as supervisor of the policy team, what I would add to that answer is just so we would want agencies, the records folks to coordinate with all the appropriate stakeholders in their agencies when rolling out these kind of platforms are using any or engaging in these spaces and that includes their general counsel, their IT staff, those kind of the traditional folks that we would normally interact with. And as a reminder to everyone if you have questions you can chat them on the YouTube or you can send them into rm.communications at narra.gov. With that Marcus will put you off the hook and we will hold any questions that come in till the end. We'll now switch to the next presentation, the digitization quality management guide. I'll turn it over to the director of the policy the records manager and policy and outreach division. My boss Lisa Lisa you're up. Thanks, Aryan. Next slide please Patty. So, we're going to talk about another digitization guidance project. And I wanted to come up first before my colleague my course Lee comes online to just present a little scene setting. It doesn't hurt to look to the past and when you talk to the National Archives, it's our natural tendency. As a reminder on this very busy slide. I wanted to lay out the path that got us to where we are with our current set of digitization guidance, starting nine years ago in 2013. The Federal Records Act was amended, and it really started us down this path as we were pushing we were heading in that direction ourselves, but just as federal records managers across the government. And it led us on the path of saying we need to transition to fully digital government transition to electronic records management. And within those the law there were some very specific items, including two quotes here, where it said again and again, we have to promulgate regulations with digitization standards that will lead agencies to be able to use the digitized images and digital records as their record keeping copy, and to destroy the source records as a temporary record. So, the overarching goal of all of our work is to help agencies transition to digital government to create records electronically in the first place, so we can create and manage our records more efficiently and effectively. But time takes time, it's like a river, a flow that happens, and agencies have created paper records, hard copy records, analog records. And our goal now is to help agencies digitize them so that they can come to the National Archives and we can manage those digital records as part of our nation's history. And the law supported that, as well as NARA's strategic plans have supported this vision, and the OMB and NARA memos M1921 and M2307 they've all pushed us to deliver this guidance and put us in this direction. And I think it's helpful to know that we're going to continue to be issuing we've digitized we've issued the digitization regulations for paper records paper and photographic prints. We have more regulations that are coming, they will handle film records, and we know that just issuing regulations isn't enough. We want to provide more guidance to help people understand what does it mean with this with the regulations they are requirements documents, they are not implementation documents. And so, on the next slide please Patty, where we want to talk about right now is our digital quality management guide. And here is the slide version of this you may have seen before, when I say we issued digitization standards for permanent records. There's from part 40 to part 56, several different pieces that lay out the requirements for each phase of digitization things that we wanted to make sure happened when agencies are digitizing their permanent records. So part 46 or I said part 46 specifically referred to quality management requirements. We think quality is the key. You can catch mistakes while you're making them and fix them easily. When it comes to the National Archives, we won't have an opportunity to do quality management on records that we've received. We can only preserve and make accessible what we've received. So we really want to talk today with our bridge audience and with federal records managers and vendors and and members who are listening to us about what do we mean by quality. And to have that conversation, I'd like to turn over to my colleague my coarsely to talk about this guide. Thank you Lisa hope everybody can hear me okay. I'm going to talk about the quality management components of the digitization regulation and do a shallow dive into the technical concepts behind the standards. I recently published a digital quality management guide as you see the link up here on this slide. And it's the help agencies learn about the various aspects of quality management in digitization, including quality assurance and quality control, as well as the role of objective testing and automation in optimizing quality control and the inspection process. I understand to some of this information may be new to many, but it is helpful for all of those involved to gain a greater fluency with digitization where the source records will be disposed and only an electronic version will be the record copy. Next slide please quality management is the overall function and underlying activities that determine the quality policies objectives and responsibilities and implement them through the planning control. There are different methods and improved methods within the system. The goal of quality management in a digitization project is to prevent defects before they happen. Digitization is an image manufacturing process that relies upon quality assurance, known as QA to establish specifications and requirements and quality control known as QC to test for an inspect for defects. This digitization operation will be proactive rather than rely upon QC inspection to achieve quality. Quality management represents the addressable corrections that can be made during the project. NAR's digitization regulations could be considered a total quality management document that brings together records management practices, quality standards, inspection, and validation steps. The quality control and inspection steps have been optimized in the regulation to rely upon automated processes where possible and limit human inspection to the phases that cannot be automated. By relying on objective testing and analysis for imaging quality, we eliminate wasteful subjective inspection by attributes. Finally, the validation phase is a high level review to verify that all the requirements of a project have been met, and the digital surrogates can serve the same legal and evidentiary purpose as the source records. Next slide please. The purpose of a quality management plan is to define the requirements to be met so that the digital products conform to the specifications of the project. A quality management plan should define the work activities related to the digitization of source records and outline a generic sequence of high level planning and management activities. Digitization is defined as the complete process that generally falls into four main phases. Project planning, the processes occurring before digitization, digitization itself, and the post digitization work. The differences in the media type quality and condition of the source record, the nature of information, or the preservation risk level will determine the approaches for digitization and the metadata requirements for each project. The essential characteristics, also known as significant properties of the source record should be identified to determine the best approach to capturing the information of the record. The quality assurance component of the QM plan must include documentation of the image quality performance parameters selected to capture the information present in the source records. The equipment and device acceptance testing methods and results include design reviews and any training conducted. The quality control component of the QM plan must document the procedures used to inspect the image quality, the procedures used to inspect metadata quality, and the corrective actions taken to mitigate deviations throughout all phases of the project. And the procedures used to verify that digitized records conform to the requirements. Next slide please. The technical specifications found in the regulations are based on the International Organization for Standardization ISO, photographing and archiving standard known as ISO 19264, imaging systems quality analysis for reflective originals. I know that's a mouthful, but that's the official title. And those that standard describes a method for evaluating scanner and digital camera quality through the analysis of the quality characteristics of an imaging system from a specified test target. NARA has adopted the federal agency's digital guidelines initiative, known as FADGI, methodologies to measure the performance of digitization equipment against the tolerances and aim points in the ISO through their digital image conformance program, known as DICE. The underlying assumption is that by evaluating the capture device's performance against the known metrics, the files that are created by that device will conform to the standard. A little bit about FADGI, it's a collaborative effort by federal agencies to articulate common sustainable practices and guidelines for digitized and born digital content. The working group study issues specific to two major areas, still image and audio visual. The still image working group publishes the technical guidelines for the still image digitization of cultural heritage materials. And it's an online guide to still image digitization best practices. The FADGI standards actually were developed out of NARA's original 2004 digitization guidelines for internal access projects. NARA and FADGI guidelines define the set of best practices, but it's the emergence of ISO standards and the improved testing and image performance tools, along with standardized reference targets and open DICE and other analytical software, they allow for greater confidence and accurate capture of cultural heritage materials. Next slide, please. As illustrated in this slide, the proper application of digitization standards is to capture accurate color, tone, and legibility that ensures all information in the source record is recorded. On the left is an example from the National Library of Finland with the various levels of information loss from the color mode to the gray to gray scale, and finally bitonal. In the color mode, all possible information sources are captured. Notice in the gray scale image the loss of color information, which may or may not represent valuable information. In the bitonal image on the far right, you can see a major loss of information because the scanning mode is not able to capture all the information in the source record. NARA regulations do not accept bitonal scans because of this risk. We suggest that agencies digitizing color if color represents the content necessary to capture or interpret all the information of the source record. The image on the far right is an example from NARA's catalog of a properly imaged record, and it includes in the image area, a preservation quality reference target. No digitization equipment or system is perfect. They all have tradeoffs in regards to image quality, speed and cost. In consumer markets, image quality is sacrificed for higher speed and lower costs of the equipment. Many document and book scanners and office scanners, particularly inexpensive ones, and high speed scanners may not meet the limits specified in the regulation, particularly for properties like image noise. Their overall goal is to create digital products of uniform quality through a common set of benchmarks. The primary objective is to produce digital master images that look like the original record and that the files maintain the essential features and information of the original. Next slide please. Agencies must meet the image quality performance parameters specified in NARA's regulations by verifying that their equipment achieves the aim points and tolerances in section 123650. NARA on advertised equipment specifications such as scanner PPI settings or camera sensor megapixels to ensure digital image quality. Agencies must use a quality assurance process to quantify the scanner or camera performance before selecting the equipment by scanning a reference target and measuring the results with analytical software to determine if the equipment meets the technical parameters. NARA must evaluate internal and external vendor imaging systems against image quality performance parameters mentioned in the regulation. And you can monitor equipment performance by quantifying the scanner and camera performance during digitization, and then you will be able to verify that resulting digital files meet project specifications. Here's the section that I don't want to scare everybody with, but it's very important to start understanding. This is a basic, oh, I'm sorry, next slide please. See, now I told you it's scary, wasn't it? This is a basic representation of an imaging target software analysis interface. This is from the Library of Congress's open dice software, which is a free government developed software analytical tool. Step one to the left involves selecting the type of target that will be analyzed and then selecting the quality level that the result will be analyzed against. The values are a lookup table of aim points and parameter tolerances. The values captured in the scan of the target, which is in the middle, the fall within the, in the values that fall outside the aim points can identify the areas that are falling out of the specification. If you look closely at this target, you'll see various things like the color patches and gray scale patches in addition in the four corners, what's known as a slant edge target. And that is useful for resolution analysis. If you see the light green that's highlighted, that is the software identifying regions of interest and it will be analyzed by the software and then on the far right, a report will be produced to show you precisely what area is in or out of compliance. Fadgie has created a set of measurement parameters for specific media types such as modern paper, photographic prints, and microforms. The digital image conformance evaluation program known as dice provides the measurement monitoring component of a compliant program and consists of two components, image targets and analytical software. The evaluation parameters established quality and performance goals for the desired quality level. A hierarchical classification of image quality attributes have been developed to form a taxonomy of digital imaging performance. The taxonomy provides context and a framework for the array of image quality terms and the evaluation of digital image files. Next slide please. I hope this might be a shock to some people, but the number of pixels in a digital image is not a measure of image resolution. Overall image quality cannot be reduced to one component such as DPI. The term resolution in digital imaging is one of the most confusing and misunderstood concepts regarding image quality. Turns terms such as DPI are often used as the sole expression to indicate overall quality with the implication that the higher the DPI or megapixel, the better the image. A more precise definition of resolution is an imaging systems ability to resolve finely spaced detail. Note that the definition does not refer to DPI but to the ability to resolve detail. Image quality is determined by how well the optical system and signal processing of an imaging system performs and not the quantity of pixels in a digital file. The quality of a digital image is determined by how accurate the image system faithfully reproduces the original subject. I'm going to go on a little deep dive here. DPI was an early attempt to relate the input sampling rate of a scanner, meaning you set it to 300 that everybody's been told to do that. To match that to the output printing resolutions needed to create digital prints, this is carried over from the earliest days of digital imaging. Now researchers had determined that the average human eye has a visual acuity of the equivalent of 300 PPI at a specific viewing distance, which is basically holding an 8 by 10 photograph at arms length. Early digital image practitioners generalized then that the resolution on a flatbed scanner should be set at 300 DPI, but it is a misnomer perpetuated by manufacturers to describe the addressable resolution that a device can be set to. But it is only indication of the sampling interval between the pixels known as samples per it per inch. In reality, scanners do not sample at a precise rate and office grade scanners sacrifice speed over quality. A 400 DPI scan on a poor performing device may not be any better than the 300 DPI scan on a higher performing device. And if the device is out of focus, it doesn't matter what DPI it was. Additionally, in the early period of digital photography was the limited amount of pixels in the camera sensor that was the obstacle for image resolution for this reason, the number of pixels is often wrongly referred to as resolution. With advances in sensor design the ability to record detail is defined by the limits of the optical performance, not the number of pixels. I think that's enough. And we'll go to the next slide please. Inspection is the process of measuring and examining testing and otherwise comparing the product with the requirements. The inspection process incorporates sampling techniques to identify the percentage of errors that occur. It is not desirable to conduct a visual review of every file that has been digitized. In most large projects, visual inspection of all files is impractical. Therefore, a statistically valid sample of digital files from each batch must be inspected. The files and metadata created by the digitization project must be inspected to detect any errors. And any deviation from the specifications must be corrected before the files may be accepted. The digital versions must be compared to the source records and verified that they meet the standards in the regulation. The inspection process uses objective measurements as well as subjective methods to ensure quality. Quality control checks may be performed using a variety of tools and scripts. For example, a script may be able to check that and verify files for correct names, proper file sequence, bit depth, color mode, compression, etc. The automated checks are usually conducted as a first pass of quality control. Meaning that if you want to find out the corrupt files that just don't pass basic criteria such as you can't open the file before you refile those records, you need to go do some re-duce. So checks may be run with every batch or at specific time intervals, but human-conducted visual quality control checks often form the second pass of the inspection process. The effectiveness of the evaluation of subjective factors such as image quality or metadata accuracy, meaning automation tool can verify that metadata exists in a file, but it could be that there's typos or the incorrect metadata terminology is entered in there. And so really only humans can determine that to the high degree of success. We'll skip to the next slide please. Finally, I wanted to, we get a lot of questions about validation and verification and if it's our fault for confusing these terms, we apologize, but we are using these terms in a very specific context. So when a digitization project is complete, the agency must validate that the versions meet the standards. The validation phase is a high level review to confirm that all the requirements of the project have been met and that the digital circuits can serve the same legal and evidentiary purpose as the source records. Validation is different from quality management. Quality of management occurs throughout or during the digitization process and involves the inspection of the attributes of each image such as their file format, resolution, metadata, etc. Validation is a higher level review of an agency's digital records and the process that were used to create them. Validation occurs when digitization is complete and must be conducted by staff that were not involved in the quality control inspections that were conducted during the digitization process. Verification is something that happens as the final step in the digitization workflow while you're making the scans. And it's part of the image manufacturing process and verification is an evaluations that the files created by the process meet the specifications that the metadata is complete and accurate. The correct file formats are used and are no more corrective actions to be taken. The purpose of verification is to create the digital file. And the idea is that there is a point in time where you can do remakes and reduce or do corrective actions, but after the project is over and you need a final say that the file itself has been created. We don't really describe exactly who it is that does the validation. The team that does it is one thing, but essentially it is a high level agency level certification. That's too strong of a word, but you're attesting that you are happy with these and that these serve as the digital new electronic version. So in final statement validation could be considered a project level activity and verification to be an item level activity. Thank you very much and I am open to answering any questions or be contact me. And I'm happy to work with you. Thank you back to you Lisa. Thank you Mike. So I wanted to close here with art on our digital quality guide with a resource for everybody watching our bridge today or looking at the slides later on. There's a link to our digitization of federal records web page, we're putting information as we develop it here on this page. And so I circled for you that the two latest items. I think we talked about the digitization quality guide you can link to it right there. We also, and thank you Lawrence for saying it was the end of the fiscal year it's a start of fiscal year. We got out our FAQ on digitizing temporary records. We realized our previous FAQ was no longer current when we issued our digitization rights. I think I think Doug you might have commented hey I like the new FAQ thank you. We appreciate that and perhaps we'll come back at another bridge meeting and cover those topics. So that is a resource for you and I think we're ready to go on to questions Patty so the next slide and Ari and I know we've had a lot come in so we're challenging your question q amp a effort here I think. Alright, wrote Michael Lisa roller fish leaves here we go. Any more information about digitization other than our sources. That's a, that's a great question and also I want to emphasize particularly there's a lot of confusion out there. Fadji. Fadji are guidelines and best practices they are not the regulatory authority that agencies should follow. So, Fadji is a great resource for getting further information about best practices. There's a lot of slight divergences about what the requirements that they talk about. The best thing to do is look at I really encourage everybody to read the new Fadji 23 2023 publication of the technical guidelines. And they include technical discussion about the technical specifications, but the guidelines are published on the website. And that is at digitization guidelines dot gov that's all one word dot gov digitization guidelines dot gov. And there are resources link on that in that website that can lead you to free government analysis software and other resources. It's under a tab called resources. Thank you. Patty, you went back in the slide deck for just a second to slide 27. This is our resource page. And if we have resources from other units are, you know, things that are other than our sources will put them there as well. Because we mean it to be a resource for people who have questions about digitizing. And so we will make some changes and I think we've even made more changes since I last took that screenshot yesterday. Thank you back to the questions. So if we have legacy records that have already been digitized prior to the new digitization regulations and requirements were where they issued, are they grandfathered in, or will we be required to meet the new requirements for those legacy digitized records. Great question we've had many times throughout the entire process we've spent years developing these regulations. So I'll, I'll say the answer again and I'll say it again as many times as we need until we get clarity on what the position is. The digitization standards that we've issued explain what it is that we need at the National Archives to take digitized records that are going to serve the same purpose they're going to become part of our nation's history that the originals that the source records would have been. So the question becomes, I've digitized records, I did my best, you know, I did what was required at the time I thought I was doing a great job, but it would be unreasonable to assume that every digitization project for all permanent records are going to meet these standards, even though we did issue a draft of them back in 2020 so you know we sort of had something out there and tell what direction we were going in. So what are we going to do. So first of all, we're going to issue some an FAQ and try to get you written answers to all of the questions and the very variations of the questions that have come in, because we're do our best to answer here at bridge but you really need to see in writing what is the answer. So in part, wait for those that FAQ to come out that will offer guidance. What we're talking about are records that have been scanned, but they're not compliant, but essentially we have to figure out are they good enough. So one of the ways we will do that is through the scheduling process. Deep breath. I am not recommending that agencies reschedule all their permanent records to determine, you know, what are we going to do. The problem is a question actually of disposition of a disposition instruction. When schedules were have been written, we have written them purposely to be media neutral. So we just said, here's these records they are permanent transfer them to NARA, we didn't usually get into which media. When NARA issued these digitization standards when the regulate when the law came out. That was the beginning of a change for permanent records. We want to give agencies and the public really clear instruction so they know what we're saying. And we may say that yes we're going to go through the rescheduling process. Look at all the factors on how records were scheduled. I mean how they were scanned what were the processes were used. What did you do like where's the gaps from our current standards, and we may end up saying yep, those are going to be good enough to come to the National Archives and be historical records. Right in the schedule specific disposition authority that says these are the scans that were done at the time we're going to take them. And then we can give specific disposition authority that says, what do I do with the source records, because those two may have been presumed to be the version that was coming to the National Archives under a media neutral schedule. We want to make sure it's clear you're going to destroy those source records as temporary and send the permanent records to NARA schedule make that clear. So we're planning to go through the scheduling process to answer that question on a case by case basis. I think that's the nutshell of like the strongest part of that answer there may be other options as well. Elena, I think you were the one who emailed that question, and we will give more advice and more guidance on other potential options, such as, can we find the born digital versions and send them to us. Can we will you have a potential exception to send continue sending the source records to the record center for a little bit longer, because as a reminder, records that are records in the records centers do not need to be digitized to send to NARA as a permanent record so the 2024 deadline is not a barrier to transfer for records that are in the record centers. I hope that was clear. Lisa in the fullness of the bridge schedule I want to keep moving on with the agenda, and we'll we'll circle back to the the other questions that have come in. After we hear the next presentations I want to give equal time to the other presenters. Thank you so much. Thank you for watching the clock. No problem. Next slide please Patty. So now I want to introduce Andrea Noel from our records management oversight and reporting program to talk about the managing social media records records management assessment report. Andrea good afternoon. Thanks, Ariane. Good morning and good afternoon bridge. Today I'm presenting a high level overview of the records management assessment report on managing social media records. Next slide please. The managing social media records assessment report was published to NARA's records management assessment webpage last month. For convenience the link will be added to the chat. Next slide please. NARA started this assessment in May 2022. The objective was to identify which social media platforms agency actively used the way in which they use them how social media records are captured and managed. And finally their level of compliance with NARA's bulletin on managing social media records. We identified participating agencies based on the number of social media platforms our team found they were on and the level of activity we observed on their social media profiles. Participating agencies were the public affairs office for the secretary of defense centers for disease control us geological survey NASA, the national endowments for the arts, the departments of education and security. The library of Congress, NARA and the small business administration. Next slide please. Our methodology with this assessment included having agencies one respond to a pre-assessment questionnaire to provide relevant agency documentation like policies, standard operating procedures, training materials. And three have relevant agency stakeholders endure a 90 minute interview with our assessment team. We also conducted interviews with representatives from five third party applications agencies employed to schedule capture or manage social media content or records. These applications were archives social sprinkler global relay coros and Hootsuite. Our team synthesized all the information gathered and developed a comprehensive report which summarizes our treatment of the topic, general best practices, agency challenges and areas of non compliance with NARA's social media bulletin. Please note, it is customary that NARA assessment reports offer participating agencies some anonymity. The report will not reveal what information was expressed by which agency. Overall, there were 12 findings and 14 recommendations. Here are highlights of those findings most agencies had in common. Lack of ambiguous or inconsistent criteria used to find what is a social media record. Inability to consistently redress established records management policies and practices when changes in social media usage occur or when social media platforms create new functionalities that resulted in new types of social media content. Lack or misapplication of agency specific and general record schedules. Lack of resources to sufficiently and efficiently manage social media records. Undefined records management roles and insufficient how to records management training for social media personnel. Poor coordination between the records management program and programs tasked with leading social media activities. Unverified feasibility of social media platform export functions and other social media related applications to sufficiently capture and manage social media records throughout their entire life cycle. And finally, lack of senior level support to adhere to records management guidance outlined in NARA's social media bulletin. Next slide please. In conclusion, all federal agencies should evaluate or reevaluate how it uses social media, what types of social media content is being generated. Quantify agency compliance with NARA social media bulletin and utilize all applicable recommendations from this report to ensure proper management of social media records. Agencies must confirm that consistent methodologies exist for one, identifying social media records among the different types of social media content. And two, informing appropriate staff how and where social media records must be preserved and managed throughout this life cycle. Finally, all federal agencies should greatly consider dedicating more resources to IT solutions that automate the capture and management of social media records. Appendix E of the social media report has links to different resources that speak to successful management of social media or electronic records. And Appendix F lists a number of electronic message social media and website data capture tools or applications agencies can explore to enhance its records management practices. Myself and the NARA assessment team offer many, many thanks to each of the representatives from the participating agencies. Your transparency time and effort is instrumental in helping NARA better serve the entire federal government. Next slide. If anyone has any immediate questions, please drop them in the chat or email them to our communications at NARA.gov. Thank you. Thank you for, and thank you for teaming up the questions. We do have one that's come in is NARA going to update the social media bulletin as a result of the findings from this assessment. And I'll do it. Yes, seems appropriate to for me to answer that versus Andrea, who did that thank you for leading that social media assessment where I really enjoyed reading that report. The answer is yes. And I can stop. Yes. What was the question? Will we be updating the social media bulletin? We're going to look at our social media guidance. We want to look at how social media is being used. Look at the recommendations in the report and determine if there's a new flavor of what we said back from 2000. 2014. Yeah, 2014. The principles are the same. So that's what we'll be looking for. We're looking to see is there anything that we want to expand on the bulletin. I do not have, however, a timeframe for issuing that guidance. It is something that will be under draft development and consideration and FY 24. If you have any questions about it, I would please send them to our communications. They'll make it to our team. You can send it to our policy at NARA.gov. The team that's working on it. We'll be very happy to hear your thoughts on how that guidance could be updated. Thank you. Thank you, Andrea. And I'll add my thanks again that closes out your section, but as always stick around, we'll see if any come in now. Next slide, Patty. Updates on the annual move and speaking about that from our permanent records capture team is Derek Kennedy Derek. Good afternoon. Yes. Good afternoon. Everyone. My name is Derek Kennedy from the permanent record capture team. And I will be speaking today to you about the updates on the annual move, specifically the 2024 annual move as we all know. The ERA 2.0 and particularly the 2024 annual move was a labor of love, but that labor did not come without a couple of bumps to get it to where we had it uploaded into the system. So Patty, if you could next slide please. On October 3, we successfully loaded 5129 candidate list transfers into the ERA 2.0 for agency review and submission. That covers roughly 73988 cubic feet of permanent records from 131 agencies. This is where the few bumps in the road comes in. And due to some form migration issues, some authorities did not migrate properly. And we are in the process of correcting those in ERA 2.0. For CD months, so roughly 805 transfers from 51 agencies did not successfully load. We will be loading those in the future. And when they do get loaded into the system, we will provide communication to those agencies affected. So wanted to bring that even though the slide says January 1, 2024 is the deadline, the deadline is actually December 1, 2023 for transfer request to be proposed by agencies. Again, not January 1, 2024, but December 1, 2023 of the deadline for transfer request to be proposed by agencies. Next slide please. The 2024 annual move tips. We have first up, the annual move job aid. So the annual move job aid itself is about how agencies will be able to transfer records that were provided by NARA for the annual move. At job aid, you will see a link to on the next slide, but it's at the top of our list because we do have some new records offices that come in and they don't necessarily know how to propose those records through NARA. So we did come up with a job aid to target that specifically that you can go to if you kind of get stuck or if you knew you need guidance, this is what the annual move job aid will provide for you. Also new for ERA 2.0 is that when the 2024 annual move draft TRs are put into the system, they automatically go to the approving official. In the old system, the transferring official would be able to see the TRs first but now is the approving official that the draft transfer request go to, which is quite a change from the original system. The annual move transfer requests are displayed on a dashboard in the my task on an unassigned task tab. That again is if you're an approving official and you're looking for these records. When you log into the system to say hey they're ready for me to do whatever with, you will have to go into the unassigned task or the my task is where you will see them act. Of course, if you would like to reassign transfer request to a transferring official or even to another approving official, you have the option to use the reassign function. And again, the reassign function is that if you go into the transfer request, it'll be next to the I want to write your C3 dots, you click on the three dots and the first option you should have is to reassign you click on reassign. You should be a list that you'd be able to go through and you select who you would like those transfer requests to go to. And also, and area 2.0 to search for a move transfer request, you can use the advanced search. Now the original system, you were able to search in a keyword search bar by saying, what's the annual move and you'll be able to get them all to come up for your agency where in this iteration of area 2.0 you can still do that, but you have to go to the advanced search function and towards the very bottom of the advanced search. So let's say user, and you type in 2024 annual move, of course the status and draft, and the type of legal transfer will be the annual move. You hit the search queue and then it should return a result for you of all the 2024 annual move transfers for your agency if you choose to go that route. Also new for area 2.0 is there are a couple of agencies that break down a transfer request by the charge account. So this not this year, but in the area 2.0 system AC charge accounts are now included in the created office field. We will see the charge accounts and parentheses and the created office field so that is for those agencies that like to break down their transfer requests and send it to the various internal check off that they need to perform and either to get to get those proposed to narrow, which I think is a big thing that will help a lot of agencies out when it comes to charge code breakdown. Next slide please. The next slide, we have our annual move useful links. As you can see, that we have a row era 2.0 training materials and these materials will help you with everything that you need to be successful in era 2.0. Of course we have the assessing guidance and policy. And we have our annual move of permanent records website that you can go to and we basically will walk you through how to go about preparing yourself for annual move with steps you need to take during the annual move. And also have links to other parts of narrow that you may need to get to in reference to the annual move so we do have those links available and we're always updating our website to best serve you the agency. We also have a classified permanent record training and resources here because we do have some classified permanent records that are part of the annual move transfer request. So we do have a link to training and resources about how to successfully get those proposed to narrow. And then like I said from the beginning slide that was before this one is the annual move job a and it's annual move job a allows you to pose a transfer request to narrow. Next slide please. So, thank you again for your time. And if you have any questions or comments, please let me know. Thank you, Derek. We do have a couple that have come in. One question. What happens next after we submit the transfer request to TRs. Okay, so after you submit the transfer request for TRs by the December 1st deadline, then it goes over to the custodial units and narrow and they will then approve or return your TRs based on the information provided in them. And they have to January the 15th as a deadline to approve or return your TRs. Okay, here's another one that's come in. If we received an annual move transfer request in the RA in the unassigned area of the RA 2.0. Does that mean we won't receive any more annual move transfer requests for our agency. Since you stated that you are still working on getting unassigned annual move requests in the system. Okay, can you repeat that for me one more time please. If we received an annual move transfer request in the RA in the unassigned area of the RA 2.0. Does that mean that we won't receive any more annual move transfer requests for our agency. Since you stated that you are still working on getting unassigned annual move requests in the system. You still will receive annual move transfer request in the unassigned task as we get those 805 loaded into the system. And if they're going to come from the annual move, they're going to come to the unassigned task on your dashboard for the approving official. We can send you that question right that came in via email so we can share that question with the permanent records capture team and you can sort of make sure the answer so exactly what you want to say. Okay. Did Derek say that code charge code sorting of TRs is not possible in the FY 24 annual move. No, actually quite the opposite. The charge codes are now included in the created office field. So if you were to download and export of CSV for RA 2.0, that would be column E and then column E you will see the created office. And then you will see in parentheses the charge code account. And if you have the charge code account of parentheses that should allow you to sort as needed so you could provide that information internally to the agencies to get approval to propose those TR to narrow. Thank you for that clarification. Are my FRC colleagues around Jefferson. Andrea, you're still here. Still here. Okay. I do that it come in sort of maybe see if you can handle regarding reference requests not found. We are still receiving notices of reference requests not found by a physical mail letters. Is there a way we can receive these digitally. Yeah, well, if the reference request was submitted through the Arcus portal, you should be able to go in and see check the status. And unfortunately, Arcus did not let you know when that status has changed. But it's, it sounds to me that you may be receiving something else. And I guess what I would say is, if you could just reach out to your account manager and let them know exactly what it is you're receiving because you shouldn't receive. I wouldn't think you would receive any paper letters sent to your agency letting you know that a reference hasn't been found unless there was something extraordinarily unusual like a permanent records and then there's some sort of investigation going so I'm just going to ask you to reach out to your account manager because we'd like to know more about what you're experiencing there. And I think that's that's a fair answer. Here's another one that came in is the FRCP making preparations for increased traffic on Arcus from now until June 30. The website is already prone to becoming overloaded and crashing. And I think, I think what that is speaking to is exactly what we were trying to accomplish in recently updating our IT network. My understanding is that we were trying to prevent the reliance on maybe one server. I am not a technology person. So, but we were trying to minimize the risk. And in doing so, through that process, there were a number of outages that happened over the summer. So to answer the question. Yes, we are taking steps to update the network and to make it more reliable. But, yeah, so the vendors are closely monitoring the equipment and the status, but I don't think we've had an outage since, I guess, late mid to late summer. So, yes, we're in anticipation that we're going to have more traffic. We are, we're doing what we can to to prepare for that. And both. So, now we'll go back to digitization so Lisa, Michael, picking up where we left off. Here's the question our agency has an MOA memorandum of agreement with our local communications quadrant, who have a contract out right now for digitizing all the paper records, we have no control on the quality once the documents are scanned. Thank you for asking that I think that one was a YouTube question so I'm glad we got here, and I appreciate you sharing that information, and I just want to say, this is an information sharing space. I'm glad I'm glad you felt comfortable stating that. Now, what I'm hearing is, it will take a while for contracts to be updated to include the new digitization standards. They did just come out in May. So, we recognize it will take time for agencies to pivot so they can add the requirements to their contracts, and then those contracts would include those requirements for quality management. I would hope that the digitizing that's being done now where you said we have no control on quality would be something that now that it's clear how important quality is would be questions you could ask the contractor what are you doing. I'm assuming they're doing something, and then think about what that would mean if these were permanent records, and we had a lot of questions on this so I'm just going to try to be clear and careful again. It's a big risk for NARA to take permanent records, these digitized images, and an agency cannot tell us what type of quality control was done, how often, what tools, what happened. So the quality management is conversations we will have for permanent records that have already been digitized, along with the fuller scope of all the other requirements on documentation and gaps and, and do you know what we scan. So I think that would be my answer to Caitlin and to others who are asking about about what do I do with previously digitized records. So here's one that's sort of along those lines. Is it possible to get a clarification about the disposition of the permanent records after digitization. There is misinterpreted information as to whether they can be destroyed after digitization. I'm sorry. I'm so excited to answer. I almost cut you off there. I did cut you off. So, a clarification about disposition. Oh yes, we have a wonderful resource for you. I'd recommend looking at the general record schedule 4.5. General record schedule 4.5 says, if you meet the digitization standards, you can use that disposition authority to destroy the originals. Going forward over time as agencies are able to meet these standards, put this information into contracts, determine how they're complying if they're doing the digitization in house, they will be able to use that GRS, that's your authority. The digitization, the digitized records are permanent, the source records are temporary, the disposition authorities under 4.5. However, if you can't meet those standards, you can't use the GRS. And that's why I'm emphasizing the path of rescheduling for disposition instructions to get that clarity, because if you haven't met our standards, you can't use the GRS. That's the disposition authority for destroying the source records. We want to have that really clear. I hope that was a clear answer for a clarity question. I'm triaging the questions, knowing that we are running very close to the top of the hour. Here's I think one that we can answer pretty quickly. Do you advise that the validation process is best performed on every scan page? I'll start and Mike will finish. Oh, did you mean the quality management on every page? Sure. The validation is for the entire process. And that's one of the things we're trying to communicate today. You do verification and quality as you're digitizing the imitation. I know what the scope is. I did the quality. We've double checked the file formats. We've got the meta. It's the whole thing is done. You've validated these records are done. We're trying to manage them as permanent records and destroy the temporary Mike. Did I answer that question right? Absolutely. I mean, I just want to thank the questioner for asking the question. And a gentle reminder that, you know, this is precisely why we wrote the guide is to help understand the terminology. I recommend reading the regulation. There are in particular in answer to the other questions. You really can read the regulation that is written in a way that you could pull out requirements, for example, and use them in your in your contracting language, etc. But it's very important to understand the different distinctions of when we use a certain term. You do not. You'll never get through your project if you look at everything, but there are times where you have to follow the regulations. Thank you. Here's another one. I think we can handle pretty quickly. Does Nora have. Quality management plan examples or references of quality management plans that don't use software such as the dice example. I would recommend going to the library congresses. Well, they're the sponsor. Um, the, the Faji guidelines. Web page digitization guidelines.gov. And there's a resources tab there. And there's a couple different ways that you could start applying different approaches to analyzing image quality. And the simple ones of the other ones that are left that have come in over the transom throughout bridge will forward to our and our standards and let the team take a more detailed dive into that. And I guess I'll call back Lawrence to close without this afternoon. Ryan, for all that hard work facilitating it was, it was a challenge today. We had a lot of great questions from the audience. So to all of you really appreciate your engagement. It's what makes these meetings as valuable as they are. And I want to thank all of the excellent speakers for the presentations today. It's a lot of substantive content there. And we're looking forward to following up with all of you in terms of the questions and feedback that you might have about the products that we've issued. And the activities that we're undertaking now as we really get into FY 2024. So you can see on the slide. Our next meeting is December 12 should be a great meeting close out calendar year. We'll have a lot to talk about as I mentioned earlier, we'll be talking about annual reporting starting in January. And I'm sure a lot of updates on many of the projects that we talked about today and work that we're planning in FY 2024. So thank you again for all of you attending and participating. And we look forward to doing this again in December. So thank you everyone, and we'll see you next time.