 Welcome, and thank you for viewing this presentation. This is a project briefing on the Naval Postgraduate School's Athena Project, a collaborative research tool for the Naval Education Enterprise. Narrating this presentation are from the Dunley-Ox Library at the Naval Postgraduate School, Associate University Librarian Edward M. Corrado, and University Librarian Tom Roscoe. That's me. The views and opinions discussed in this briefing represent those of the individual presenters and not those of the government agency for whom we work. This project briefing will explain what Athena is, who is involved, and give some background. We'll give a project update providing development history, progress, and challenges faced. And finally, discuss future work, next steps, and conclusions. So what is Athena? Athena is a student-originated project at the Naval Postgraduate School, being developed in collaboration with NPS, the NPS Foundation, and industry partners. Athena is an online collaborative research discovery tool that allows researchers, sponsors, and leaders across the Naval Education Enterprise to access and query information currently stored in numerous separate databases. Completed in ongoing research projects, as well as potential future research projects, are organized into communities of interest, and then presented to potential researchers based on their education, military occupational specialty, and expressed research interests. To its mission, the Naval Postgraduate School provides defense-focused graduate education, including classified studies and interdisciplinary research, to advance the operational effectiveness, technological leadership, and warfighting advantage of the Naval Service. NPS is a graduate university offering masters and doctoral degrees and more than 70 fields of study to the U.S. Armed Forces, DOD civilians, and international partners. Primary areas of study include science and engineering, information science and operations research, international defense studies, and defense management, with all research and education focusing on military and defense applications. The school has approximately 2,500 students, with 1,500 students in residence, and 1,000 distance learners in degree programs. In addition, NPS offers executive education and short courses to over 10,000 students per year. The student body is made up of primarily mid-year, mid-career officers, most from the Navy but also representing other branches of the military, with some civilians and international students. NPS is located in Monterey, California, just south of San Francisco and Silicon Valley. School is run year-round on a quarter system. Almost all graduating students produce a thesis or capstone project for their research. Most of this work is unclassified and submitted to NPS's open access institutional repository, though some projects are classified and restricted. NPS is part of the Naval Education Enterprise, NEE, which can be viewed as a consortium of the United States Naval Universities, which include the Naval Academy, Naval War College, Naval Prosgraduate School, and the newly created Naval Community College. For purposes of Athena, we should also include the United States Marine Corps University, which played an important role in supporting Athena's early development. In today's rapidly changing world, the importance of education and the ability to understand, create, and share knowledge has increased. This is also true of the military, and the importance of managing information and sharing knowledge are key components of military education. Naturally, a library plays an important role in supporting this. At NPS, the Dudley Knox Library administratively consists of the Dudley Knox Library, Graduate Writing Center, and Thesis Processing Office. The library has a staff of approximately 20, not including the Writing Center or Processing Office. The library has approximately 1,050 print titles, about 700,000 electronic resources, and about 170,000 serials in print and electronic. The library receives approximately 1,000 visitors per day, and has a growing number of virtual resources and services available. The library also manages CalHoon, the Naval Postgraduate School's institutional repository archive for research materials and institutional publications created by the NPS community. The repository currently houses approximately 63,000 items. The library also has a classified facility, Restricted Resource Services, which in addition to print materials manages about 2,000 restricted electronic theses. The NPS Alumni Association and Foundation supports NPS by funding defense innovation research and supporting both student programs and NPS initiatives. They prepare our next generation of military leaders to best serve and protect our country. The NPS Foundation has played a key role in helping to shepherd the Athena project. Most recently, the Foundation has supported the technical development of the project, managing the software development and industry engagement, with the ultimate goal of having produced a viable version for handover for testing and implementation at NPS. As a collaborative research discovery tool, Athena allows users to browse, search, filter, and view completed, ongoing, and proposed research across databases. The system consolidates the projects and then provides users curated collections of research sorted and filtered by topic. Completed research, ongoing research, potential research, researchers, research sponsors, and resource consumers are matched based on key words and other attributes, including the user's expressed personal preferences. The objective of Athena is to leverage the untapped potential of thousands of technically skilled and operationally experienced officers at the Naval Postgraduate School and attending in-resident joint professional military education. Linking this researcher capacity to service headquarters and fleet research needs will provide insights about how to overcome our most complicated challenges, while simultaneously sharpening the officer skills and ensuring that these future senior leaders have early exposure to the complexities of service level concept and capability development. In addition, Athena will help the larger Naval Research and Development establishment and Naval leadership stay abreast of research trends across disconnected organizations and to track an individual researcher's trajectory after his or her graduation. In its development, Athena leverages the Microsoft Teams platform because of its collaborative functionality and Teams current widespread adoption at NPS. As was mentioned, Athena was a student-originated project. It started as a student thesis project and began in 2019. Early on, it had support from the Marine Corps representatives at NPS, as well as Marine Corps University, and the attention of MCUs and NPS's bosses. Athena originally leveraged data from NPS sources. These included our institutional repository CalHoon, where student theses are published, our homegrown student information system Python, which houses thesis topics, and NPS's research database of funded and unfunded research projects. In addition, the original data included scan theses from Marine Corps University. As the project has progressed, it has both focused on refining how data could be pulled from the various NPS data sources, while also looking to expand and connect to data sources outside of NPS. The library, as well as NPS's central IT unit, ITACS, has been involved in advisory capacity from early on, with the library contributing expertise particularly related to discovery and metadata. The following slides will show a few screenshots of the original wireframe and original working beta version. Thank you, Tom. Although in some ways Athena has similar goals to an institutional repository, the goals of an IR are only one part of what Athena hopes to accomplish. Development of Athena has so far focused on using data from CalHoon, because it is available in public release, but there are other potential sources being considered in the development of Athena, including research performed at the Marine Corps University and other DoD universities, Naval Research Laboratory, and items deposited at the Defense Technical Information Center, or DTIC. For those that don't know, DTIC is the repository for research and engineering information for the United States Department of Defense. As mentioned earlier, NPS has an institutional repository operated by the library that we call CalHoon. CalHoon is the Naval Postgraduate School's digital repository for research materials and institutional publications created by the NPS community, including completed theses and dissertations, technical reports, scholarly articles, and more. But only those that are approved for public release in unlimited distribution. We do have items that are not public release, but they are managed separately due to their restricted status. Notice that I mentioned that CalHoon contains completed works, which is true, of course, of most institutional repositories. This is great for its purpose, but NPS, NEE, and the Navy, and the DoD as a whole, also have a need to understand what research is in progress, not just what the completed artifacts of research are. The information may be stored in various systems, such as a database containing descriptions of funded research or the thesis tracking system used at NPS that contains approved thesis topics. We also want research funders, sponsors, and military leadership to be able to propose topics that they would like studied. This is being developed through what we're calling communities of interest in Athena. Communities of interest can be any relevant interest. It could be undersea warfare, unmanned systems, or oceanography, for examples. One of the key strengths of the communities is it will be a virtual place for researchers and others with similar interests to connect with each other. Researcher profiles may include research interests and past experience that may be helpful to other researchers. When I went to library school, I took a reference class. This was back in the days before the internet was really a viable information source for most things. The professor asked, what was the most important reference resource? Various students in the class guessed different things. Was it social science index? Was it dialogue? Was it the card catalog? How about an encyclopedia? No, proclaimed the professor. Instead, the professor said it was the phone book. Finding the right person to provide information to help you answer a question is a key, especially if you are looking into something complex. However, what if you don't know who to ask or where to look? We are hoping that the researcher profiles in the communities of interest will help with this. We often get asked at MPS who is researching X or who knows about Y. We hope that a combination of the communities of interest in the researcher profiles will help answer these types of questions. Perhaps it can be kind of a new version of a friend of a friend for the Naval Education Enterprise. We don't know for sure, but we hope it will help and connect in people with similar interests and people with experts that can help them with their research challenges. The DOD also needs to connect research problems, research sponsors, and others with researchers. However, with such a large organization as a DOD or even just within the Naval Postgraduate School, this is virtually impossible since no one knows what everyone's done. How many people listen into this talk? Think they know what all the faculty at their institution are researching or that they could even answer that in a matter of days if they were asked. Maybe at every specialized or every small institution, this could happen, but I would say at 99.5% of universities it just isn't reality. One of the ideas here is to help students also choose thesis and dissertation topics that are focused on Naval or DOD needs. This will ensure that relevant research is done that is important for the Navy, but also for the future careers of our students that are military officers. Another goal is to have news feeds from various sources that are relevant to different communities of interest. This can be current news, new research, and other relevant information. Right now we're in the process of gathering which new sources to include and are also investigating how to include them. One of the challenges with current awareness functionality from the library perspective is how do we provide access to license resources? One part of this is how do we curate them? That could easily be a full-time job in NSOM. Is there a good way to automate all or at least most of this? The next part is how do we provide access to them? If Athena was just for MPS that maybe wouldn't be as big of a problem, but our licenses to scholarly content do not include access to the whole Navy or the whole DOD, we likely could provide titles, impossibly even abstracts from past experience. We know that users from outside of MPS will want access to items and will expect us to be able to provide them. So is it better to not include items that are behind a paywall? Or is it better to include them? We haven't quite decided yet as there are good arguments for both ways but it is something that we're thinking about. The current interface for communities of interest is still being developed but this is an example of the community's tab from an earlier version. As you can see people can join different communities. They can also browse all communities. It may be that in the future we need private communities but we're not quite sure yet. We will see how researchers use it in the future and then determine what their needs are. The purpose of this presentation is not to give an overview of the specific technology being used because the goals of Athena are agnostic to it. However, I did want to point out that Athena is being developed with the Microsoft Teams environment in mind. MPS is an Office 365 subscriber and therefore we have access to this platform that supports the social interaction features that are a key component of Athena. Using these built-in features should make this part of development go much smoother and faster and also it will be familiar to many of the users. While there are various data sources being considered for Athena, one of the key ones is Calhoun which uses dSpace and we harvest the metadata from dSpace for Athena. Currently this is a one-way street but I can envision some updates of metadata happen in Athena that will want to migrate back upstream into Calhoun but this won't be part of version 1.0. We also have a homegrown student information system that includes thesis and dissertation tracking module as well as other information about users such as which curriculum they're molded in and these will be leveraged for Athena. The original wireframe in alpha version of Athena was developed by marine students with some support from our central IT department. For various reasons the MPS foundation is now taken charge of the development with the goal of transferring it to MPS once it is completed. There are weekly project meetings with different stakeholders and also there are special meetings with experts or leads in certain areas. For example, I have had multiple meetings with the foundation development team about metadata and how the metadata editor should function in what features are needed. One of the aspects of this project that the library has been most closely involved with is metadata. The MPS foundation hired an undergraduate computer science student from a local university to work on metadata related portions of the project. I think she quickly learned that metadata and keywords are not nearly as straightforward as she might have thought. I think another challenge for her as it is for me sometimes my lack of service in the military is that many of the terms are specific to the military. Because of this looking at keywords, taxonomies and other metadata related terminology from other institutions may not be that good of a match. It's maybe does not guide you as much as you would think it would or maybe as it would in other situations. One thing that is being worked on is a simple to use metadata editor that community champions who are not necessarily library and information science professionals can use to simply edit, add and assess keywords. They also will be able to move them into different broader categories as appropriate. This is still in development so I don't have any interest in screenshots or a demo to show you but it will only be open to certain people who will be provided some level of training in that we trust that they will only adjust metadata as appropriate. But we are still working on some approval and review mechanisms as well as coming up with some policies, procedures and best practices for modifying metadata within Athena. The descriptive metadata in Calhoun from our theses and dissertation collections like at many places is a mixed bag. We have some fields that are rather consistent and controlled and others that are more freeform and inconsistently applied. Mostly this varies with age of the thesis. While new fields are occasionally added they are not retroactively added so you know we just don't have the staff to go back and update the metadata for older theses and dissertations or for other content for that matter. You know a less is something that can be done in bulk. More on this in a minute. We also have a need for ongoing metadata maintenance and creation of metadata. We need sustainability. We think this is where the community of interest champions can play a big role. We also hope to be able to leverage domain experts including students that might be available for temporary duty assignments after they graduate but before they are transferred to their next position or if they arrive at NPS before their classes begin. Returning back to existing metadata here's a screenshot of the typical descriptive metadata that we have available. For more recent theses is in dissertations we have pretty good categorical data but that I mean things like advisor service discipline degree department and the like. What we don't have is any subject information other than author supplied keywords. These as you would expect varying quality and usefulness some are great some just repeat the title and like author supplied keywords can be sometimes they are more author focused than they are user or reader focused. That said while there are some limitations to author supplied keywords they still are valuable access points and can inform indexers that are assigning controlled vocabularies. We found about 50,000 unique words in our theses in dissertations with the help of domain experts including two former provosts and some stash students who happen to be US Marine officers. These have been cut down by about 60% by one of the people hired by the foundation. That's still a lot but it is beginning to narrow down. One of the things that we hope to come out of this list of known or preferred keywords is that future students can pick from them when assigning third keywords. Aneodotally we hear that some students struggle with picking keywords and we believe having a curated list will help them while at the same time helping users of Athena to discover research on a given topic. Since we have only been working with materials out of our IR so far for the first release of Athena, we haven't really worked much with metadata from other systems. We do know however that this may be a challenge as some of the sources have no descriptive metadata at all and some may have a controlled vocabulary but it's at a very high level that is not specific enough for our needs but one step at a time. This obviously is going to be an ongoing challenge. If we get it right at this moment in time it will need to be continually updated as time goes on and as we add new data sources. You can't do this once and it will be solved forever. Librarians of course are aware of this but it's important to keep this in mind when discussing projects such as Athena with others stakeholders who may not be actively thinking about the ongoing support needs of metadata. Also the military has many unique acronyms in terminology so some level of domain knowledge will be necessary to move this forward in the future. One of the challenges that we have with many projects at MPS is a class of cultures. We have a military culture, an academic culture, a government bureaucratic culture and in many cases including this one also industry and entrepreneurial cultures. While having people with a diverse set of workplace cultures has many benefits can also lead to some challenges. One of the main challenges we have is the language used to describe things. We talk a little bit about this when covering metadata but this challenge is also apparent when working together on a project. That said we have tons of good people that have a shared goal and it's working out and it will work out. There's also a push and pull related to open access versus restricted access. While academics in an academic reward system often rewards openness sometimes there's a push not to make things so open. How we will deal with this in the future is up in the air but for now access to Athena will be restricted. Currently version 1.0 is in development. We anticipate it being available for a small set of people outside of development team to start testing in the next few weeks and then it will be more open more broadly to users in early 2022. The early testing will let us know where we need to put more resources. We don't want to share it too broadly too quickly because we need to get the feedback to make sure it meets the minimal expectations of our users. If we share it too soon we're afraid that people won't come back when it's more fully developed. We see that Athena is an ongoing project and we are looking at ways to ensure that it's sustainable over time. The people involved all believe that Athena feels an important need and if that is correct we are sure that ongoing support will be available but we need to lay the foundations to make that happen. As you know with almost any technology project you can't just build it once. There's a need for ongoing development plus a need for metadata support and of course user support. Implementation of version 1.0 as I mentioned is expected in early 2022 and then we're going to start working on adding other database connections besides Calhoun. We need to connect it to our research database and some other databases. Of course there'll be the continued challenge of open access versus restricted access just how open should Athena be and also what about restricted resources that are either classified or have other restrictions on our distribution. And of course who knows what the future will bring and what other future possibilities there will be for Athena. We hope this project briefing was interesting and useful. We believe this work is important and could have wide breaching applications while Athena is focused primarily on the needs of MPS in the Naval Education Enterprise at this point. We believe that Athena could be useful throughout the DOD industry and in academia. It's quite clear from my standpoint that the library has a lot to offer a project such as this. Metadata is key to making this work and while there have been some experienced programmers involved with Athena I think they were a bit surprised to learn how complicated descriptive metadata could be. Librarians have a lot of education and experience in this area to offer Athena or any such project. Thank you for listening and please reach out to us with any questions comments and suggestions. All along the process employed has been very thoughtful inclusive and iterative. There have been many other projects that we've been involved with but few have been on such a thoughtful path so we have been pleased to support. And we see the potential in the tool itself as well as potential benefits from other developments such as the metadata editor. Also it always helps if you can call on the U.S. Marines.