 Welcome to the fourth event of the National Academy of Sciences, Engineering, and Medicine Conversations Series on Accessibility and Inclusion in STEM. I'm Dr. Rory Cooper, and I'm Chair of our Planning Committee. I'm a white male wheelchair user due to a spinal cord injury and a distinguished professor at the University of Pittsburgh and senior career side just in the U.S. Department of Veterans Affairs. This conversation is the fourth of five distinct conversations. We hope that you were able to watch the pre-recorded keynote lectures by our speakers before attending today's live discussion. But if not, you can always view them after the event. This event is also being recorded and will be available after the event. Our goal for this event will be to offer the opportunity for active discussion among the speakers, panelists, and those of you listening in via the slide out. For those of you via slide out, you can ask questions in the Q&A section of the website. We will consolidate your questions and bring them into the discussion as available. At this time, I'd like to introduce each of our planning committee members. I will call on them and please give your title and affiliation. So we'll start with Dr. Emily Ackerman. And the white woman with brown hair and glasses. I use she, her pronouns, and I'm a postdoc at Harvard Medical School. Thank you, Emily. Next, Dr. Sheryl Burgstahler. Hello, I'm Sheryl Burgstahler from the University of Washington in Seattle. I go by she, her pronouns, and I'm the director of Accessible Technology Services and the Duet Center there. Thank you, Sheryl. Next, Dr. Chris Acheson. Good morning, everyone. Chris Acheson, University of Cincinnati, Professor of Geoscience Education, white male bald glasses. It's good to see you all this morning. Thank you, Chris. Our next panelist is Dr. Carolyn Solomon. Hello and good morning, everyone. My name is Carolyn Solomon. I am the director of the School of Science, Technology, Accessibility, and Mathematics at Gallaudet University. I am a white woman with brown hair and glasses. And it's good to see you all this morning. Thank you, Carolyn. Our next member of the planning committee is Dr. Julian Brinkley. Hello, I'm Julian Brinkley, assistant professor of Human Center Computing and Director of the Drive Lab at Clemson University. I'm a black male with a beard, and I use he, him, pronouns. Thank you, Julian. Thank you, everyone. At this point, I'd like to turn over the meeting to Dr. Julian Brinkley, who will be our moderator for today's discussion on Accessibility, Inclusion, and STEM in the context of computational based research and education. Thank you, Julian. Thank you, Dr. Cooper. Before we dive into our conversation with our panelists and speakers, I want to provide the opportunity for our speakers to give a brief overview of the highlights from their prerecorded talks. We hope that those of you who are joining us today have had the opportunity to view those prerecorded keynote presentations. If not, they are available on the event website. We had two keynote speakers for the conversation. First, I'd like to call on Dr. Nicholas Judici, Professor of Spatial Computing in the School of Computing and Information Science and cooperating faculty in the Department of Psychology at the University of Maine. Dr. Judici. Thanks, Julian, and hello to everybody at this panel. So yeah, my name is Nick Judici. I am Ken Generally Blind, which kind of influences some of the work that I'll be talking about. And I'm a professor at UMaine, as Julian said. So my talk in this series was about best practices for improving how we create and how we visualize data in STEM fields. And this really related to any type of data or numeric representations for scientific thinking. So diagrams, figures, graphs, maps, what have you. And my key take home, I guess I would say from the talk, was that traditional approaches for data visualization are often exclusionary and often outdated in terms of, one, their understanding of human information processing and how it works. So they often do not maximize the brain's native abilities to integrate sensory information. And two, outdated in terms of the technology that they use. So often this technology does not utilize modern multimodal interfaces and interactions. So what I was advocating and am advocating are bio-inspired, multi-sensory data visualization approaches that better mirror how the brain actually works and therefore are more intuitive, more realistic, and more inclusive. And so I structured my talk around a series of kind of problem statements that I think kind of situate the issue. And then I followed these with some positions and some hypotheses that I argued move us toward some new and promising solutions. And I'll just very briefly go through this. So the first core problem that I talked about was the following. So despite its name, which I think is a bit misleading, visualization is not about vision or visual information. And I described that the reality of what is being visualized is actually spatial information and spatial relations. And I went through a bunch of different examples of this. And then this kind of position motivated my second core problem with current approaches. And that is that most visualizations rely only on the visual channel for conveying information. And I argued, this is kind of silly because it's not how our brain actually works and how it actually processes spatial information, which is done through all of our senses and really a highly efficient way of integrating all of our senses. So therefore, effective visualizations should model this and they should also be multisensory and include multiple channels of information delivery. And so the rest of my talk, I discussed some various reasons why considering these issues are beneficial for supporting inclusive learning in STEM across lots of different disciplines. I provided some underlying theory for why non-visual techniques work so effectively and in explanations of how that makes sense from a cognitive and neural standpoint. And then I gave some ideas on how multisensory visualizations could be designed and implemented, including just a couple of examples from work in my lab. And so I dig into a lot of that in the talk. I think I'll stop for now. And I'm excited about hearing our next keynote's overview and then our panel discussion I think will be really fruitful and your questions will be, I look forward to entertaining and discussing those. So thank you. Thank you, Nick. I'd like to now introduce our second keynote speaker, Dr. Sherry Asencott. Dr. Asencott is an associate professor at the Jacobs Technion Cornell Institute at Cornell Tech and the Technion. Dr. Asencott, floor is yours. Thank you, Julian. And hi, everyone. Thank you for inviting me to be your keynote speaker along with Nick. So I'm an associate professor at Cornell in Information Science and my research involves leveraging technology to support people with disabilities and specifically people with visual impairments. And for my keynote, I discussed a couple of projects that fall under the theme of supporting people with low vision. So low vision is a visual impairment that falls short of blindness. So people with low vision do have functional vision and they typically use their vision in their daily lives but they still have a visual impairment that affects their everyday life and their ability to perform everyday activities. So in my lab, we try to think about technological trends and also about the role that technology is assuming within our society. And we attempt to leverage evolving emerging technologies to solve problems that people with disabilities experience that are currently unmet. So I spoke specifically about two such problems that people with low vision experience. The first problem involves finding a product at the grocery store and specifically finding the product once you're already in the correct aisle, actually finding the specific product on the shelf. And then the next problem is navigating elevation changes or terrain changes in the path ahead. So for example, stairs, curbs, ramps, and so on and so forth. So these are very practical problems but they actually represent perceptual challenges that currently have no tools to support people with low vision. So in the case of finding a product on the grocery store shelf, this is a visual search task. So this is where you have to scan the environment and to locate a certain target of interest, a certain object of interest. And this visual search task, this is very distinct from different types of perceptual tasks like reading fine detail, right? So if you're like looking, if you're reading something print on a sheet of paper or on your computer, if you're looking at a street sign and trying to read that street sign, so that's a very distinct perceptual task that's reading detail, whether it's close or far. But, and there are existing tools to support seeing details, seeing fine details. But what we found was that there are no existing tools to support visual search. With the second task that I mentioned, the walking, navigating terrain changes or elevation changes in your path, this has to do with depth perception. So once again, depth perception is a very distinct perceptual task than seeing details or visual search for that matter. So a lot of the work that I do looks at these practical tasks but tries to go deeper and understand the perceptual and behavioral challenges that are involved in supporting people in the process because all of these things are very much linked. So these are the two problems. And then I presented our process for first understanding specifically what challenges people with low vision experience through studies that we conducted and then designing a technology that attempts to support people in these tasks. And the technology that we used is augmented reality. So I showed examples of two prototypes to augmented reality prototypes. The first one is QC and QC presents visual cues with augmented reality. It uses computer vision to locate a target product on the shelf and then it uses the cues to make the target product more visible and help the person with a visual impairment perform the visual search task. And then for the second task of navigating elevation changes, we focused on stairs in particular and here we used augmented reality once again. So we are detecting with computer vision the presence of stairs and the exact location of the stair edges. And then we superimposed with augmented reality highlights on the edges of the stairs to once again make them more visible. So a lot of the design challenge involves once again understanding perception of people with low vision and which is extremely challenging because of the range and complexity of visual perception and human ability. And to incorporate those into designs that hopefully one day in the Nazi distant future we will be able to include in commercial products. Thank you, Sherry. Very interesting. I'd now like to introduce our invited panelists who will join us for a structured conversation with Dr. Judici and Dr. Asencott. We're joined today by Sadia Gopala-Krishnan, an associate professor in the Department of Agricultural, Environmental and Development Economics at the Ohio State University. And Mr. Andrea Sundaraman, a PhD student in Rehabilitation Science and Technology at the University of Pittsburgh's Human Engineering Research Laboratories. Welcome to the conversation to both of you. I'd like to kick off our conversation with some structured questions amongst our panelists and the planning committee members. For those of you listening in the webcast via Slido, please add your questions and thoughts to the Q&A chat and we will work to incorporate your ideas into the conversation. So I'm gonna start off with the first question. Really, when I think about collectively the work that you all have been discussing in your keynotes, it brings to me the thought of the interdisciplinary nature of computing research and how human-computer interaction is really involved in that. I believe all of you conduct research that in some way is focused on the human. So I'm wondering what principles and practices can we borrow from human-computer interaction that can be applied to accessibility and inclusion and computational work? Sherry, would you like to start us off? Sure, I can start us off. I think that we all agree that the first thing is to understand the user, understand the person involved. And in human-computer interaction, that's a central part of our work. So in my keynote, for example, I started out by each project by talking about two studies that we did where we observed and interviewed people. And I think that especially when it comes to people with disabilities, there is such a wide range of human ability. And there's also typically, not always, but typically a difference, a distance between the person that is providing the accommodations and the person that is receiving the accommodations. So what I mean by that is that typically the people providing the accommodations are not the ones who have the disability. So it's incredibly important to not just understand the user, but take a further step and actually involve them in the process of in human-computer interaction, I would say design, but in the case of helping someone with accommodations, we can use that term too. Designing, forming, figuring out exactly what they need and how it will be incorporated into their work. Wonderful. Satya, would you like to chime in? Sure. Thank you. Great to be on this panel. Some of the work that I do has, in my own work, I think about interactions between humans and the physical environment that we're in. And one of the things that stands out is that when we try to do, when we bring humans in computing together, we end up with unexpected ways in which we can discover things that not only increase accessibility or make something more user-friendly for people specifically with disabilities, whether it's visual or for someone like me in a wheelchair or with the quadriplegia, but it also can be scaled to a level or that's why when we design things that are specific to a particular problem, we end up missing the fact that it could be integrated. And so one thing that I have found is rather than have targeted solutions for specific problems, we need to be thinking about integrating the ways in which we can address specific needs that can then go on to become universally designed that will not only meet needs of people who specifically need, for example, speech to text or text to speech again, but now that's being used everywhere. So people use audio cues all the time, not only because they need it, because they may be visually challenged in any other way. So that to me is key to understanding the way we can explore human computer interactions. Thank you very much. Andrea, what do you think? So disability is, just should be thought of, perhaps just among the variety of human experience. And so in the same way that in developing any product, you would want to bring in a diverse focus group. The same thing needs to happen as other panelists have said when developing assistive technology or when developing any technology if we include people with disabilities and to learn the way in which they might interact and what variety of disabilities and it might be somebody might have visual disability or a hearing impairment or maybe a combination of disabilities. I'm both visually impaired and quadriplegic. And it's, whenever we design any piece of technology, it's important to understand the breadth of users for whom we are intending this technology be used. Thank you very much. Nicholas, would you like to add something to that? Thanks, Julian. And yeah, I agree with the panelists and starting with knowing your user and designing based on needs is important. I think another important thing is when you're trying to promote inclusion, it involves, I argue, it involves a lot of multi-sensory cues, but that might, those can take lots of forms, whether they'd be a different sensory or motor or cognitive disabilities. But I think what's a really important is to understand the sensory channel that you're using and the technology. So I think what happens too often is that, for instance, we understand a lot about vision and visual technologies and we use that knowledge as designers and developers, or many people do, and try to just apply it to some other modalities. So for instance, touch shares a lot of commonalities with vision in terms of spatial relations, but the sensory bandwidth of touch is about 500 times less. So you can't just take everything that you see and spit it through some visual or tactile interface, even if you have some elegant algorithm and expect it to make sense. And so I think what's really important here is for designers to understand the, the, to make something more inclusive, you might use more types of user interfaces and more channels, but you need to understand how those channels work, something about the underlying physiology of the human and then something about the technology that you're using to try to display that information. So it matches and maps onto the sensory abilities and strengths that you're capitalizing on. And I think that that doesn't happen enough. And when it does, you get a much more integrated, seamlessly integrated product. I think that's important. Yes, definitely. Thank you very much. So in terms of the next question, I have a student in my lab with low vision who is a key contributor to much of the research that we're involved in. We've actively worked to make our tools and materials more accessible for her to include the emphasize in the use of some, you know, largely inaccessible materials like PDFs and incorporating the use of more accessible project management and collaboration tools. In terms of supporting researchers with disabilities, is there enough emphasis in your opinion being placed on making computational research more accessible from a tooling and a quick perspective? Sherry, would you like to start us off? Sure. I think the answer is no to give you a concise answer there. There's definitely not enough emphasis. I mean, I think any of us who publish in conferences that actually try to do something about accessibility that try to make their process, the submissions and the published papers accessible, we know how painful it is. I mean, just the fact that we're still using PDFs, PDF accessibility is a nightmare. Some, I mean, I just don't, I have my students do it for me now because I just can't handle it. They're instructions on the web, they require special paid software, somehow they never seem to be quite right. And to be honest, I'm not convinced that the guidelines that those accessibility instructions are trying to meet, I'm not convinced that those actually make the PDFs accessible for whatever quote unquote accessible means. So I think that there's a lot of goodwill in our communities, but some, and I think that's a wonderful thing, but somehow we're still falling short and we need to figure out why that is. And when you speak to the guidelines, that makes me think about many of the guidelines that I attempt to follow in terms of trying to make PDFs accessible for conference publication and things like that. So just to kind of follow up on what you were saying, you think there could be some improvements at even a baseline level to even those types of directions? I do, and this is mostly for my personal experience. I haven't studied this, I haven't studied this issue. So for example, I have low vision, I have a visual impairment, but I'm not a screen reader user. So things are often very inaccessible to me because of the software that I'm using. I use magnification and I do use text-to-speech software, but it's not a screen reader. But for example, with Overleaf, which many people use, somehow the PDFs end up being all jumbled up. That's just one example. Another example I would say is with all of the, again, good intentions of adding alt text, a lot of the times you still get alt text that's just not helpful. So once again, I don't think, this is a tricky issue because I don't think that the burden should be necessarily on the research community, on the researchers. I think we need to collectively try and figure out a way to handle these issues. And right now, unfortunately, I still have more questions than answers when it comes to things like that. Interesting, thank you very much. Nicholas, do you have a viewpoint on this? Yeah, I agree with Sherry. I mean, this is a challenge. I don't think it's ill-will, but I think a lot of it comes down to education. And also just what it means to be accessible. Someone can run an automated accessibility checker and it doesn't do anything about the semantics of an alt text. So if they have something that says graphic 394 in the field, it will still pass, right? So... Or a graph of blah, blah, blah. Yeah, I mean, it's very much not what you would get when you ask someone to visually describe what they're seeing. So I think also in part of the education is better crosstalk. So you get people on the back end that maybe you're doing very little with accessibility and then people on the front end of some UI trying to make something accessible but don't have the right hooks to get into to make things work. Or my favorite, you get like the front page of something or the front interface of some app works and then once you get to the next level it doesn't work with it and I do use a screen reader. It's just very frustrating because a lot of these things are not labeling a button that says submit is a really simple thing to do. And I understand that it's probably something that a lot of people just don't realize if they don't need to have it. It has an icon or it has something that lets them know but if you're using a screen reader that can be the ability to submit a review or a paper or whatever it is or not. And it's always this is getting worse I would say not better and it's weird because there are more guidelines presumably to help but as Sherry said, they don't always even work and there's conflicts and people don't know them. And so I think that there needs to be a better way of disseminating this information and getting it across to all the people that are in a project in a process and a design and maybe not just a few HCI people that are at the end trying to make a UI work. Thank you, Nicholas. Tadja, I wanna ask you this question as well. Do you think that there's enough emphasis from your perspective being placed on making computational research more accessible from a tooling and equipment perspective? No, I agree with everything that Sherry and Nick said and I'm not visually impaired. So my challenges are different from what we've heard so far but it goes back to something that Nick mentioned in his when we were talking about the previous question about understanding your user when you're developing technology. So I have tried to use dictation software just to reduce the load on my single hand. I'm a single-handed user with the left hand amputee and the first thing that I'm often asked to do is use a one-handed keyboard and a one-handed keyboard really is not, it is not adept, is not easy to learn and it's very clunky and dictation softwares don't update in the same ways that Nick mentioned a couple of minutes ago. So there is really this disconnect in terms of what will be helpful to what is made available and some of the fixes are actually quite simple but involve more potentially human resources and in thinking about what would be an effective design I don't think we have enough people testing it out for their own needs. Things like, my work is pretty computational and I work in a data intensive environment and I also use spatial data quite a lot. Some things that are just taken for granted is selecting spatial regions which involve holding three buttons at the same time and being able to select specific rows and columns in parts of the matrix. These are tasks that are impossible to do with one hand let alone with someone who may not have the use of a single hand either. And so the design of software hasn't quite gotten that far and if you have to use speech to select specific rows in a spreadsheet that would increase the time taken to complete the task by at least a factor of 10. And that is the sort of time that we're talking about in terms of doing simple tasks. So I don't think I have an answer here but definitely there are a lot of open questions in terms of getting software ready. Type equations is another example of how something can be tremendously more time consuming even if you were to use a dictation or any other software. Thank you very much. Andrea, would you like to weigh in on this topic? Is that I think we've been covered but following on from what Satya was saying, the I'm a user of multiple assistive technologies for accessing the computer voice input and screen magnification and a screen reader. And what I find is that many of the engineering softwares that MATLAB and Mathematica and even to a certain extent, SPSS that there are a lot of limitations as to what you can do with the accessibility software. And even in MATLAB, I found that perhaps it makes sense that you wouldn't be using a screen reader with certain visual elements. But even if I can see them with the screen magnifier having the screen reader on running in the background on the computer causes the program to crash. So these are programs that are used essentially throughout the field. And so we need to push more onto the software developers to take care that their software is compatible to the maximum extent possible with assistive technology. Thank you very much. Julian, if I may, can I add something? Yes, please. I think that for the last few questions we've been talking a lot about technology and in a sense that's what this conversation is focused on technology to support people. But I think that one important thing that we should not just gloss over is that the more important thing here is people to support people. And yes, that's a whole other conversation in itself. But as I go on in my career, I mean, I started out as a grad student thinking that technology was really the answer. And in some cases, in some small ways it was the answer for me. Technology got better than many things that I couldn't do suddenly became possible as a person with a disability. But at the same time, the larger issues and many of the things that we're talking about now I think need to be addressed with human infrastructure, with human support, human understanding, human resources. So I think that the technology should not be discussed in isolation. It's part of our relationships with one another, our institutions, our society, and we need to think about it in that context. Yes, yes, very apt comments. And I think that leads us right into the next question, which is one of the type of support that may be available in research settings. I was reflecting on a conversation that I had not too long ago with a colleague of mine in computer science. She is a person with disabilities and she was basically reflecting on her experience in graduate school. She wished in our conversation, she expressed to me that there was a greater emphasis on accommodating her needs when she was pursuing her graduate studies. So what can those who run or lead labs do to create a climate or culture of accessibility and inclusion for researchers with disabilities? Nicholas, would you like to start us off? That's an important question and it's a challenging question because people always have different needs and different accommodations. I think there's a couple of things that I tell students. First, they are their biggest advocate and they are not their only advocate, but in many cases, people are, even if they shouldn't be, are unawares and don't know the challenges and as a student coming in or a young professor starting as well, I think it's often equally as challenging, being able to reach out to whether it's the person in your lab or moving up to disability services or an OEO person, but in saying these are the things that I need to do my work really important. I think within the lab, it's really important to also just promote a culture that's based on in my lab, we have people from lots of different disciplines and with different abilities and that includes disabilities is just part of that diversity. Being open to that as a culture sounds maybe trite, but I think it's really important because it allows people to kind of want to, not just wanting to fit into like some clique, but wanting to be able to talk about their needs, their differences and what they might bring to help someone on one thing and someone else may help them on something else and just kind of fostering that as a, as part of the ethos of the lab. For me in my experience, both, in my experience being as a graduate student starting as a young early professor, but also in the lab that I've developed and has had lots of students, I think makes a big difference and it just sets that culture that really is more opening and accommodating and makes people feel comfortable. If someone feels comfortable, they're gonna be much more willing to speak up and to thrive than if they're trying to make something work on their own. Thank you, Nicholas. Andrea, what do you think? I think that many of us might come into a new lab or such a situation with feeling that maybe we need to prove that we can do the work and that tends to make us not ask for accommodations and that's something that's certainly difficult for the person with the disability to get over and ultimately something that we do need to do, but that it's made much easier if there is a genuinely accepting environment that people are encouraged to ask for assistance in whatever way they need. We all have different abilities, whether we're talking about disability or not and so when an organization embraces that, I think it does make it easier for new people coming in to say, well, actually, yes, I do need some assistance with this particular task. Thank you. Sacha, what do you think about just this concept of leadership and what leaders in these lab environments can do to create a climate and culture of accessible and inclusion? Do you have a perspective on that? Yes, I think what both Nick and Andrea said is part on that to a large extent, the leader of a lab, whether it is the PI in a lab or the head of a department, it really sets the tone for what becomes the culture and what is acceptable and in creating that level of comfort for someone to go and advocate for themselves. People, as Andrea said, people are often hesitant to go and ask for something unless they know that this is a welcome request and that it will be received positively. I can speak for myself a little bit. I'm not someone that is scared to advocate for myself but this idea that I can do things on my own and being independent is also something that I place a very high value on. When I started teaching at Ohio State and this was almost 12 years ago, I was assigned a TA for my class and it took me two years to actually request the TA to carry 100 exam scripts to the classroom. Now, carrying 100 exam scripts with one hand while also writing my chair and navigating the chair with that same hand, as you can imagine, it's not easy. And it's really not hard for me to assign this task to my TA who was assigned to me to help out with the class. And for some reason that wasn't immediately the my go-to response, which is not the case now but it takes a little while. So as a PI or a faculty member, I feel like opening the conversation while introducing students to your lab or introducing them to your research group with one acknowledging my own needs as well as opening up the space for individuals to express what they need in order to be most productive in their own roles is critical. And there is a power differential as much as students postdocs junior faculty can advocate for themselves. There is a power differential which makes it easier if the person who is in a position of power actually opens that conversation and creates that space. And then anyone can go ahead and it makes them comfortable to ask or to express what their needs are. And this is tricky as Nick started out as said in the beginning that a lot of the information about people's abilities or disabilities as the case may be, are privileged information. So we don't, in some cases it is visible in a lot of cases it is not visible. And so we can't make judgments about whether someone needs a certain technology or needs accommodations of a certain type without them actually talking to you or without them telling you what they need. And so it's important to balance that fine line but we won't get to know if someone needs anything or not unless you actually open up that conversation and give them the right to say, no, I'm good and I think I'm okay, I don't need anything or here are the things that would really help me to do better or to be more productive. Thank you very much. Sherry, you really led us into this discussion with your comment about putting people at the forefront and really that type of human infrastructure. What do you think really about this topic of creating climates and cultures of accessibility and inclusion? Yeah, I think everything that's been said is absolutely correct. And I think when in doubt ask, it's really important to just ask someone if they need support in any way, whether or not they have a disability. I think that's an important thing to ask anyone who's working with you to show that you are there to open up that space. I think that we all have needs and problems and a friend of mine who lost her sight as an adult, she says to me occasionally that, I had problems before I lost my sight too. So it's not something that's unique to people with disabilities and it's important to support everyone. Well, sticking with that topic, I would like to kind of talk a little bit about mentoring and support. Are there ways in which, and we've talked about this somewhat, but I wanna kind of delve into this a little bit further, are there ways that we can, or any kind of specific strategies that we can leverage to provide mentoring and support to persons with disabilities and research settings? So how can we foster a culture where people feel comfortable asking for help? Satya, would you like to get us started? Sure, the first thing I would say is, and this is not limited to disability as she just said, the first is whoever is the leader of the lab to set the tone for not just in the beginning when someone joins your lab, asking them what are the things you need, let me get you set up. And there is an orientation process which is often overwhelming for a lot of, whether it's graduate students or for stocks moving to a new place and starting a new position. And often that's where everything, that's where the initial support comes, we'll set you up with someone who provides assistive technology, go figure it out with them. And that's it, like you hear from them in that first few months, you decide what kinds of technology you might need. And there's absolutely no follow-up after that. And people's needs change, people's needs change based on even having used or being in that work environment for a little while. This is especially true for lab work as well, right? You don't know what you need until you have actually been part of a lab, knowing whether it's space, whether it is navigating to a specific space in your lab using a certain experimental technology, or when it comes to computing needs, I think it's very hard to know whether the learning curve is very steep or whether it's easy to pick up or not. So the first thing I would say is, of course, the beginning and the initial opening that space to have a conversation about what one needs is essential, but it's also essential to follow through and say, is it working out for you? What else can we do? Or do you need to talk with someone on a regular basis and make sure that we're keeping up with what you need? That I think is critical. A second thing is in fact, and I don't know how much this works and different people have different levels of comfort, but I do think actually opening up conversations to share one's own experiences and actually letting your lab or letting people in your lab who may or may not identify with similar types of scenarios to just understand how things go. So having open conversations about what is easy, what is hard, can one student in the lab help out with certain tasks that another student can't do or can take longer or might be more challenging is something that can be done and that can be facilitated and coordinated quite well. And the third of course is having conversations about careers and what is out there, what are the things that one can do or how people navigate their own spaces and having those conversations on a regular basis rather than only when the need arises. I think these are fairly critical in terms of mentoring and creating a culture where folks are just comfortable expressing themselves and talking about what their needs are. And I think it's also really important to not make assumptions ahead of time. I think one thing that's really promising and encouraging to me is that we don't make assumptions for example, about an individual's gender anymore or at least we're moving in that direction. And if you talk to kids these days, they really don't assign genders to someone that they don't know. And that's becoming the culture of the way we interact and speak with one another. I do think that is the direction in which we wanna go in terms of not assuming what one's abilities or needs might be but rather holding space to have open conversations about them. Very interesting, thank you very much. Andrea, as a current student, I would really like to get your perspective on this. What do you think with respect to what we can do regarding providing mentoring and support? Well, I am in the fortunate position to have a mentor and my dissertation advisor has a disability and although it's not the same as mine, I think that gives it great empathy and it makes it much easier just to speak to him about any issues that I have. I think it becomes more difficult. There certainly aren't that many labs. I think we all know they're not that many labs that are run by people with disabilities. And so I think it's important then to be able to reach, if that's not the case, to be able to reach outside the lab and if the director of a lab does know someone else, maybe from another institution who could provide some of that mentorship that would be helpful. But certainly there's a need to see role models who have disabilities in your field and to know how they accomplished what they have. Also, it's important for people who are running labs who don't have disabilities to try and keep the conversation open and not, I think sometimes maybe there's a fear of inquiring too deeply into somebody's disability and you do have to be careful of that. But for those who run labs and don't have disabilities, I think making sure that your students know that you're open to those conversations is important. Thank you very much, Ray. Interesting commentary. Nicholas, what do you think? Do you have a perspective on this really? I have no perspective, Julian. No, that's not true. I have many, many perspectives on this. I mean, I agree with the theme that we're talking that most people are hitting on is communication. I think that's hugely important. A couple of things that I think are, that I've done and modeled on other people that I think make a difference is, some labs are really hierarchical. Especially medical labs and there's a certain fields or like that. I think that's problematic. I think there needs to be someone that, someone in the lab wants to be able to talk with me about something that is a disability or a challenge of whatever it is that that's part of my role, I feel as someone that runs a lab. And I think that's important. We also do a buddy system. So people that are come in are often assigned with a more senior student. And that just gives them a really an immediate person to be like, to be able to have as a sounding board. And I think finally, one of the things that I really feel strongly about, and this is independent of disability, but it ties into any types of abilities is that we really encourage people to be willing to fail in the lab. There's very little room in academia anymore to fail. No one wants to fail a test or not do well on some project, but ultimately, if you're failing on something, that means maybe you're trying something new and it's not working and it gives you a new way to think about it, or maybe you're trying something that really isn't working because of an accessibility issue. And you say, okay, I've tried this and it didn't work. Great, let's figure out how to make it work. But I think being willing to put yourself out there and that kind of something that Andrea talked about, you know, wanting to prove yourself in some ways. I think that's true with many students and even more so with people with disabilities. And, but if you try something that doesn't work, then that's also okay. And how you move beyond that as part of the learning process, I think is important in accepting that and having that as part of the lab culture. Thank you very much. That's very interesting. I wanna follow up on that a little bit. So you talked about how you basically use a buddy system to really pair students up. Just keeping or moving forward that cultural, you know, conversation. Is it sometimes difficult to establish those pairings? Are there any challenges in that regard or any best practices that you think could be leveraged to really support the ability of others to use a similar type of process? It's particularly challenging because we have a lab that pulls from a lot of different backgrounds. So some people are coming from, you know, kind of psych and experimental backgrounds and some more from engineering and computer science. What we try to do is put people on that in the beginning, they move around to some different projects and they're paired with someone that has some knowledge of at least a part of their background. So they may not be a computer programmer, but maybe they have some computing background and they'd be with someone that has, you know, some connection to other types of coding. We don't wanna match people exactly up because we want them to be exposed to different skills but to have some relation of interest. So if you're coming in as just or primarily as someone that's doing experimental behavioral work, then you probably wouldn't be paired with one of the hardcore developers. But since people are moving around to some different projects to get a kind of their feet wet, they're getting experience with some of these different things. And sometimes that changes the pairings, but that's how we start. Thank you very much. I would like to pivot a little bit to talk somewhat about institutions and organizations and their role in supporting inclusion and accessibility and computational research. From your perspectives, how can organizations that provide funding like the National Science Foundation and others play a part in providing resources to support greater accessibility and computational research? Do you see a role there? I saw you nod your head, Nicholas, so I'm gonna go back to you. Sherry and I were talking about this in an earlier call and I won't speak for her, but we were both saying, yeah, I mean, there's certainly a role. I think there's a lot more of a role in agencies considering how resources can be used to help buy equipment, for instance, assistive equipment and technology, no matter what the disability is, is often way more expensive than generic equipment and often that's stuff that's not, maybe in the original grant, there are supplements, but they're challenging and they have limitations. So building in this idea of how to support technology and as Sherry also said, not just technology, but human resources, someone that can help, as an assistant, help me to format a paper that may take me an hour to format because I'm going through very slowly with my screen reader and trying to find where there's an underlying word that I don't want to be. My site assistant can do that in two minutes and so it's a better use of time. I think supporting that more explicitly in grants and there's a big push in the computing side of some grants that have broader participation and these plans to increase participation, which I think is really important, but it often doesn't get, it's often focused on other groups and not necessarily disability and I think that that needs to be a part of these diversity and the DEI movements and certainly in grant funding because it makes the grant run better, it makes it more efficient, it makes everyone feel like they're able to be maximizing their abilities. Thank you. Sherry, Nicholas referenced the conversation that you all had. Would you like to add some commentary there? Yeah, I think Nick hit on all the key points there. Thanks, Nick. So I agree, there's definitely a role for funding agencies to better support people with disabilities with funding. I think the DEI effort is good, but it's just a start. We need to move further. I think the National Science Foundation has been pretty good about including people with disabilities as underrepresented groups, but many other institutions and organizations need to do the same as well. And just like he said, I think that for funding purposes, we need to raise money to fund our students and postdocs and ourselves also for the summer and it can already be extremely challenging for people. So when a faculty member is faced with having to raise additional funds to support a student with a disability, I mean, I'm sorry to say, but that's a disincentive to work with a student with a disability right now. So we need to make that a whole lot easier and smoother. Thank you very much. At this point, we'd like to open the discussion to those of you listening in via the webcast. We've been monitoring your questions and we'll bring them into the conversation at this point. So please feel free to add your ideas to the live question and answer and we'll start discussing those. So we have the first question and I'm just gonna read this verbatim so that I am presenting the information as in a representative perspective as possible. I would appreciate hearing the speaker's thoughts on fair, findable, accessible, interoperable, reusable data practices, which the federal government has moved towards but is not necessarily accessible. Anyone have any thoughts on that? Yes, I think fair data practices again is something that we often try to get to standards of data management and agencies like NSF actually require a data management plan that does meet the standards of being able to access and replicate. But I don't even think that accessibility is included in what the guidelines are for managing your data, making sure that the data are readable or are maintained in a certain format. The only thing that they do is again, going back to our earlier conversation that things should be kept on accessible PDF documents but not really anything beyond that. So I think it's a really good question but I don't think that there are any, at least I haven't seen any codified or systematically laid out guidelines for how you store and manage data in a way that provides accessibility to users with different needs. What do you wanna like to follow up on that? I think fundamentally it's, if you have a data set that's available and interoperable, if we come back to the issue that the software used to access it manipulated may not be accessible. So if we're talking about, in that case, accessibility, I think accessibility from the disability, accessibility perspective, it comes down to whether the software used to access the data is accessible rather than, if you have, as long as some piece of software can access that data in that is an accessible piece of software, I think that is helpful. But that's, we're not there yet, it seems. Thank you. Anyone have any additional thoughts before we move on to the next question? Okay. So here's the next question from the chat. And again, I'm gonna read this verbatim. Learners with hearing loss benefit from visualization of data, which would be counterproductive for learners with vision issues. How do we reconcile these two issues? Would anyone like to take the lead on addressing that? Multi-sensory visualizations. I mean, that's a glib answer, but this is kind of the point of my talk and others that people have mentioned, but the advantage of trying to find, the advantage of having a spatial substrate for most of what's being visualized is that space can be specified through vision, through touch, through different types of auditory, sonifications, through kinesthetic actions. So the more that we can combine these redundantly, the more that we are supporting different types of learners, whether they be different sighted people that learn differently, blind person, a deaf person, other types of folks. And so the more that we can build that in, much like how our brain processes this kind of mosaic of information from different senses, the better we'll be able to accommodate and support people. Would anyone like to follow up on that? I think Nick is right. I think you need to present people with options. And this will help everyone. I mean, it's not just a disability issue. People are different. They learn differently. They grasp the material better in different ways. And it's important to give, if you're working with students, it's important to allow your students to access the information in multiple ways. I think all educators should do this, whether you provide slides in advance and a live lecture and a video recording. You know, any one of those modalities can be helpful to different students. And the same is true for presenting information in an accessible format to different people. This is Cheryl Berkshaw. I'd like to ditto that, both of those last statements. But tying in with another question you had about funding agencies, including NSF, we have a lot of experience with the Dewitt Center and with the different grants. And we have several under the cyber learning program has changed the name now, but for digital learning. And those projects, hundreds of them that NSF has funded and many of them creating digital learning products that can be used in K-12, they do not require that accessibility be considered in those projects. And so we in our projects have encouraged researchers to do that. And many will come back with, well, we just, it's not a very large grant, which is true. And we can't deal with it with all types of disabilities. They at least should address the issue and report in their findings that a limitation of their study was that they didn't test it with students with these types of disabilities. But it's not even on the radar screen to even that degree. But that's one thing that funding agencies can do too is require that these products being created for K-12 schools and post-secondary education other environments where legal issues apply regarding accessibility shouldn't be created these products without even giving it some thought. Yeah, that's so true, Cheryl. And I think that there's an analogy here to regulations in medicine, right? If medications aren't tested on certain populations, then they will potentially not be able to be used on those populations and those populations will be able to benefit, right? I mean, this is one of the core principles of IRBs of internal review boards. It's the principle of justice if I'm getting my IRB history straight. And I think that the same is true with technology and people with disabilities. There's all this new technology that's being developed and it's not being tested or designed with people with disabilities in mind. And so the outcome is that people with disabilities are excluded. The technology is not being designed for with people with disabilities are not being supported and the result is that the technology is going to be inaccessible and people with disabilities are going to be further marginalized. And that seems to go against this principle of justice to me. And I think that, I mean, I agree with that. And the medication, the medical model, people say, oh yeah, well, people can hurt and can die and what have you. And I think it's somewhat of a silly argument, particularly because there's so much evidence with inclusive design, universal design, whatever you wanna call it, that when you build this in, as we were just saying, something that's put in for a blind person like voiceover on the iPhone was put in because you had a flat screen, it was totally unusable. Works really well for all different types of people with different types of learning disabilities. It can help with people as English as a second language. There's so many secondary uses that end up helping all different types of people that just make it good design. And I think, again though, I think a lot of this comes down to education. I don't know if it needs to be legislated, maybe as different policies, but certainly education saying this isn't, you're not just supporting some niche, you're actually supporting way more people in a better way by considering these things in your research and your tech development. And I just feel like that message is talked about in a lot of the groups that we may talk to, but it just doesn't seem to be getting out there even though I fundamentally believe it to be true. And one issue to consider too is intersectionality. We'll work with projects that work with students with learning disabilities, for example, in one way or another. And often they're creating products or strategies that are not accessible to someone who is blind or someone who is deaf, making the assumption that someone who has a learning disability doesn't have any other access issues as well. And sometimes it appears that it's because, well, it'll be easy for us in technology to just deal with learning disabilities. Other disabilities are more complex. But again, we need to at least get it on the table that they can recognize that as a limitation of the work they're doing. If it's a pilot or something, maybe it makes sense for what they're doing. So I don't wanna stop research, but I want people to realize that they're not completing the project as it should be as far as creating a product that can be used by everybody. Well, I have a question as far as that goes. And this is one that I get asked quite a lot. And I oftentimes have people reaching out to me about this and it kind of ties into what you all are discussing with respect to involving people with disabilities in the actual design process. Big question I get asked is, how do you get access to these groups? And this kind of ties into the next question that I'm getting ready to ask from the chat. But how does someone who's interested in well-meaning and really wants to include individuals with a specific disability in their research process get access to those individuals to assist? This is Cheryl again, but one good way, if you're at a post-secondary institution or you have a partner who is, is through your Disability Services Office. They're not gonna give you names of people in their office, but many of them have discussion lists, of course, and you can send out messages. And so if you create a short paragraph describing what you'd like as far as input on a project, and I think in most cases, you should be offering funding for that, by the way. But you might be able to, you might be able to have some of these students contact you to be in that study. We've been very successful in that in institutions even beyond our own. And it's a good way to recruit students with disabilities and let them know that their input is of value to your project. Anyone else like to, yes, go ahead, please. I mean, it's a good question. And it's a hard issue, you know, Cheryl's right, reaching out, it is somewhat biased by geography. So there are often people, for instance, the group I know best blind and visually impaired people are often more in urban settings because there's more transportation. So place like Humane's kind of out in the woods makes it hard. So, but that doesn't mean, you know, you have to think of different ways to do that then. So go to, for instance, I do a lot of work at conferences where a lot of people attend. One thing that I think might be, what I wish would happen is that there was a better network. You know, we all have our own networks and we can reach out to people. But one thing that maybe is part of this education that the granting agencies could potentially do is help to connect people that are doing similar work on other grants and try to figure out a, I don't know if it would be a database or at least some other just ways of collaborating, connecting with other groups that may be working with people with similar demographics or similar interests and similar disabilities and help to kind of get other people to be involved. I agree with Cheryl though, I think it's important that these people feel like they are getting compensated and that they're being a part of something that actually is going to benefit them as part of these grassroots kind of giving input, not just feeling like another data point. I've been in a lot of studies where people are like, oh, you're a blind guy, great, can you run on my study? And then you just kind of feel like a hamster on a ball. And you really, I think it's important for any type of experiments like this, but for people that feel like their input is actually affecting, potentially affecting the future and development and that they're having a role in that. I couldn't agree with that more, both what Cheryl and Nick said. You know, we have in our professional associations, we have emphasis on DEI, we have, I can speak to our own Applied Economics Association, we have subcommittees that focus on women in particular or people of color, what we don't have is that intersectionality across different ways in which people may be marginalized, but also bringing disability to the center of those conversations. In fact, we don't even really think hard about where conferences are organized or there is access to the conference venue. I can think of at least a couple of times where I have not gone to a conference because it would not be possible for me to get there with public transport and just, you know, how am I going to get to a place with an accessible transportation or taxis and whatnot. So I don't think that disabilities are even part of the conversation in a lot of professional communities and I think that needs to happen for exactly the reason that Nick mentioned, which is getting to know what is going on and being able to contribute. And of course, yes, being aware of and intentional about the time that individuals spend in providing feedback. And one thing that would be fairly simple is to then reach back out to the population and share where that project is going or what is happening with the study. I often, we get asked to be on studies and say, yeah, we want to do this. Are you willing to participate? And more often than not, you'll find that people are willing to participate because they do care about the issues. But then you never hear back about what happened in the study or where that project went. Did it lead to anything? And so just keeping people involved and informed I think is really important, especially considering that there isn't much information out there in how quickly and how effectively technologies are advancing for universal design and inclusivity. Well, I want to ask this and thank you for those comments. And this kind of leads into my final question from the audience. What groups are actively supporting disabled people in STEM that organizations can work with? This is Cheryl Berkson, where we have a lot of room in the Duet Center, Disabilities Opportunities, Internet Working and Technology here at University. We have a lot of resources, but also communities of practice for practitioners and various types. And also we have a group of well over 600 students with disabilities that are at high school, college levels and mentors for those students as well. And so feel free to contact me and I have a lot of ideas. Thank you, Cheryl. Anyone else have any comments? I know personally I've done a lot of work with the National Federation of the Blind and I know they do a lot in terms of trying to support different research activities. So I know there are many organizations that are out there, but I think again it comes back to awareness. So I'm afraid that we're at the end of our conversation. I want to thank everyone for sharing ideas. I really appreciate it. Thank you everyone for the fantastic conversation today about how we can improve accessibility and inclusion for computational based research and education in STEM. As we heard at the beginning of the conversation, this is the fourth of five discussions. Our final conversation will be on Wednesday, April 13th from 1.30 to 3 p.m. Eastern time and we'll tie the whole conversation series together to discuss career pipelines and the importance of mentoring for disabled scientists in STEM research and education. We hope that you can join us. Additional information about the conversation will be available on our series website. This webcast from today will be, this webcast will be recorded and posted on the website as will all future conversations. Enjoy the rest of the day and we hope to see you all in April. Thank you very much and thank you to the panelists.