 I'm Nancy Frischberg and you know me as an organizer of the Linguistics Career Launch this summer, and I'm going to talk to you about these topics and give you a bunch of resources, and ask for more at the end because I have just blocks of stuff here on my walls and in my head to tell you about. So what am I going to do today? I'm going to do a little introduction about me because I don't think I have done that generally. Some of you have met me at the Mixers or know me from before, and that's great. I want to talk about what is design and who is a designer, and I want to talk about what's user research, and I'm going to leave plenty of time for questions. This is a 45-minute session, so let me zoom through. Now if you've read my LinkedIn, you see, and you've read the about to the bottom, so that means you have to click something to get to it. The last line in my about says, I create works of lasting value, and so I thought I would get a chance this morning to tell you about three, maybe four of those. One of them is I was the first sign language dissertation at UC San Diego, and I said to somebody in the mixer yesterday, I convinced Ed Klima that American Sign Language is a language, and how did I do that? And I also wanted to let you know that you can find me on Quora. I don't know if people are familiar with this. Quora is a question or answer site, Q U or A, and I wrote the one sentence. I responded to the question, what's your PhD thesis in one sentence? So I'll read you the one sentence now. The signs of American Sign Language have changed from more pantomamic gestures to linguistic symbols over roughly 100 years by applying several structural principles that may destroy transparency of meaning in favor of articulatory and perceptual ease. And if you want to read a short version of that, there's chapter three in the signs of language. Another thing that is of lasting value, I wrote a book that changed the paradigm for sign language interpreters by bringing them in line with the paradigm for spoken language interpretation with a few exceptions. So this book treats sign languages like other languages. It offered descriptions of skills and settings that emphasize where deaf adults are interacting. The previous books about sign language interpreting had all talked about deaf people being less educated, deaf people needing rehabilitation, and that this we were doing such good work and being helpers, and I tried to bring it up to a equal status point of view by invoking linguistics and language hood. And anyway, you can look at that book if you want. It's still the basis of the written test for sign language interpreters 30 years on actually more than 30. When I went to Sun Microsystems many years later, I was the manager of the group that produced the first open-source software usability test. Oops, here we go. And so there were a bunch of us who were co-listed as authors on this. Susanna Smith, the first author, ran most of the sessions and the others of us helped her by structuring the experiment that we were doing and helping write the report and so on. And I included in here part of the executive summary that tells about the report. So we gave 32 design recommendations. And who is our audience for this? This was the open-source software community because this is software that was not being developed only in one organization. This is software intended for the world, being developed by the world. GNOME is a desktop or a, yeah, is a desktop that's put on top of Linux. Sorry, we did that one. Let's go forward. So I think this illustrates that user experience is a team sport. And I thought I'd share the most recent publication. I got published in the Design Management Institutes Review, which is their magazine. That is the, it's a soft cover and it's not a heavily academic magazine. And we talked to my co-author Gregorio Convertino and I had an idea that we were going to produce a book about collaboration. And I gave a case study here and he helped me contextualize it for the design community. And we talked about the UX function being a new function in a particular organization and what were some of the cases where we either goofed or made, or avoided goofing because of the collaboration that we had among legal, the risk office. This is a financial services organization. So we had to have risk in there as an issue because we were regulated by the SEC, legal risk and compliance. And so I got a lot of good advice from them, which helped me from stepping in bad things too many times. Unfortunately, the collaboration book may be on hold permanently or it may take a completely different shape because we lost Gregorio in the winter. And he was very young, from my point of view, he was I think just 50, but that made me very sad and stopped me from thinking about that. And I threw myself into our events, the Linguistics Research launch for the summer and we'll review whether that book is going to happen sometime in the fall or winter. Let's go on. So you can think about if you have questions for me about any of those things, but I don't think I want to take the time now because you came here for what the heck is design and who's a designer. And I'm going to, this is certainly not comprehensive. This is sort of like three to five, seven maybe slides that are going to tell you some of my interesting ideas. So when did I first become aware of the ideas behind user research and use and design? And I'll say in 1971, there was a summer school at UC Santa Cruz for the first time. I think it was on the competing year or the alternate year with the summer Institute. Anyway, it was a great time. We had fun and UC Santa Cruz was brand new and the parking was not very well laid out yet. I mean, it's like they had dumped three or five buildings into the redwoods and kind of made some parking, but they hadn't like regularized it yet and they certainly hadn't finished building out the campus. So we went into a complaint about our parking ticket and the guy there was quite understanding and he says, well, you know, we haven't quite finished, you know, laying out the parking and all. And you know the three E's of parking, don't you? And we said no. And he said, well, let's see, the first E is engineering. And by that, when he explained it, what I understood he meant and my today terms for that are that engineering for him meant the design and implementation. What is the space? Is it flat? How are the lines? How wide are the lines? Where's the accessibility places? Of course, in 1971, that wasn't regulated. So design and implementation of the engineering education was his second E. And that by that he meant signage and pointing people at the at the right places to park. Where is the parking lot? Where are the parking spaces? And the third thing he talked about was enforcement. And he said, so we can't do ticketing and towing until we've fixed the signage and we've finished the engineering. And so he forgave our ticket. And I thought, wow, okay, design's on my side. Now, one of the things you may find out is that there are some heuristics about design, especially design for software. And Jacob Nielsen, and of course he's Danish. So if we were in Denmark, we wouldn't pronounce it that way. But he's been living in the States since around 1989 or 90. Jacob talks about these 10 different heuristics. In 2020, he gave it a kind of rework, but it's all the same 10 things. And you can go and look it up. I gave you an article reference at the bottom of that. And I'm sorry, Marcus, I didn't connect you to my documents so that you can't repost those easily in the chat. But I'll release the slides and we can make this all available. I'm just going to call your attention to a couple of things here. For example, error prevention is a heuristic. Are you letting people do have undue? Does your system have undue? And then the other related thing to that is how do you let people know they've had an error? Look at the bottom, second of the bottom one. Help users recognize, diagnose, and recover from errors. And I'm making that a highlight here because our friend Kate DeHair is going to be with us next week, next week, two weeks from now. And she's going to talk about how to write the error messages so that people can use them. And it's not like we were talking yesterday at the mixer. You got a 775 on your television? What did you do? So I did have a 775 this week on my television. Another great book by Don Norman. Some of you may know his name. He was also at UCSD when I was there. And he was head of the psychology department. And his famous book at that time was memory and attention. And the reason he was interested in those two topics are those are things that humans are not so great at. Sustaining your attention. You're constantly scanning and being able to be distracted. So keeping your attention on the thing. And second of all, remembering all those things that are happening all around you. He changed the title after feedback on the second edition. And he turned it into design of everyday things. Because when people heard the title psychology of everyday things, they thought it meant something about clinical psychology. And that was not what he meant. He talks in the book a lot about the physical environment, but the implications for software are clear. And he rejects the idea of human error. He says human error is the result of bad design. So it's not your fault. However, you should listen to yourself and your colleagues because we still have internalized the business about, oh, I made a mistake on the computer. Probably it was an issue of your attention being distracted or your memory being faulty. So I just got a little distraction there because my phone lit up. Turned my phone around. Okay. So Don Norman is another great resource. And it will teach you a lot about the abstraction of design with lots of practical and specific examples. So I like it for that. He talks about, he introduces some terms like affordances and constraints. And anyway, it's not a long read, but a useful read. And my next example, which I love a lot, is Jeff Johnson's designing with the mind in mind. And I rather than give you a picture of the cover of the book, I thought the topics in the table of contents would be more interesting to you. And that could have been laid out a little better. So it's very much about perception. Whoops. So let's go back. It's very much about perception, visual perception and a whole bunch of things that linguists probably haven't been exposed to, but are part of just the next extension beyond what we have been exposed to in cognition. And Don goes on, and of course, number seven, he talks about memory and attention. But that's not Jeff goes on and summarizes a lot of what Don has learned there. And Jeff also talks about how reading is unnatural. And I think for us, that's a really unusual and exciting idea. Why is reading unnatural? Of course, it's a reading of a technology that didn't come built in, right? He talks about recognition being easy and recall being hard. So think about how that helps you to be a better designer. Anyway, lots of great topics in here, Jeff writes very readably. You may want to look at his other books as well. He wrote a book called Gui Bloopers. And the second edition is probably still very accurate, except the examples are a bit dated. He wrote a book also with Kate Finn called, I'm going to forget the name, but it's about access for elders and guidelines for designing for older adults. Since hello, we're the pig in the Python, this baby boomer generation is all turning old. See, I'm not disabled, but I have trifocals, right? Okay, so now I want to ask and let's put it in the chat. What's the difference between art and design? Everybody's going for design has function purpose. Okay, that's those are great. And here's the definition that I got from a dean at the School of Art in Philly many, many years ago when she was giving us a tour. And what did she say? Oh, I have to focus over here. She said art is about self expression. Design is about communication. And so these are really distinct for me. I am a big fan of art, and I'm a big fan of design. And sometimes they come together. But I think in ordinary definitions, we often think about design as number two, we'll come back. Number two, the arrangement of lines or shapes to create a form, a pattern or decoration. But in fact, I'm thinking that the definition that's more useful to us is the number one. And I just took the first definition that came up when I googled. So however, or and sorry, this is very trigger happy. Another definition of design is the one that Jared Spool gives us. And that is design is about rendering intent. So for example, where do you keep your hot pad holders in your kitchen? I suspect that they're in a drawer near the stove or oven and they are or they're hanging someplace nearby there where you can get at them easily. You probably didn't put them across the room in with the tablecloths, because you use them so frequently. So that's you acting as a designer to design the functionality of your kitchen. You want the hot pad holders near the thing that gets hot and easily available. Same thing for all the other stuff that gets designed. And the this article about redesign of the design process, probably as of interest, and I hope will spark your your excitement for learning more about design. So I want to also say that design in in our world is not only about the aesthetics. We already covered that, but not only about visual stuff. And that's what I wanted to mention here. So I left you the coffee from the barista who is obviously a designer or maybe artists that might be art. But we have lots of kinds of designers. So graphic designers have been around for a long time, and they used to be strictly limited to flat stuff, right posters, brochures, the design of the logo for your envelopes and your business cards and how your business cards look and all that stuff. Graphic designers, many of them have moved on and become visual designers, which means that they're not necessarily designing for print. There's a whole bunch of stuff that has to come along with designing for print that many graphic designers end up learning, like how the printers were going for a print check at the site of a giant machine where the paper rolls are bigger than the room I'm standing in. And then you want to see the first copies coming off and make sure that color and the lines are all perfect. We don't do that so much. And I mean, people do that. But graphic designers are very often now your visual designers, and so they are doing things with the screen and making icons and images look right. UX designer is a newer term, and it kind of spans a couple of these. So if somebody invites you to apply for a job called UX design or UX designer, make sure that it's very clear what they're asking for. If they're asking for somebody to push pixels around, that may or may not be you. It certainly is not me. I'm not good at that, but I appreciate the skills that come with that. It may be that they're looking for somebody who's an interact, go back, an interaction designer. So rather than a visual designer, it's somebody who's planning the flow of the work. How do you get from point A to point B? How do you get from looking for a new bathing suit to putting it in your cart and paying for it in a clean way and making sure that it's actually the color and size that you want and that it's going to arrive in time before you have to go on vacation or should you have it sent to the vacation place? So the interaction designer is probably the person who's looking at that kind of workflow. Sometimes the UX designer is the same as interaction designer. There are instructional designers as well, and those are really different. Those are people who plan what a curriculum should look like or a training program should look like and when there should be opportunities for interaction with the audience and when it's just the instructor and when are one of these exercises going to be take home things or not. So there's a lot of roles for instructional designers not only in educational settings because lots of big companies have big training departments and need people to provide uniform instruction. Okay. There's also sounds. Yes. I have a quick question from the chat. Someone's asking if you can clarify what you meant about pushing pixels around. Okay. Sure. It looks here like there's a lot of words on the screen and then there's a picture, right? Actually, those are all represented by tiny little squares on your screen and when you say what the resolution of your screen is, you know, 680 by 420 or that was an old one. Right? Now we're up to 1040 by something else. I can't remember the exact numbers. Those numbers refer to the number of little tiny squares in one dimension and the other dimension. And so if you know Photoshop, you will be very familiar with pixels. If you know Illustrator, you will be very familiar with pixels. And what good graphic designers and visual designers know how to do is clean up the little flaws that might be artifacts of the photography rather than artifacts of the world that I was photographing. So all that airbrushing is playing with pixels and making models appear slimmer than they actually are or, you know, that their face is slightly more cheeky and less jolly. You can do that in Photoshop. And so that's what I mean by pushing pixels around. Does that help a little bit? Maybe you didn't like the color of my skin this morning. I was looking too sickly. You're going to make me more, more healthy looking in whatever shade that would come in. Does that help? I'm going to go on until I hear no. Okay. That was a good spot. And we can say, are there any other questions about design? I did not cover everything. I just gave you a few resources. We have another, oh, sorry, we have another question. Asking is instructional designer similar to a loss of space, similar to curriculum designer. Does it involve creating a curriculum beyond academia? It can be. It depends on what you guys mean by curriculum. So if I'm writing, for example, we heard, I heard somebody talk about this the other day, and it was like a throwaway onboarding. That's the process of getting new employees oriented to their workplace. And it's a very complicated process in different organizations, not small organizations, but large organizations that have a whole bunch of processes and procedures that they go through. You need to report your expenses in this way. You need to log your hours in this way. Oh, if you're in this job category, you have to log your hours this way. You need to ask for vacation time through this process. You need to let your, you know, doctor, your benefits programs know how you're doing in this process. So there are a bunch of different ways to orient you to all the different processes of the new workplace and the new organizational stuff. There are competing companies that offer software for all those different things, and they can all be tailored to a particular workplace. So if I'm at Acme, I like Acme because, you know, Wiley Coyote works there, and he's a good friend of mine. And Acme Corporation, I guess he doesn't work there. He's the thief. He's the stealing the things from, anyway, Roadrunner works there. He must be my friend, or she must be my friend. So I go to work for Acme and somebody, God willing, somebody has put in place a decent set of modules for me to learn all those distinctive special case things and that I would be able to go back and find the resources later, because I'm learning so much in that first week. I can't even tell how to open my email. So yes, there could be instructional designers will be involved in helping to prepare you, you know, to give you the materials that you'll be able to be prepared with, and then also to evaluate are people actually able to learn this in the amount of time that we're giving them? What are we missing? Where do they get stuck? Because if you have to keep going back to the benefits person and saying, tell me again, how do I file this doctor report, you know, this medical bill that I want, that's taking a lot of personal time. And what you want to be able to do is make sure people have the resources to be independent and so that there it's new questions that you're asking. Let's make new mistakes. That's my favorite expression. So we want to be able to the routine stuff should be easy to find, easy to discover, easy to learn for humans. And so that's the kind of thing. Yes, it's like curriculum design, but it might be much narrower, smaller modules than you're used to from a university curriculum standpoint. Other questions. So returning back to the pushing pixels question. Okay. A follow up. So UX designers are really involved in the weeds of how an app or website is laid out. Absolutely. And layout is a skill that you can easily learn. We call it wireframing. So there are important things about and even forms are designable. And there is a brand I think I put Carolyn Jarrett's book into the slack as because it was just released this week from our sponsor Rosenfeld media. I am not a Rosenfeld author. I am a contributor to a couple of people who've written Rosenfeld books. And I'm a great admirer of many of them. There's a friend of mine named Steve Portagall, P-O-R-T-I-G-A-L, who's a Rosenfeld author. And he wrote a book called Door Nobs. I don't, Doorbills. I can't remember what it was called. It's got three words and danger is the third one. And he's talking about war stories from user experience research. And how, I have a story in there. And it's, I can, we can tell it another time. But anyway, it's Steve collected, I don't know, 40 or 60 people's different war stories about surprising things that happened when you went out on field work or when somebody came into the laboratory. And what do you do now? So I think that's another opportunity for you to look at that. But back to pixels, Carolyn Jarrett has a book on form design, which talks about things like when you have a slot for a name, where do you put the label for name, name, name? Is it above the the slot where you're going to fill it in? Is it in the slot where you're going to fill it in? Is it below it? What's the spacing on that? How does the spacing on that relate to the next slot that you're going to put in, which is address or course you're enrolled in or whatever the next thing is? And how do you represent dates and how do you account, how do you let users put in their phone numbers in multiple different formats and let the system regularize it for the machine? So one of the great principles here that I didn't mention in writing, I'll say in speech, is often automation is the best user interface. If you get it right. So think about direct automated dialing on maybe people don't even remember that. Oh, well, we have it on the phone. When I pick up my phone and I see your name, I can click on that. And it'll, I don't have to remember your phone number anymore. I have no idea what half the phone numbers are that I call regularly. And I bet you're that way too. But when I was younger and we were using rotary dial phones, or even push button phones, the I had in my mind, I probably knew 75 or 100 people's phone numbers right off the bat. I have that I don't have to remember anymore. I can use those, those pixels, no, those little memory slots for other stuff. That's why Carolyn Jarrett's book is useful. And then you can go through, I'll try to mention other books from the Rosenfeld list that may also be useful in connection with this stuff. Ready for the next thing? Okay, so we've got got another 15 minutes, we can do it. When people say to me, I want a focus group. I don't say, Oh, of course, what I may end up saying, of course, but what I understand them to mean is, I need some qualitative responses to my product or service. Now, when I run focus groups, and I have defended focus groups among user researchers who think that they're just terrible, because scripted, the usual way is somebody writes a script, somebody moderates, maybe it's the same somebody. I write a script of the questions that all my stakeholders have about this topic. How do you like your new Alexa device, whatever it is, or your Echo, is Echo, Echo is a Amazon product, then the Google Home is a non-Am is a Google product, right? So let's do let's say we're, we're talking about that. How do you like your voice activated devices? And so I will have recruited people into my study, who are this focus group, who I've already screened to know that they have such devices in their home. I may do another group, which says people who don't have them. And have you considered it? How do you want to have one? What's holding you back? Is it price? Is it the invasive privacy stuff? Is it something else? Or maybe you feel like the quality of the recognition isn't good enough because you have an accent that Alexa hasn't learned yet. Those are the kinds of questions you might ask. And in fact, in focus groups, they often ask them directly. I don't do it that way. What I do is I try to get everybody to do a group activity that relates to the question so that they're kind of warmed up, primed on our topic. And they can learn to talk to each other and argue with each other. And they're not talking to me. At the end of the group activity, and I plant observers in each one of the small groups, if it's larger than a group of six or eight. So I can have 20 people in a room as long as I've got enough observers. And so little groups of five, great. And then at the end, we come together and say, okay, so how did you approach this problem? Or what did you think the right answer was going to be here for your needs? And was there any conflict in your group? And talk about that. And then I can ask those other questions that the stakeholders thought were important if we hadn't already addressed them in the small groups spontaneously, organically, to be able to come up with it. So I want to focus group means I need qualitative research. I want a survey. And here's an example of a fabulous survey, right? How satisfied are you? It's a little glary because it's really me there, my shadow on there taking the picture. And what what somebody says, when somebody asks me for that, I say, oh, you want some quantitative answers about your product or service. And let's decide if we know enough to be able to create such a survey. And then there's lots of books. And I think Rosenfeld has one, too. And I'm sure O'Reilly has one about making good surveys. I just read somewhere, somebody's got a new book coming out about surveys. We'll have to look at it to see if it was a Rosenfeld title. But there's a lot of issues. And I'm sure you're aware of them about the wording of surveys and that's a number of the types of answers you get. And I will come to a I'll come to a framework for considering several different kinds of methodologies. The important thing to think about, though, is you are not your user. This is a big motto of the usability professionals association. They change their name user experience professional association UXPA. So what product or service are you working on? If I'm working on an enterprise product, that means a big company's product like Salesforce has. And I don't think everybody here probably knows what Salesforce does. But we'll have several Salesforce speakers and presenters coming up. Salesforce makes software as a service. S A A S. You probably touch Salesforce multiple times in a day and have no idea about it because it's running the back end of all kinds of other things that you're touching. And I'm not even going to get into that kind of level of stuff. But you are not a typical user of the administrative part of Salesforce. You may become one once you get into industry. And your experience and your knowledge of the product, if you're part of the team that's creating a new feature or upgrading to some new operating system, you know way too much. And so your intuitions about what's going to be easy or hard, confusing or clear are going to be off. You need to listen to your users, which does not mean your users know what they want. And it does not mean that they know what the right solution is. You listen to them because they're telling you where things don't work for them or where they get what they want. And so and here's the slide that I missed putting in three things. Usability, usefulness and delight. You're aiming for all three. Usability means can somebody make sense of what's on their screen or what's on this paper document you gave them or any other device where you might interact with some machine. Usefulness means is it going to lead to them accomplishing the task they thought they were going to end up accomplishing? Is it something that they will use or is it frivolous and just frosting for them and they don't like sweets that much anyway. And delight. Have you removed all the frustrating and friction developing parts of this? Neutral is better than bad, you know, bad attitude. But best is you finish, you get there, you do the thing and whoa. I'm done. It was fast. It was easy. And I'm pretty sure I got the result I wanted. Quick example, I have the my secondary email address is pretty easy to remember. And that means I was a pretty early adopter of Gmail. However, lots of other people think it's their email address. So I know about people's orthodontia appointments and Buenos Aires and their refrigerator repair in Cleveland. And when that refrigerator is or the other appliance is going to be delivered in Florida. And I mean, I just got it myself, I think off a list of people who were in some learning community that I never signed up for. I end up with a lot of excess labor on getting myself off those lists. And sometimes I feel like it's important on doing a service and, you know, somebody really needs to get this information and it's not me. However, the ways in which I write my little note, you know, and say it's not me, you need to check back with your person and get a better email address for them. And people say, oh, no, everybody in our learning community is now involved in it. No, no, no, no, no, I am not your user. That's a different sense of I am not your user. But one time I got a receipt from some store. And at the bottom of the receipt, it said, oh, not you. Click here. That's me. I said, not me. And immediately got sent to a page saying, we're very sorry. Never mind, your email has been taken out of our system. Wow. And I never got another thing from them. Yes. They figured out I wasn't their user. Yeah. Okay. So that's a different sense of I'm not your user. But it's from the user point of view. But from the point of view of a user researcher, you aren't that person who's suffering the problem, probably. And you aren't suffering it. You don't, even if you do have that issue, and you overlook it, or you get annoyed or whatever, how do other people react to it? So the emotional content there is something to pay attention to. And I put this picture in, because I don't think any of us uses a giant magnifier the way this particular individual did. Who I, I was intentionally going for people who had various kinds of difficulties and how well they could use a new system we were proposing. Okay. Two parts of user research, and we're going to be done very soon. One is ask the right question and the other is ask it of the right people. And that's where your expertise comes in is defining the problem in a way that you can get an answer to it. So people will, supervisors, your stakeholders, the heads of various departments will say, I need to know X. And you then need to push back and say, well, X is hard to, hard to ask. Let's figure out what is it you really want to find out? Why do you want to know X? Is it because of this, this or this? Because I think we can frame the question more narrowly, more specifically, and therefore get more informative answers. And then the right people is people who have used our product, people who've never heard of our product, whichever it is, I mean, you know, you decide what the criteria are. And sometimes it's just the person on the street. Can anybody figure this out? Good. Then we can make it more specific to our particular users. This is the slide that I really wanted to spend some time on. And I know we don't have a lot of time, but let me see if I can move this thing out of the way. I'm going to encourage you to go look for this. And did I remove the link? Okay. I'll make sure you get a link to this. Because I think I covered it up. A guy named Christian Rohr developed this, this two by two grid effectively. And what he says is the y-axis is attitudinal information that you're getting from the users versus behavioral information. And the x-axis is qualitative versus quantitative on the right. And he calls it direct versus indirect. I'll take one or two examples to show you. But then the little symbols that are attached to each of the methodologies tells you, is it natural use of the product in real life? Is it a scripted use in the lab? Is it decontextualized so that I'm just asking about the functionality without showing you any of the specific product? Or is it some combination of those? Let's take, so here focus groups at the bottom left. Can you see it? Focus groups in the traditional way get you information about people's attitudes. And so the fact that they're in a group talking to one another and to you means people will more likely give you answers to your questions that are what they think you want to hear or what they, what the loudest person in the room says, rather than what their authentic opinion is. And they may not know that they're, they're answering differently from how they would behave until you put them into a different setting with a different kind of methodology so that you get to see what's actually happening. Take A.B. testing, which is in the opposite corner. A.B. testing is a methodology that has, that you are not aware of as the end user, because it says we, as the software development crowd, can't decide whether that hero picture, that first giant picture on the screen when you land at our site, should be all popcorn or it should be people eating popcorn because we make the best popcorn. And so what we're going to do is we have been using that giant popcorn picture for a while. Let's still show that to a lot of people, but let's take some small proportion, maybe 5%, 10% of our audience and show them the other picture and see what the proportion of clicking into our site is just based on the change of what the photograph is. And people actually do these studies often. Okay. The problem with A.B. testing is you have to be a big enough organization that's getting enough traffic to give you tiny little statistically significant differences so that you know whether you should take the new direction or the old direction for that hero picture. And maybe you just want to refresh is the new picture at least equivalent to the old picture. So A.B. testing is very much beloved by the data scientists and the and the you know people who care about that users are coming in and staying longer because the value is eyes on the screen. Are we keeping you here? Are we providing you with something that's interesting enough to keep you going and completing the shopping process, for example? My concern about A.B. testing is too often we haven't done enough of the early studies to say, well, it's not just the size of the picture or the content of the picture, there's a few other things like where are the words wrapping around and so on. And as long as you keep those the same, you might be able to tell some differences. And the other problem with A.B. testing is startups. Startups can't use it because they don't get enough traffic to be able to know what those differences are. Anyway, there are things in here that you already know how to do. You know how to do ethnographic field studies, I claim, because you've probably taken a field studies class. You certainly, I expect many of you know how to do a semi-scripted or a semi-structured interview. Those are exactly what has to happen in a usability lab study or in other kinds of field observations. And we could talk about some of these other methodologies at another time, but I know our time is short. So let's keep going. I said that one already. Here we go. What's in a typical day? And I actually wrote somebody this recently. I say it's probably some degree of project and program management, reporting to your stakeholders, making sure your team knows what's going on. You know, all those kind of regular stuff. Then there's some researchy stuff like maybe you're writing a script, maybe you're doing an analysis, maybe you've got a couple of sessions to run today. That was one of the issues that happened for one of our panelists as she ended up with user sessions scheduled at the same time as this session. And the third is operations, which is stuff like how is my recruiting going? Do I have a big pool of possible people participate? Can I just go to employees who aren't down the hall for me for this particular study? Or do I have to go out? Is it a specialty recruit? And in some large organizations now, operations are their own function so that there's designers, researchers, and operations people. So you really get great support. Marcus, anything more that I missed? I think that there was a general question asking about the link between linguists in UX design. Like what sort of role linguists can serve in UX design? I think that was a question that was asked earlier in the chat. But I know that we are pressed for time. Well, the answer is all of those roles. And I don't think there's anything that's going to stop you. And I will say that Abby and Peg and maybe Sarah also would be delighted to talk to you. And I will make sure to put there information. You can do informational interviews with them and ask those questions again. But I think linguists can be designers as well as researchers. Okay. Thanks, everybody. Have a great day. I'll see you later.