 Yes, thank you. So here we are. My talk today is about user support, which is something that, well, as Seymour had said, you'd think would be a very obvious thing that connects to everything. But in reality, I've seen many times the user support is sort of secondary, or you provide the infrastructure and you just hope that the users will be able to use it. And if not, well, sort of not your problem. So this has been recorded. But if anyone speaks on Zoom, you should not appear in the final recording. So the material you should have the link to, and I will paste it again into the chat. There's also the HackMD, which you can ask questions at any time. As you see, it's down here. So if you write something, it will appear as the question, and I can immediately begin addressing it. So let's begin. Actually, we can start up here. So this talk is about auto-scientific computing user support. And it's not just what we do, but sort of takes a critical view about how the whole thing fits together. Like what's even the role of user support. So down here you can see the three components of auto-scientific computing. Actually, this is an official. It's basically something that I've made up. But we have the infrastructure, which is what you typically think we're about, the computers and so on. There's the teaching and training. And then there's hands-on support. And all of these feed into each other. So any problem in one of them can be partially caused by the other two, and then partially solved in the other two. So as one example, whenever we realize that our user support is having our demands there are increasing, maybe the answer is we need to make our infrastructure easier to use. Or maybe the answer is that we need to adjust our training courses. So let's go on. So about us. Auto-scientific computing. We're traditionally known as Science IT, but we created this new brand name to indicate how we are, we like have this bigger picture than just the School of Science and just IT. And we have close collaborations with many other people at Alto. So our national collaboration used to be called the Finish Grid and Cloud Infrastructure. The current proposal shows that it will be called the Finish Computing Competence Infrastructure, which really shows that user support is going to be more important than ever. And I think here at Alto we're proud of our user support, but it's a very multifaceted approach and really requires thinking about it in the right way. So what's the role of user support in scientific computing? Well, we can start off by showing how it sort of has the bad reputation. So in many cases customers just think it's bad, like the support staff, hey, me, I never get a good answer, I never get anything out. A lot of times support staff really don't like doing it. We have these customers that are asking us questions, they don't know what they're doing, they ask the same thing over and over again, they can't read the manuals and so on. And in fact, the whole mindset is often it's an issue to take it, which shows that it's something that just you want to end it as soon as possible. There's a start and the end. But maybe a conversation model is more appropriate. Like someone has an issue and you give them some hints and then they start working on it and you can give them more hints and so on. Let's see. So what makes this so difficult? Well, technology is hard, as you know. Oftentimes the users send something and ask a question and don't give enough information to even solve it or even know what the issue might be. We often have to pick up slack when something else isn't taught well. Like for example, people who don't know how to install Python packages or know how the shell work that really increases our workload. Sometimes we're disconnected from the user community and we don't understand things from their perspective. And in the worst case, user support is something that we're forced to do in addition to whatever our main job would be. I think there's different types of support here. So one easy way of doing support is to say, okay, the customer asked a question, we say here's a used a manual link, just read this by yourself. We can be a little bit more advanced and tell them what to do in the message themselves, like give them some context about how it applies to their case. Then you can give a live demo where you're basically showing, okay, here's how I'm doing it for me. Now you go try yourself. One could take this one step further and do some sort of pair programming thing where you're working together with them and you're both solving the problem. And finally, it could be a solve the task for them. So you're not even teaching, you're just solving their problem and then they're getting it. So of these, the lower letters are sort of the traditional user support method and the higher levels are more time consuming and approach some sort of mentoring or research software engineering task. Okay, more reasons why support is hard. So I've coined this thing called the crisis of computing. So most user skills are much less than what is needed to actually solve the issues and do what they need to do. User interfaces are bad. There's all kinds of internal state. There's different abstraction layers on the inside, which people don't really understand. There's this common thing called the XY problem. So people ask for something, which is what they think they need to solve their problem. And then we give them what they asked for because, well, why not? But then it turns out that this X is an even a good way of doing what they want. And we spent a huge amount of time trying to make something work. When had they just said, oh, I'm trying to transfer files to Triton, then we can say, oh, here's the simple way to do that from Windows or Mac or Linux or whatever. And this even has some Wikipedia article about it even. And then I flip this around to what I'll call the XY solution, which is when the support person says, oh, well, this doesn't really make sense, but if I give them what they want, then maybe they'll leave me alone. And that's the fastest way to close this issue. And I might be measured by that. Okay. So now we're getting to practical stuff. So when you're talking to people and supporting them, we're really on the front lines of people doing their research. And we have the ability to make someone feel they're a good researcher or let them know that, look, you're just not what's needed here. I gave another talk on this about diversity in computing, which is linked at the end of the talk. Let's see. There's a quote here, Harlan's razor, which is one of these legends of online, never attribute to malice, which is adequately explained by stupidity. So I think in our case, we should never attribute to malice or stupidity, which is adequately explained by just having never been told something obvious. Where do people learn to use a cluster, Linux shell, all these kind of tools? Really, these days people don't. And we need to pick that up. So when you're talking to people, make sure you don't express unhappiness or displeasure or some sort of condescending attitude when someone does something wrong. Because most likely no one has ever told them what the right way is or they don't know the expected limits. If someone ever does something which is actually wrong and actually affects someone else, well, isn't that the system's fault and not the user's fault? Our systems should be robust against someone doing something wrong. Okay, so next up is our user support tools. But in the meantime, we've got some comments here. This is a pretty good philosophical question. Should the support staff be supporting most of the time or be doing other things at the same time? Actually, some of these questions are discussed at the end, so maybe we'll get down to there later. But keep them coming and we will answer. Okay, so now we're at the point of what we actually use for our support. So first we have some general guidelines. So we have a help page here which says how to get started with help. You notice it even starts with how to formulate your question, which I think I will cover more later. You can get back to this at the end. But it sort of goes through a lot of the same things that we're talking about now, the different ways to reach us. Sometimes I'll link this page with someone asking a question and they're like, X doesn't work. And they don't even say how it doesn't work or what they're trying. And I'll link these four key questions about what the solution or what they should give in order for us to even be able to start answering. Then we have our documentation, which is the site itself, skicomp.alto.fi. It's open source built with Spinks, findable in the general web search, which is really important. So at some point people, our documentation was behind a login in a private wiki. If people can't easily find it, then how can we expect people to use it? In fact, now that it's public, sometimes we have problems and we try to search the web to solve it ourselves. And then we end up on our own documentation. And that's how you know that, well, you're in some serious stuff. We have a GitLab issue tracker. So as opposed to an email tracker, we use GitLab because we get a lot of the common things from the tracker. The issues, opening, closing, labels, that kind of stuff. But most importantly, it's available with the internal permission. So basically anyone who can log in can see all of the issues and search them, which is really important because most of these issues are not something we need to do for someone, but something about the technology itself. And the solution will help many more people. So it's not just us that's searching for old things, but we tell people before you make a new issue, please search the other things and see what you can find. And we can go linking to other existing issues that solve the problem instead of typing it all out again. One interesting point that comes up with these kind of trackers, what is an issue closed? So do we close it as soon as possible? Or do we close it when we're sure that the user is happy? So both of these have some problems. So one, if we close it as soon as it's possible, then that might be telling the user, okay, go away, we are done. We don't want you to comment anymore. But if we leave it open until we're sure that they're happy, like it's a conversation, then it may never be closed. We have lots of things open, even though they're not really needing some action from us. Yeah, like I mentioned before, are these really issues or are they more like conversations? We have an email tracker, which is run by IT services here at Alto. Email works well for things like please open my account or please increase my quota. But it's not a very good tool for other kinds of things. So for example, how do I install this content environment? Well, many people have asked this before. And if it's private, then there's no way for other people to search it and it just doesn't work. So by email, we have a quite low threshold to direct to the issue tracker instead. So if someone asks something which should be made public because other people may need to refer to it later, we have no problem saying please direct this to the issue tracker. Sometimes I'll go and I'll open the initial issue for them and say, okay, here I've started answering this. Please comment more there if you need anything else. Yeah, but it's really important to when you're doing this to say it in a very nice way to encourage people to continue the conversation going and not discourage them or make it seem like we just want them to go away as soon as possible. Computer science has its own RT instance, which is really a much nicer than effecting, at least for powered users, it's much nicer. We have three email groups. SkiComp at Alto.fi, which is our general support. Skip at Alto.fi, which is for teaching related things and RSE Group at Alto.fi, which is for research software engineering services. And we can bounce the issues around between these different things. Okay, we have a daily garage. So every day for one hour or less if no one comes, we have office hours via Zoom. So we show up there and people can join and ask us any question. This is really great because people can, then instead of sending by email, we can get the direct feedback with this conversation model. We can share a screen, we can do it together, give demos, all that kind of stuff. And the great thing is if no one comes, then it's the admin chat time and we can use it as a social hour. So since it's both this social hour, but has these people coming in asking questions, it really connects us together pretty well. Chat. So lately we've started with chat. So we previously had Microsoft Teams, which is recommended by our IT services, but really is not that great interface. Now we've started using Zulip Chat as our internal discussion board. We're inviting users there too in some public channels. So people can ask us questions. And one thing that's an open question, is this a good idea or is it going to get out of hand? It really remains to be seen. Our current philosophy is that we need this community. So not just answering issues, but we need the users to talk to each other. We need a low threshold for people to tell us what's going on and what they're doing so we can know what's going on there. And, yeah, so that is just started, but it seems to be working pretty well now. And again, like with email, there's a low threshold to direct to the issue tracker. So I think our instructions even say, feel free to use chat, but it can direct you to the issues. And if you don't know if it's worth an issue or not, then that's a time you can definitely use the issue tracker to ask us. We have office drop-in. So obviously not during the pandemic time. This has been replaced by the daily garage, which is really better anyway, because unlike finding one person, you can find all of us at once. And we can divide ourselves up based on whoever can answer the problem best. So in normal times, our offices are spread around the different departments that we serve, and we accept drop-ins well, whatever basically. It keeps us connected to the community, but of course it can sort of get a bit out of hand when you need to focus on something. But I think we can see what most people think, but I think that the focusing really hasn't been much of an issue lately. Okay, personal networks. Some of us came from the departments which we're serving now, so we have these personal networks to know what's going on. And this is actually a very important part of the user support. People have to know that we're there and we're open to reaching them, and we need to know what people are doing. Teaching. So teaching is somehow also part of the user support. So you can't just answer every questions that people have, and help be able to reach many people at once. So this is, well, there's a lot about this. Maybe we'll talk about it later. I don't think we need to really talk about it more later. Private email. So sometimes we get questioned by private email, which I really discourage. So if something goes to my email box, that means it's an extra thing I have to keep track of, and I'm probably not the right person to answer it, but it might be another one of us that's better at it. And, yeah. So I want to direct this to the issue tracker or to the ski.com.alto.v address. But I can't just say, please don't send it to me, because that will discourage people. So my phrasing is something like, I'm sorry, but if you send this to me personally, I'm almost certain to forget to reply to it, and you won't get a good answer. So then I direct them to the issue tracker or the other thing. Okay. So the final part of this talk, what's the strategic vision of support? So support is not one of the very cool things that happens. So, yeah. So support is answering one-to-one questions. Teaching is one-to-many, and the research software engineering is sort of the I'll do it for you, or let's do it together. And I think the lines between all of these is blurring here at Aldo, and that's really a good thing. So risk in teaching, or more like in the way we provide support. So whenever there's budget cuts or things like that, it's this middle layer of science, the middle support that always gets cuts first. And that means that the researchers are left more and more alone. So they have more and more technology and less and less ability to use it. And our load is continually increasing, but our funding isn't. So this could make us more unhappy as we're dealing with all of these basic issues, which we really hope people know again. Our support level goes down. We just have to close things as soon as possible, and there could easily be a downward spiral here. At Aldo, we're not there, and I don't think we're even close to that, but it's really something we need to think about. In fact, as an anecdote, a few years ago, I started this emphasis on making the cluster more usable to everyone. So there was Jupiter Hub, there was like more organized tutorials and so on. And then I realized, oh, now we've certainly got all these extra users that have slightly less skills than most of our traditional audience. And this was a problem. And this increased our load and caused us to adjust our other things. I need to make sure that doesn't happen too much. Actually, you can put it the other way around. How many times do people design infrastructure so that basic users are not able to use it so that the user support doesn't go up? In some cluster, if you make a cluster and you say, okay, the minimum job size is like one node and all the processors, and you better be able to use all of it, is that basically just saying we don't want new people to use our cluster who aren't able to use everything? Okay, what are the benefits of good support? So these can be used to argue for the funding of our teams. So one is diversity. So I gave a talk on this a few weeks ago and it's available online. I won't really go into details, but the summary is that, as you know, computational science has a crisis of its demographics. And we're sort of at the front line of this battle in making sure that everyone's able to do their research. And if you go into the rest of my talk, you see all the reasons why other people are not so able to handle this problem other than us. There's open science. Without good user skills, people can't make their computational work reproducible and shareable, which is the big thing. And universities invest a lot of resources into this kind of open science support and data management and these kinds of things. But what's the point of this kind of support when the people that are supporting it aren't the ones that are actually dealing with the data and the computing and all that? And I think we can carve a little chunk of that problem space and use it to increase our funding for good science. Okay, so I started some exercises here about problematic situations, but I see that we're already getting some of the things, the same things in HackMD. So we can have a slight pause and think about this and then we will get to the other questions. Yeah, so that basically gets us to the conclusions. So if you go further, you see links to more resources, which you might want to look at. So let's take a look at this HackMD now quickly and see if there's anything that I should answer before I stop the recording. Should staff work and support most of the time or be more divided up? I think personally think that it's better to have most staff work on support some of the time. So that keeps the people that are designing the infrastructure connected to the users. So you make sure that you're designing something which people actually need. And then when users are being supported, they get a good expert there. And all the questions people ask should go right back into our infrastructure design. So if people ask the same question too many times, okay, time to improve the documentation or maybe change the system so the question isn't even there. Should a support person be an expert in the infrastructure or closer to the customer? Why not both? So I think that we should act as the bridge between the infrastructure and the customers. And I think it would be best if someone started from the customer group and then we trained them to be an infrastructure expert so that way they have both aspects there. But of course there's many different models. By the way, if anyone in the live meeting now talks, you should not be in the recording. So you can go ahead and ask questions that way also or let me know if you want to be in the recording. Okay, here we have a difficult situation. Do something without a proper background. Yeah, I've got a lot of these. Sometimes someone will come and say, okay, I'd like to use TensorFlow to do this. And then I go and I start telling them how to install things but they're trying to use it through Jupyter and they don't even have an idea how to install Python packages or use a terminal to run any of the commands that we're recommending them to run and so on. And that's hard. And in this case I even gave them a suggestion like, okay, I suggest you find someone in your group who's doing these things and then work with them to get started. And they said, I don't even have anyone in my group that does this kind of things. And then I really didn't know what to do. But I think we really need to be clear about these things. So if someone doesn't have the background needed to do the work, then we need some way to have this discussion and say, okay, we aren't really able to help you. There are some sort of things that are missing here. We can direct them to courses. Of course, courses take weeks and may not be open right now and may not even go into the practical side of things that they need. The research software engineering services are a good option here to connect with someone who can give them the first model and then they can work together. One of my hopes is that with this mentoring and diversity series of talks, we can turn the SkiCom team into more of a community where we have all of these good users within the research groups, these power users or people that really care about mentoring and how to bring them on board to fill in this kind of gap. Yeah. Okay, and I see some of these same ideas here. Yeah. There's a good example here of directing someone to Kaggle to say, learn the basics. Yeah. If we can do that, that's great. Okay. Balance information past the users. Yeah, this is a good conflict here. So some people want to make instructions as easy as possible to provide the minimum needed to do the problem, but then when the problem isn't exactly that, then you can't really move on. So I think we should, when we're helping people, try to make it clear but sort of motivate people that there is something deeper behind here and that's really cool, so you may not need it right now, but like add in these little bits of information in order to get there later. In fact, what's when I was reading about, let's see, it's the concept of lurking in online forums, which has some other term like passive participant or non-active participant. So the idea is you might have this forum or community and in it, there's only a small minority that are contributing to all of the things. So let's say it's a forum on, let's say forum on baseball to use a non-technical analogy and you have a small number of people that are making most of the material on that forum but a very large number that are sort of watching things and maybe commenting here and there but not really taking an active role. This is actually a good thing. So it means that you're learning the terminology, the norms of this community and this sort of absorbing information which will help you to become an active participant later. So I think we have the same sort of model here. So we can do these advanced things sometimes and show people but make it clear that you don't need to know this right now but let them know, okay, this is what you can know in the future. So basically like instead of turning off this screen share and doing something and saying, here's the solution, we say, okay, here's the screen share, I'm going to use the command line to solve this problem and install these packages. You don't need to do this right now but keep it in mind in the future and then you might be able to start doing it yourself some day if it becomes relevant. Yeah, for our user documentation, there's definitely the manual and the tutorial that need to go together. So with the strategic risk, middle layer of support vanishing, so a few years ago, like five or so when there were the government budget cuts, Alta really had to cut back and it was all of the laboratory managers or group leaders or basically everyone between the researchers and the professors were gone. And these happened to be the people that mentored the researchers. So science IT has been pretty good and not really been cut that much but I think it's a common thing in other organizations. Are there any other comments from the Zoom meeting or should I stop and then go to free form voice discussion? Well, I think that's basically it. So thanks to everyone for joining and we'll stop the recording and go to Zoom now.