 Welcome everyone this afternoon, good to see a number of people in the room and coming into the room to to share in this webinar that we are doing about writing powerful articles for publication. My name is John Gunther. I'm the editor for the evaluation journal of Australasia and joining me this afternoon from the team is Jeff Adams is the manager for the for the journal. The way that we'll structure this afternoon is to introduce the team and then run through the the set of slides that we've got there to talk about various aspects of publication and at the end we'll stop the recording and there'll be an opportunity for you to ask questions if you want to. So I shall begin. First of all, I'd like to acknowledge the country that I'm on today and the elders past present that emerging. I'm on our under country from Bandwa Alice brings in the Northern Territory this afternoon. It's great to be in this beautiful part of the world and great to be in a place of such rich cultural heritage and and beauty as well. And if you'd like to let us know what country you are on this afternoon then feel free to just describe describe that or write that into the into the chat window as we go along. I'd like to just introduce you to the evaluation journal of Australasia's editorial team. So as I've said Jeff and I are both editorial team members and a few of the team are not able to make it this afternoon but joining us on the team. We have a quadrilla as you say a sandy is from Western Australia, and the other who is from Melbourne, Minghua team who's from Singapore, and even on the Rinsky who is from New South Wales. Together we we we manage the publication process for contributors and take your papers into the portal and or you put them into the portal and we manage them until ready for publication. So, it's a good team with a wide variety of different experiences. And what we try to do is assign papers to people that have got particular experiences where that's appropriate. Just also to acknowledge our editorial advisory board. I'm not going to read out all of the names that are on the on the list that you can see in front of you but our advisory board are responsible for doing reviews where we can assign them an appropriate paper. They're also responsible for providing advice and a little direction for us as we as we progress forward with the journal. We meet with the advisory board twice per year and we provide a report back to them and they provide input and feedback about how the the journal is going. As you can see by the names there these are well respected people that have a lot of experience in evaluation and also experience in publications as well. So we value their contribution to the journal and they make an important they provide important input into the journal and its direction for us. So the aim of this session is pretty simple. It's about how we or how you can prepare a good article for the EJA, the evaluation journal of Australasia. And when I think of good in this context, I mean an article that is well written, widely read and cited often. We are looking for a range of papers that will encourage our readers to to think to do their practice better and to engage in scholarly thinking around evaluation and evaluation theory evaluation practice. So think of this as an opportunity to think about well what could you do to make to write a good article for the journal. Some of us may not be used to publishing in journals and you might be asking yourself why would I even want to publish you know you write evaluation reports. We do that a lot. We write for all sorts of reasons but why on earth would we want to actually have something that's published in a journal. Well I think there's several reasons why we might want to do that. First, from a more general perspective, it's pretty important to publish because we want to build an evidence base about good practice, about the findings from our evaluations, about evaluation theory and about other aspects that relate to the evaluation profession and the academic work that goes on around publication. It's very important from from that perspective just building the evidence base but also I find that from a dissemination strategy perspective sharing the learnings beyond the evaluation and and beyond the commissioner is really important so that other people can use the findings that we've got. Use the thinking that's gone into evaluation to improve their own practice. So I think that second reason is particularly important as well. And thirdly, I think publishing increases the value of our evaluation work. I find whenever I'm putting in an evaluation plan or a proposal. I always include the option at the end of the of the project to do some publication work and I do this optionally because I know that some people don't want to have their work published. It needs to be private and needs to be kept private, but more often than not, I find that there is something worth sharing from the evaluation work that we've done. And I offer that to a client often as a free option, free in the sense of it's not going to add additional cost. But it does add additional value to the work that you have done and to the work that your team might have done in the evaluation because it means that it's more accessible to the broader public. Yeah, there are of course lots of different journals in the market. And the evaluation journal of Australasia is just one of many. So why would you want to publish in this particular journal? Well, for those of you who are members of the AES, which I suspect is most of you, this is your journal. There is no other journal that is like it in Australia. There are other international journals that you can publish into but this is Australia's journal. It does have their name Australasia in it. So it implies a broader audience beyond Australia, but it is a journal for our membership. And so it provides an avenue for you and me to share our findings with other members and to, I guess, make our work known to a much broader audience as well. So I think for those reasons, it's important to publish and it's important to publish specifically in the EJA as well. So anyone can, pretty much anyone can publish, whether you're a research student doing a PhD or a Masters or whether you are a policy professional or an evaluation practitioner. You might be an academic. It doesn't really matter. Publication in our journal is open to anyone. And we do have a variety of different articles that can be accepted. And I think Jeff's going to run through the different kinds of articles that we would normally accept. Thanks, Jeff. Yes. So the journal really can accept a range of articles. And essentially, there's two main types that we get from people. One is academic articles that really going to be focusing on developing new theory. And they may also include reviews. So that could be a systematic review of a hot topic in relation to evaluation. I think the important thing for all the submissions and types of articles is that the word evaluation is in the title of the journal. So it really needs to focus on evaluation. So we get some quite nice articles about topics. So for example, a health promotion topic, but there's nothing evaluation in it. So we need to kind of make sure that there's a focus on evaluation. Those academic articles might, for example, include a mix of qualitative quantitative and mixed methods kinds of evidence. And so they'll be based on what's been found in that evaluation. The other kind of main type of article is a practice article, which is focused more on, you know, what what is being learned in the in the practice of doing an evaluation. So it could be, for example, reporting on some innovative evaluation practice or some techniques that were put in place will perhaps some challenges that you faced and overcome within your evaluation. We're kind of pretty open to any topics. And you can see on the list on the right there is a list of kind of hot topics. So some of the key things that that we think are important in relation to evaluation. But you don't really need to be restricted by those. If you've got something that's interesting that you think other evaluators want to hear about. Then we're, you know, really open to receiving articles about that. So I think, you know, John will probably talk about this in a minute, but one of the things is to think about who are the readers of the journal. And essentially, they're people like you. So there's a bit of a range. I know I can see some in the chat. I can see that there's some academics. I can see that there's some practitioners. I can probably work out there's a couple of students in there. And I can also see that there's people internationally in the in the session today. So there's a pretty broad audience, but we of course, we're all focused around evaluation. Thanks, John. So the different types of articles what what we've provided here is a bit of a structure to help guide what a good article might look like. I think what we would suggest that especially when you're starting out is to try is to kind of follow that the tried and true method of structuring journal articles. And if you if you, you know, look at this list, I won't kind of go through through everything on this list, but often what we're going to find is that an article that kind of covers these types of areas within the within the, you know, what what's being presented is more likely to be kind of sort of acceptable to readers. I wouldn't say that you can't vary it, but I would always start off with this kind of basis in mind. And if you have a good reason to change things, then that's fine, but you don't really need to be that creative in the way that you present an article. So, for example, if you're doing an article around basically using qualitative findings. What you need to do to start with is to talk about, you know, introductions are kind of a bit of scene setting doesn't need to be extensive but kind of, you know, where does this fit into the into the grand scheme of things. And you might sort of expand on that by looking at some literature and some evaluation theory. Then you're going to talk about the methodology and approaches that you actually use to to collect your data. And then you might have a finding section and then a discussion and conclusion section so you can see it's kind of pretty a pretty standard kind of format. So again just using these kinds of formats will help you structure your your article. I think to me the important thing about an article is that is to think quite clearly about what you're trying to communicate and think about that before you start to write the article. So one of the things that the journal asks is that you, as well as providing an article you provide a few dot points for us at the front of the article just to say things like, what do we already know about a topic. And what's new that you are offering in your article. And I think that's a kind of really good thing to think about is what are you trying to communicate. Often when we do an evaluation, we've we've found out a whole lot of things. I'm, for example, I'm just writing an evaluation report now and it's one of those horrendous one that's 250 pages long. Now, we can't put that into a journal article so if I was going to use that I'm going to have to think about what is the key thing I'm wanting to to get across to the audience, and I'll focus my article around that. And I could use one of these kinds of frameworks to help me set that up. Thanks, John. Sorry. That's over to me now is it that's right. So before we get into what makes a good article. I just want to focus a little bit on what makes a very ordinary article. An article is not the same as a report. And what makes a good report is not the same as what makes a good article. Jeff's mentioned the size is one issue. But an ordinary article just reads like a report. It's states facts. It's got lots of dot points that might have diagrams that explain things. But that's not necessarily what you need to have in an article. So an article that's got lots of dot points and lots of diagrams is can be very distracting for an article reader. You're better off having less rather than more of those kind of things. As Jeff mentioned there that that that question about what don't what don't we or what didn't we already know is important. The so what question that comes out of an article. So when I'm looking at a at a manuscript that comes in and it tells me about findings from an evaluation. For example, my question at the end of that is always going to be well so what and that's so what should encapsulate something that is new or different that this study has found. A third thing that makes an ordinary article is if it's expresses a lot of opinion and it's just a sales pitch. That says how wonderful we are as a we might be trying to sort of describe a process that we've developed a framework that we've created. And it's then if it's full of opinion and it's full of sales pitch messages then it doesn't read like an article an article should have something that's more critical and more nuanced in it. That doesn't just present one point of view it presents some balance. So, so avoid lots of opinion. An article that does not describe limitations does not describe ethical issues and doesn't describe any theory is also fairly weak as far as as I'm concerned so every study that we do will have limitations associated with it. Every study that we'll do will have ethical issues that need to be spelled out I think for for the readers and and every bit of practice that we do is based on some kind of theory and and that needs to be put into an article. We all know what those things mean, but sometimes we forget about the importance of those things when we're trying to put this good article together. Another another thing that works me I suppose is the regurgitated article the one that's got lots of old literature that might be fit for the 20th century but not for the 21st century. So, when you're presenting something and trying to argue the case that what you found is new, or what's different, then you need to be sure that your literature is up to date and fit for the contemporary time so as a general literature that's more than 10 years old, I would urge you not to include unless it's something that's of seminal importance. So, quoting a theorist from the 1980s might be appropriate if that's the foundation of your work, but quoting findings from research in the 1990s would not be appropriate because things change very quickly. Acronyms and abbreviations are another pet hate of mine. For those of us that are working in a particularly industry setting, whether it be in the social services or education or health setting, we get used to writing with acronyms because we know that everyone in our profession knows what those acronyms mean. But our readership is not necessarily from your industry experience. And so it's really important I think for us to be mindful of our audience as Jeff mentioned before, and not include confusing acronyms that we don't all understand. The other thing to bear in mind with that is that if your article goes out for review and your reviewer has to continually interpret what an acronym means and go back to its first used every time, they're going to get a bit frustrated with that and will be likely to pull you up. And you're going to have to change that at some stage in the future in any case. Avoid acronyms and keep everyone happy. It is one good way of dealing with an article instead of making it sound like a report. Think about your reader. What makes this article interesting to them? What would challenge them to think? Is there anything contentious or controversial in your article that would challenge conventional wisdom? Is there something about positionality or bias that you need to sort of also admit? I'm a non-indigenous white old man and so I'm going to be biased by my positionality as just that. So when I'm writing, as I often do, about Indigenous issues in Australia, my bias and my positionality needs to be brought to the front there so that it's recognized by, for example, an Indigenous reader who sees that, yes, I have been a little bit reflexive. I have worked with Aboriginal team members and I'm writing this with my own biases admittedly, but I'll try to address those biases in a certain way. So it's important to bring those things out. And finally, is this an article that you would cite yourself down the track in another article? So is this something that is worth mentioning elsewhere? And I mentioned before that a good article is something that is well read, that is well written and is well cited. And that's something that's worth considering as you wrap up your article. Article titles are also important. If you are wondering what should I title my article, sometimes it's tempting just to go for something that's catchy, or at least catchy for you, but doesn't necessarily describe what you've done. And so if you've got a thought in your mind that you want to test out, just type your proposed title into a Google search or a Google scholar search and see what comes out. And ask yourself, does the results match your field of research? Does something similar to what you are trying to say in your article come out in the Google search? If it does, then it's a fair chance that your article title is something that will help readers find what they want from your article, and then they'll be more likely to cite it as well down the track. So article title will help readers find your work. It should be descriptive, and it should make sure that it reflects what you are actually trying to say. I don't know how many times I've seen a title, I read the abstract, and the title doesn't marry up with either the abstract or the actual content. So make sure that the title is connected. There's some evidence to say that having two clauses separated by a colon makes a difference to citations. I'm not sure about that, but I've read that somewhere in an article just recently that having a mix of catchness with descriptiveness can be helpful. I've given you a few examples of some of my more successful, I think they're mostly research articles, but you know, posing a question and then putting an answer after it is one way of dealing with that sort of the two clauses. Having a colon and then having a catchy phrase might also work. But in general, try to be descriptive, try to use words that reflect the key words of your articles, and try to reflect the content of your articles. You might also be talking something about the evaluation design in your title. So you might describe this as a realist study in, for example, which describes something about the design of the evaluation that you were talking about. And again, avoid acronyms, even though they might be common to you. They won't be common to readers who don't know anything about your particular acronym. So avoid acronyms in the title, unless that's going to increase searchability, of course. But by and large, an acronym that describes a department or a particular industry specific descriptor won't necessarily work for you. In your submission process, you're asked to provide an abstract. An abstract is just a very brief summary of the article that tells you about the topic, the purpose of the article, what evidence you are drawing from, perhaps the theoretical framework, the methods that you're using, the significance of the work, and uses key words that draws the reader in. And generally speaking, most journals will sort of require this to be no more than a couple of hundred words. That doesn't leave you a lot of space to be able to do all of that work that that's described on that slide. But if you use your words wisely, it's not too difficult to create an abstract that covers off on all of those particular issues there and provides a clear argument and a clear picture of what it is that you're trying to do. So the abstract can be really important. I know for myself the first thing I will read, apart from the title, is the abstract. The second thing I'll read is the conclusion in an article. So if those things are the things that I'm looking for, wanting information about, then I will continue reading onto the bulk of the article. So a lot of times I read articles and they're very verbose. They use a lot of repetition. Long, flowery, adjectival descriptions which aren't necessary to actually describe what it is that you're trying to talk about. And one of the things that I think we should all avoid in writing is using the passive voice. Turning something around to make it sound like it's objective when we don't need to. One of the things I love even here is the ball was hit by the boy, which makes a lot more sense if you say it the correct way around, which is the boy hit the ball. And so avoiding the passive voice and using an active voice reduces that floweryness and it makes the reader able to understand much more quickly what it is that you are trying to do. So be succinct. Don't use unnecessarily flowery language. Unless it actually adds something to the work. So Jeff mentioned before, if you haven't published before, don't try to reinvent the wheel. Don't try to reinvent the standard structure for an article. Articles that follow this kind of standard structure with an introduction of literature views of theory, positionality statement methodology findings discussion conclusions and references will work fine 99% of the time. It's very rare that I would actually change the structure of an article to something completely different. Unless it was for a different purpose. This is for a journal article, not for a report or a minister's briefing or anything like that. This is the standard structure and a reader will expect to see these kind of headings in your article. And of course there are those variations based on quantitative and qualitative and mixed methods focuses where you won't necessarily have an evaluation question in some studies but you will have a hypothesis that you're trying to test. So that that might change. And depending on you know whether the focus is purely theoretical, you won't have findings to report in a purely theoretical article. So this is a standard structure but there are variations within that as Jeff mentioned earlier on. So one of the things that we need to include in a good article is something about what we already know. We don't want to repeat what's already known right throughout your article you need to find something that is is original and contemporary. And that's that's what you need to report on in your in your article literature review that you prepare. So to find what's already been written about your topic. Of course Google scholars are a very easy way to access that but I sometimes use a library database and sometimes I will use another tool like research gate. To help me better get a handle on what's already out there in my field what's been done in the last 10 years that is similar to mine. Now if you don't have access to a library. As a member of the AES you have access to a number of evaluation journals outside of the EJA and I'd encourage you to look in both the EJA as a source of information and the other journal articles that you automatically have access to. And then towards the end what you are trying to do is present something new. You might be wanting to challenge your readers you might be wanting to demonstrate some kind of innovation you might be wanting to challenge assumptions that are behind the work that you've done. So when when you're coming to a discussion section you've gone beyond your findings you presented what you you've already found. Think about how this might challenge practice how this might connect with theory how this might add to innovative ways of doing evaluation and report on that. It's not enough to just present the findings of an evaluation that doesn't make a good article that makes a fairly boring presentation of results at the best but it certainly doesn't make it for a good article. Tease out the things that are new and different in your work and you'll find that people will be challenged to think differently about your your work. There's there's also a good case for including a strong methodology in in your article. Regardless of whether in your report that you've written about your evaluation there is a methodology or not and regardless of whether or not it's a theoretical or a quantitative article or a qualitative or mixed methods article. There's a good case for having a strong methodology that's based on strong evaluation theory and that is based on methods that are that should be reported and reflected on in the article. And the methodology should of course include a research question or research questions and hypotheses if you are using a quantitative method and describe what your analytical approach is. So every article with with only a handful of exceptions should have a methodology that describes what you did and how you've arrived at what you've you found. I think I mentioned earlier about positionality statements. This is a relatively new thing that sort of seems to have come in particularly in the social sciences and in particular the qualitative research literature and it's a reflection of the way that I think evaluators are now becoming a lot more reflexive about their position and acknowledges the I guess the difference in cultures that we we come from as evaluators and so if if if I as an Australian citizen is working with migrants I come with a set of biases that I don't even consciously articulate or think about because I've grown up in this country. But for migrants coming from a different country and if I'm doing something about a migrant program I need to be aware of the difference in culture that they they bring to their practice or to their program and try to reflect that in my reflexivity but also as make sure that I've got a team that reflects at least in part the the makeup of the the program or the participants of the program that I'm doing an evaluation article on. So if you're an outsider working with another group, then it's really important to acknowledge that you are different and that you come with different values and different biases. So it's about describing yourself in the in the article. You might not have seen this before but I can guarantee that you increasingly will find it in articles that you are coming across in the new literature. Ethical considerations are also important. We have a policy that the any article that reports on findings should be based on an ethically approved evaluation unless it's a purely theoretical article for example. But if you're reporting on findings, then this the your article should not only say that you have approval from an ethics committee, but that you have thought about the ethical issues that are associated with the the the findings that you're reporting on. If you're working in Aboriginal and Torres Strait Islander contexts in Australia, having a look at the cultural safety guidelines that have been prepared by the AES would be of incredible advantage to you. Reference list is another important thing that you need to include. Make sure that there's a citation for every reference and a reference for every citation. It's much easier to do good referencing if you've got end note or some other kind of referencing software to help you. A little tip. If you, if you think a reviewer, if you, if you, if you think a reviewer might like to see their name there because they are an expert in your field, make sure that you've cited them. And it could just help you have a favorable impression if you, if you are citing people that are not experts in their field and have ignored the experts, then you can be sure that a reviewer will pick you up on that. So what makes a really good academic research article? It's one that that really clearly expresses the rationale for why it's important that you write about this. It's an article that has a risk, a writing style that is accessible. It's got a logical, clear structure that conforms to usual academic conventions and keeps the language simple at the same time. It's an article that's got that's based on theory, whether it be evaluation theory or whether it be social theory or both. It's an article that expresses the ethical considerations, the risks and the benefits and the cultural considerations that are associated with your work. It's an article that that has got clear statement of findings and succinctly presented. You don't want to waffle on. It's an article that's got a discussion section that tells us about what's new, what, what contribution you're making to, for example, ethical practice or evaluation practice or evaluation theory or philosophy. It's an article that perhaps challenges and makes people think. It's an article that perhaps reports on the failures as well as the successes of an evaluation. And it's an article that expresses conclusions that answers that so what questions about what really matters. And it's an article that that uses references and citations really wisely. Jeff, briefly, if you can just explain the submission process for us, please. Sure. So once you've got your nice polished manuscripts taking account of all the points that John's just been talking about, what you need to do is you go online and submit your manuscript. And essentially it comes to the editorial team who have a look at it and decide whether to send it out for peer review, which is the usual kind of process, potentially accept it straight off the bat, which never happens, or to reject it. And also at this stage, because what we try to do with our processes is to be very supportive of publication. If we feel that actually there's some good stuff here, but it's not quite ready. We may send it back to you and say, there's some good stuff here, but it's not quite ready. How about addressing this, this and this and then we can send it out for peer review. So we're going to be kind of as helpful as we can at that stage. And then what happens when it's accepted into the system is that it goes out to peer review. So peer review essentially means two or three other evaluators will have a look at your manuscript and offer suggestions, offer their view on the strengths and weaknesses of the manuscript. And then they send it back to the editors and the editors will then decide whether the author needs to do a little bit or work on it, or whether we can accept it or reject it at that stage. The typical processes is that there's some suggestions for ways to improve it. So it goes back to the author, the author makes the changes and then it goes back through the process. So there's kind of a bit of a circular process that goes on until we get to the final decision, which is likely to be that the manuscript's accepted. So if you get back suggestions for change, it's a pretty good indication that people think that there's some likelihood that this will get published. So if it comes back, even if there's a lot of comments that comes back, do address them because it's a strong likelihood that it will get accepted if people have offered those kinds of views. And so it goes around in that little process, circular process until it's ready. Two rounds through that process is quite common. Sometimes it can be a little bit more. And then as I said, at some point there will be a final decision made which will either be an accept or a reject. But if it's been around a couple of times and you've been responding well to the comments and improving your manuscripts, as I say, it's likely to be, it's more likely to be accepted than anything else. And then after the exception, after it's been accepted, it then goes off to Sage for production. And within two or three weeks it would be available online and people will be able to look at it and read it and see what you've got to say. So that's the process. Thanks, Jim. It's not that complicated. You put your stuff in and we deal with it. We'll let you know when it's ready basically or when it's not. Don't be too concerned about the flowchart there. If you need more guidance, let's say you've got a paper or an idea or an outline that you want some help with, then feel free to email us. Jeff and I are more than happy to sort of engage with you before you submit the article. If you want to read the guidance that's available online, the QR code there will take you straight to the page that gives the author guidelines. But certainly just talk to us, you know, or send us an email. We'll have a bit of a yarn about your ideas and we'll take it from there. And when you think it's ready to be submitted into the system, then it'll be ready. So we are not sort of in an ivory tower somewhere. We are practical people. We know what it's like to put stuff into a system and wonder what the hell is going to happen with it. And we want to be able to help AES members to be able to better put their good work into a public domain where other people can read it and get the benefit from it.