 Hello and welcome. My name is Alon Friedman and with me is Zal Basely and we are from the University of South Florida, Tampa, Florida. And we are going to talk about teaching art with peer review and a neuro break. The idea of a peer review is well-documented in the literature and has been a lot of research on the subject. It is becoming the idea that in conducting peer review we are benefitting from implementation, ability to scale the whole aspect of the teaching and ability to collaborate and to learn from other people their point of view about our code. So one of the new implementation of peer review can be found in GitHub. How posted on Medium how to conduct peer review. And his recommendation for best practices for students is that start review with each part of the code, make sure it works as expected with a clean code guide. However, in the art community we do not have still the idea of a clean code guidance that is missing. Another aspect is the missing part of the peer review is a rubric. What is a rubric is the outcome of a lot of development. And a rubric is also a great facilitator for teacher, educator to setting an agenda for the assignment. Which means that we as a reviewer has an agenda that we want the students to go over in order for them to become aware of how we criticize or how we evaluate their work. And the idea of it is that the use of rubric is the idea is that we are looking for quality. And when students use a single rubric usually to judge their own work and begin to accept more responsibility. And the literature again the academic literature quite defined the benefits of a rubric in order for us to do that. It is one of the things that Adele in 2000 says using rubric to promote thinking and learning. And this is basically the objective of every instructor and every teacher when he comes to the class. So the common rubric in Java usually outline how to scale their assignment. And it usually talks about correct output, the design output. But the question begins is how do we create a rubric that will reflect accordingly to the R. At USF we started to teach R since 2019 and we started to implement the rubric only in 2019. Because we did not feel comfortable yet to establish a rubric that will be transparent to the students. And this is what you're going to see. So we came up with the idea that a rubric can help the students to understand R. And we created a rubric, a rubric that they will see, they will understand. And the rubric will consist of tense categories, focus, the idea of analytics, the idea of code. And we divided the code into two aspects. The code itself and the design of the code, the documentation, the testing, the visual and the accessibility. All those components usually cannot be found in a written document or a writing assignment. So when we, I started my conversation, I said what is a student engagement? Is the idea that the engagement refer to the number of words students wrote in the feedback in the peer work? So meaning that we use the peer review, we gave them the ability to write back the feedback and we count the number of words in order for us to illustrate if they are engaged and what is the process. So a study research or the study research is can a new rubric design for the introduction of R engage USF students? Again, this is not a large scale study. It is only 26 students and the backbone of the study was Canvas. Canvas is an online system and one of the drawbacks of Canvas, you cannot pull the data out of it. You need to do it manually, which is not easy to do. So we collected also in addition to collecting the students comments, we also collected open questions, questionnaire in order for us to get more input. So all students attended the class participating the peer review. The study found that more students wrote more in the final project than the weekly assignment. The ratio was two to five. And we also found the most common and common terms that they use throughout the assignments. And we also ran time series and word count in order for us to see this idea that in the beginning they did not feel comfortable with the peer review and the rubric. But as the time goes on in the class, they started to understand what it means and how they can learn from it in order for us to do that. I want to add two comments from students and one of the comments, I think, illustrates how powerful the rubric can be. So the rubric make me focus on the entire framework of R and R package. At the beginning I focus on the attention of the R code and by reviewing other works make me realize that the rubric gave me a structure of the development. So this is the benefit. But what I want you to take from this study, which is what it's all about. So this is a case study. This is not a generalized study from the entire population. And this is important. The results suggest that the using of rubric increase students' engagement. As I said, they put more words in the final project. The study found that students show increased engagement and that's something that we need to do. My hope in presenting this result is that many R educators will adopt this rubric and collaborate and extend their knowledge based on the idea of the next generation of R user will try to be more understandable to new users and how we can implement the rubric in order for us to succeed. That's it. I'd like to thank you and I hope to hear from you. Thank you.