 Hi, my name is Adam Guita and this is my talk on student practice sessions modeled as ICAP activity silos now If I was to be teaching you something like lists or recursion in a perfect world I just talk about it and you've mastered the topic But we know that that's not really the case and in fact computer science has a number of different Exercises that we give to students to build and hopefully refine their understanding to Hopefully move towards that topic mastery Each one of those activities can be modeled using the ICAP framework which Categorizes exercises based on their levels of engagement. For example, you're currently watching a video that I'm Starring in and it's a passive activity You simply need to just sit back relax and hopefully learn something the entire time But then we can add in a little engagement For example a typing exercise requires students to look at source code and then physically Retype that code out. They can't copy and paste it This is very similar to rehearsing your lines as an actor or practicing scales as a musician or learning to play an instrument We can go even further by adding in more engagement into a constructive Activity something like an output prediction shows a snippet of code to students And they must process what that code does does and then say here is what is going to be outputted if you were to execute this code Finally, we have interactive engagement and this is where the standard computer computer Programming exercise comes into play Students are instead given a prompt and they must figure out a solution and implement that solution using a programming language But they may not get the answer correct the first time And so the feedback that they're receiving from the system or even an instructor Creates more of an interactive activity now For our study, we're not looking at these activities and seeing which one is better than the other But rather how when students are going to practice, how are they selecting these activities? Are they going from one to the next and is there any sort of connection with these iCAP categories? We also want to look at platform silos and I will butcher this name But a sachadri had found that students when they were learning computer science on Multiple educational platforms would focus on just one platform during any particular study session So if you started the session on Moodle, you are most likely to stay on Moodle If you were on github, you were most likely to stay on github, etc, etc So that leads us to our research questions RQ1 asks well, can we replicate the platform silos using a new data set? RQ2 says well given those activity types that we talked about are there common transitions between students RQ3 then says well of those transitions Is there any connection to the iCAP framework and finally? When we look at completers versus non completers are their differences behind their behaviors So for our study, we had 67 students enrolled in a professional development course teaching Python and data science online This was a prerequisite for a larger AI program and this course had 10 modules Each module had reading materials slides videos and a module assassin Each module had a module assessment You had to receive a passing grade on that assessment to move on to the next module and to be classified as a Completer of the program you needed to pass all 10 modules in addition we used a another platform known as typos and typos hosted 24 optional practice exercises of those exercise types for each module So for example in the image, we can see that there are three typing exercises for lists There are three fill-in-the-blank exercises for lists three Parsons puzzles, etc, etc so our data set in total is that we had 37 completers and 32 non completers now the first thing we wanted to do is first look at those completers and just see What was it? What were their activity transitions and to do that? We built a diagram So for this diagram, we have we have two separate types of lines a blue dotted line is indicating an activity that was held on the typos platform in a solid red line indicates an activity that was Involving the Moodle platform. We also do have a single solid black line in which a student transitioned from Moodle to typos In particular when they were on Moodle 39% of the students starting behavior to a session was going to our module assessments This makes sense since to be a completer you needed to pass those assessments, but 26% also was on Content consumption so looking at the Moodle slides or watching the Moodle videos However, 36% of their starting behaviors also involves typos practice each individual exercise doesn't have a large Percentage, but if we were to add all of these up it does come up to 36% One thing to take note of is you may notice that something like the fill-in-the-blank activity here The this would be the start of a session. They do the fill-in-the-blank and they repeat through this and then The highest probability was that they would end the session However, when a student would then go to another activity They always either stayed at the same iframe itap level or upgraded. They would never downgrade So for example a self-explanation exercise here with se We see that there was a high probability that they would move to a coding exercise And in this case se is a constructive activity coding exercises are an interactive activity And we can see that again on three other or two other locations So with this in mind, we wanted to strengthen our observation that students were staying within our platforms and there were particular Types of practice sessions and to do that we built a we used a factor analysis To start we wanted to look at each one of the sessions a student may have and in particular The presence or absence of an activity so for example this first graphic is showing that This particular session had Moodle slides and Moodle videos and nothing else The second activity shows that they had output prediction find the bugs and fix the bugs and nothing else So once again a factor analysis is a dimension reduction Method that looks at all of the observed variables and then attempts to create new ones based on that behavior And for our eigenvalues we chose to use a three and five factor analysis So to start our three factor analysis with completers our F1 score shows heavy loadings to constructive and interactive behaviors our F2 is showing heavy or high loadings to active and constructive behaviors and Finally our F3 is showing high levels of passive behaviors This is very similar to what we were seeing in our Activity sequence diagram, and so this is why we went with increasing our factor analysis So once again now when we look at the five factor analysis that F1 is still showing us constructive to interactive behaviors and our F2 is still showing us the active to constructive behaviors and Finally F3 is still showing us the passive behaviors However, we do now have two additional variables that have been created F4 and F5 and these are showing Interactive behaviors solely so for example the module assessment by itself and a coding exercise by itself so our next step was to then look at the non completers and in particular we use the same three and five factors for these as well and Coincidentally you notice that The F1 F2 and F3 scores are still showing the constructive to interactives the active to constructives and the passive behaviors Even though we're now dealing with non completing students when we expand this to the five factor analysis We're still maintaining these constructive to interactives active to constructives But we do start to see some slight differences when we get into three to five factors Namely because a non-completer did not complete the module assessment And so as a result they would not have a high loading for that but the key takeaway from this Talk was that these individual activities are showing that same never Downgrading from an icap motion if they start with an active activity They're going to either stay in an active practice mode or they graduate only one level higher Likewise the completer to non completers. They practice the same way Thank you, and I look forward to your questions