 Let's start this out with an example of what the helper chain is going to have in half a month. Grabbed by the function. Yeah. We're going to be fine. Well, in order to understand it, we're going to have to do that. So we're just going to level it once, and now I can no longer get to my bookers. We got a video demonstrating inter-observer agreement. And of course, we're going to do it in typical psych course style. One of the things that you'll need to know is that we have broken this video up into 30-second intervals. Those 30-second intervals are where we're going to count our behaviors within. So with that said, pay attention. We've got Brad recording some here, and we've got Ryan recording some here, and you will see how it all works. All we're going to do is put a tick mark for each time we hear the word meow, and we're going to start looking at how well Brad and I agree. So with that said, watch. Have fun. There you are. Might as well come and join you. Anyway, so we've been doing all sorts of stuff, but I figure it's time to talk about what shall we say. Measuring behavior, right? There's lots of behavior that we can measure meow, but ultimately speaking, there's ways that we have to think about measuring behavior. So when we think initially about measuring behavior, we want to think about three things. Repeatability, right? These are the dimensions that we're going to worry about. So repeatability, I don't know if I can say that again, repeatability meow. We can then think about temporal extent, and then we can also think about temporal locus. Repeatability pretty obvious. Temporal extent, how long does this go on in time, right? The extent. And then temporal locus location, where are we at? And I'm sure by now you guys have figured out where we're at. We do a lot of recording in Spokane for now. So it's kind of fun. But hey, they've got all these really cool things to worry about. So meow, let's take a look at some of the behavior that we can work with. Because we're really talking about measuring behavior, right? So we can dance around like this and do all sorts of things. That really has nothing to do with what I'm talking about, but I just figured that I would because it sounded kind of fun, meow. Ultimately, meow, we have to think about how we're going to do the measuring meow. Number one is count, right? How often meow does the behavior happen, meow? That is a very, very basic one. One that's more important to us is often rate or frequency. They're different, but we're not going to really differentiate right now, okay? So rate and frequency, it's a count over time, meow. The next thing that we want to think about is duration, meow. How long is meow a behavior going on for? It could be going on for a long time, that's our duration. In a response time, meow, meow, meow, I mean meow, in a response time is the time between the responses meow, which is something that we often worry about the frequency and over time, which also gets us to that inner, the IRT stuff. Folks, I'm hesitating a little bit right now, meow, because I want you to understand, meow, that there's an entire set of videos just on this. So please go watch our other videos on each one of these topics. We have some great examples that go along with them. We've got some other things here that we need to think about when we're talking about measurement. We also want to focus on, let's see, magnitude. We could measure how intense meow the response is. Meow. So we have that sort of responding along with it. The inner response time, magnitude, duration, count, frequency. Brian, am I missing anything else? Latency. Latency is the time from the stimulus till the time, another thing that we can measure about behavior is the latency of the response, right? So how long it takes the response to occur from the time that the prompt happens. So meow, if Brad says, hey, Ryan, start a video and I go like this. Oh, now it's time to start a video. That's kind of a fairly long latency. Or if Brad says, hey, Ryan, start a video, I go right now, I start getting into my videos and blah, blah, blah, blah, and I start doing my lecture. See the difference there, response latency. So again, folks, watch our other videos on these. They're much more in-depth, the little penny-packer dance videos. And remember that we work as a team, meow, and meow, and you're eventually going to realize, meow, that what we're doing with this video is also setting up some other meow videos. And hopefully we can even start, meow, breaking this down and let me out something that you will be able to have some fun with for inter-observer agreement stuff. All right, so anyway, more on all that stuff later, I'm gonna go play on some blocks. Bye. All right, because we're working with event recording here, we can look at three different types of inter-observer agreement. Number one is total count inter-observer agreement. This is where we're just gonna look at the total independent of the intervals. So we're gonna take the smallest number, like brides or mine, and divide it by the largest number. And that's gonna give us a figure. It sucks. It's gonna give us a figure. The other thing we're gonna do is mean count per interval, where we're going to take interval one, and take the lowest number, divide it by the highest number. So if I'm lower than Brad, then we're gonna take mine, divide it by Brad's score, vice versa. Then we're gonna get a percentage for that, and we're gonna write that for each interval. Then we're gonna take an average of all of those intervals. How do you do that? You add up each interval, right? So the percentage that you got between each interval, so the first one, the second one, the third one, we've got eight intervals here. So we're gonna add all those up, and then we're gonna divide them by the total number of intervals. That's gonna give us an average amount of agreement. So in other words, we're saying a mean count per interval, right? The next one is exact count per interval. And this one is where we're only going to say we agree, if we're only going to count that interval if we actually agree. So then we're gonna look at the total number of intervals where Brad and I agree, and then we're gonna divide that by the total number of intervals available. As you can see, based on the type of inter-observer agreement that you choose, you're going to get completely different accounts of that agreement. So if we were on total account, which by the way is the worst one that you could do, so total count, we had 88. On mean count per interval, we had 86. And then of course on the exact count, we had 50. So overall, not too bad, right? Exact count is clearly more strict than any of the others. And total count is the weakest. And let me give you an example why we really, really, really don't like total count. Imagine if I had 23 observations of Meow in the first interval. In the remaining seven intervals, I had zero. And then imagine if Brad had zero for the first seven intervals. And in the eighth interval, he had 26. We would still have the exact same number. We would still have 88% inter-observer agreement. But we never saw the behavior happen at all at the same time. And in fact, there's no overlap whatsoever would be horrific. But the inter-observer agreement number would give us 88. This is also an example of why we can't call this inter-rater reliability. Because it's not a measure of reliability. It's really a measure of believability. How believable is the data? So a lot of times, we like to report multiple. You choose one, but you kind of want to analyze as many as possible to figure out which one best represents your data. Of course, the exact is going to be hyper-strict. The mean count for interval in this one, I like it. So if I was trying to present this data, I would say, you know, I'm going to choose mean count for interval because I like that one. But I'll probably put the exact count up as well just to let people know there was some variation. I mean, those Meow's were coming fast in a couple of those intervals. And for the record, by the way, folks, neither one of us got the correct number according to these observations. I'll let you guys figure out what it was and argue in the comments. But I can tell you for sure that there is not 26 Meow's in the video.