 All right, so let's take a look at reading first, and then we'll talk about writing. I mentioned two documents that are on the resources page of the flight website. And the first one I want to show you is this reading worksheet that I've developed, a master. And it ties into essentially your assessment, which I really like. And I can see it follows that narrative arc. It gets students showing their comprehension of the text, but also getting them to write in simple sentence form. The question that I really want to address when I'm thinking about assessing reading, there was a study that was done a number of years ago, and I can't right now think of the name of the people who did the study. I apologize. And it has to do with what can be called cultural gaps in reading. So when we're reading a foreign language, and especially we fall into that trap perhaps more readily with languages that have cognates, because we take our meaning and we attribute that meaning without realizing it, and all of the cultural associations that go along with it to that word in order to make sense. So there was this study involved a text that was about a wedding. And these were two anglophone groups, one in India, one in the United States. So it was a description of a wedding. And what the study showed is that the way that people conceptualized and all of the associations that went along with what a wedding is, the kinds of human relationships and the symbolism of various things, they ended up interpreting those texts quite differently, even though these are two native speaking groups of English. So the notion of culture gap, how words encode meaning beyond just the direct referential, all of the social values that may be tied to it, the many cultural layers of meaning that may be tied to it. So how do we get students, I mean, other than constantly giving them footnotes or glosses and explanations, oh, well, in this culture, this means this. So my quest really is getting students, again, intuitively to start to get that feeling of when they're encountering some kind of cultural construct. Where are they in their reading going astray, potentially? That's what I want to get at. So this worksheet, so let me just walk you through quickly, and you'll start to see where I'm trying to go with this. So first is read the text once quickly without consulting a dictionary. Write a summary of the text in the target language in three or four sentences. So this might be, let's say, again, if the instructional language is in English, you could do it at the introductory levels, or you may want to hold off on doing this until maybe the intermediate level, but essentially getting them to write a summary in three or four sentences, just as their first reaction to a text. Read it, boom, what is it about? Then identify the type or genre of the text, because often misunderstanding occurs when they don't really get what the genre is. They're not attending to it, so they're not thinking about what kind of relationships may be involved because it's a particular genre. Then going further, so read the text a second time, underline the words, phrases that you cannot decode, then okay, they do their work with dictionaries, et cetera. Then what do you understand to be one of the main themes, one or more of the main themes of the text, and then do some basic research in English or again, depending on the level in the target language, about the text, the author and any cultural or historical phenomena that are mentioned or referenced in the text. And note three or four pieces of information that help you to better understand. And that could be almost anything, right? But if they can glean something that say, oh, okay, yeah, actually I hadn't understood that, right? Then we get to the reassessing stage. And here, after having done your research, if you understand the text in new or different ways, modify your interpretations. So your text summary. So sometimes students realize, whoa, I didn't get that at all. Now that I understand it's a little better, they rewrite their summary. Your categorization of the type or genre of text, they may have misinterpreted what sort of genre it was. If a student is not recognizing that, oh, there's this hidden journalistic writing in there and what sort of layers of meaning does that bring? So maybe they have to write a description of a genre that is a blended genre, right? That helps them to come to another understanding. And then what words or lines in the text most attract your attention and why? Here's to this notion of appreciation. I keep trying to tie this in. I know it's such an outdated word, but I want students to love language. I want them to be interested in language. So, and it's so fascinating to me what different students come up with as the word or the sentence that really something, that strikes their fancy in some fashion. Or that disturbs them, any emotional reaction to it. So then write two or three words that express what the text or author makes you feel. And then finally, questions. Write two questions that the text raises for you and write a question that you would like to ask the author. So this gets them working in a way that's getting them noticing language, hopefully I mean these are the goals, noticing language, questioning meaning, questioning their own assumptions and hopefully coming to a more grounded interpretation, finally of the text and then their discussions with their classmates, they learn new things, different ways of thinking about it, et cetera. So that is something that I do as out of class work, but of course you could adapt that in some fashion to do a formative assessment as well. In fact, I want them to know that so many gray areas of language and that you can't just assume that you get it, right? Do not to feel fearful of it, but it's always that open-mindedness of, hmm, maybe I need to rethink something here. A lot of students, you ask them what is the theme and they're gonna be like, I don't know. How do I find the theme? What is the theme? And that's why I felt like, you know, when you ask them to summarize it, I feel like they need some kind of guide to figure out how to summarize it. True, and this is collegiate level, so keep that in mind, right, so. Yeah, so for example, what would you use to help them figure out, I mean, so for example, the steps that I gave you to help students summarize it, what kind of steps would you use to help students find the theme of a story or the theme of the literary work? You can break down, give examples of what you mean by themes, and you can use the word ideas, what are some of the main ideas in this? What do you think the author is trying to, what are some of the, what is the author trying, what points of the author is trying to get across? What is the author getting at, or something like that, right? And that's gonna depend on the text as well, so you can always play with that according to your needs. Again, this is meant as just a kind of a template, right? But for me, the key is some kind of an activity, process that gets students reevaluating their initial impression, because going back to the study that I mentioned, here's the fascinating piece of information, and this is what so struck me when I read this a number of years ago, is that students' first impressions when they read are very hard to undo, almost impossible, right? So even when you give them that gloss, so this is what this means, they're still kind of going ahead, they're hitting their heads against the wall, right? It's not to say that they never reassess on their own, but it's hard to get them, so this is a way of working that just consistently gets them reassessing, reevaluating, questioning, wondering, you know? Where might I be going wrong? I want them developing those intuitions. Sometimes what's called a reading matrix is also used for that, and depending on the text, that can be a really productive way at getting at a theme without asking the question. So one example that came to my head was a poem we work with that has a lot of oppositions between urban spaces and nature, and it's kind of dealing with the tension, and one of the things I've asked students to do is just list the nouns. First, where are the nouns? Underline them, and then say, do you see categories of nouns? Could you group these in any way? And often they'll come up really quickly with concrete, skyscraper, all of this, okay, this is about city or about urban or something like that, modern, and then the other words that are all about plants and about the birds, they come up pretty quickly with that has something to do with nature. Well, now we're talking about theme, but I haven't asked them what the theme is, and they're paying really close attention to the language. So sometimes I think those kind of graphic organizers that you use can actually be a way into talking about theme as well, especially if you, because one thing, if your students have never written a summary, I'm thinking my fourth graders just learning to write summaries at this point. If you're teaching fourth graders, you can't assume they know how to write a summary in English, let alone in the second language, so then you have to kind of give them different kinds of support. Yeah. What did you call it again? Reading matrix is what they're, and that term I stole from Janet Swaffer, who was a professor here at UT Austin, and she's got a book called, Reading for Meaning. Remapping. Oh, well, that one too, but the Reading for Meaning, which is co-authored between her and Heidi Burns and Catherine Arons. Catherine Arons is also a professor here. They have a lot of great examples of just how to kind of find people's way through reading. They're talking about collegiate, but they do a lot of graphic organizers. And actually Janet in her book, and I can't remember her co-author, Remapping the Foreign Language Curriculum, it's also okay, that's where I got the example of this study that addresses cultural gaps. So yeah, so she's actually good. So social reading, I'm gonna give you just a quick introduction to it. And on the Coral website, because Coral has created its own software, which is open, right? You can, it's now also utilizable in any management system. So if you're using Blackboard or whatever, Cornell will use Blackboards, that comes to mind. You can work with it in pretty much any system. And it's free. So social reading. So essentially what you do is you upload a text, and that text is going to then generate of course a word cloud. And when you click on a word, it will highlight it in the text. You then have the possibility of writing comments. So essentially what you do is students are reading at the same time, either at the same time, either synchronously or asynchronously, but they read a text and they're commenting on the text and they're asking each other questions, they're making comments, they're making sense of or interpreting this text together. And they can work with the word cloud, they can write comments, they can have a dialogue. And as a teacher, what's very nice is you can then see all of the users. So if you're just doing this as far as completion of homework, you can just quickly see, okay, who did it? Who didn't? Did, and how many comments? You know, if they only did made one annotation, that may not be sufficient for you for considering that successful completion of homework, et cetera. And then you can click on and you can see what those comments were. And essentially you have, it's kind of like what used to be a read aloud protocol. Essentially you have a mapping and a documentation of all of the thinking, the interpretation that students collectively went through and who is helping who to make sense of certain things because students have different areas of knowledge that they bring to reading a text and sometimes someone might know something or notice something. Oh, well, you question this, but look, in the next paragraph, it actually says this and it seems to me that, so instead of it always being teacher-fronted, I mean this is a very genuinely learner-centered kind of activity where the students are working collectively and I prefer not to participate at all. For me, this is a first reading of a text. I allow the students just to go at it and do whatever they can do with it. I read through all this work before class and it allows me to then tailor my comments and input because sometimes they still will have a problem of comprehension that they're not able to resolve collectively, so we can address that in class. Sometimes they notice things that I didn't notice. I mean it's just, it's wonderful. I mean it really opens up this springboard for very rich discussion. And years ago I had done a, when Eekama first came on the scene, I thought, oh, this is wonderful. So I was one of the people who wrote a little case study for it and I used this in my first semester French class and again the instructional work and the comments that they made, this is for reading comprehension were in English, but I got these wonderful, this wonderful range of comments. Co-constructing meaning, connecting reading to personal experiences, interpreting meanings of textual features, reflecting upon cultural differences, evaluating meanings of foreign words. So students were doing all of this on their own and unprompted and I came to some basic conclusions about that. Then in my intermediate level class I started thinking more carefully about the process that they were going through in making meaning where they were having trouble and how I might address that more effectively and back in the resources, the teacher's guide, the interpretation strategies and social reading document. And here is where I talk about these cultural gaps and mental landscapes and I actually, at collegiate level, I actually give this information to students because I think it helps them to understand my objectives, what it is that we're doing and why, right? Why it's important to get them to rethink, reassess, not fall into this trap. And here's where I cite that example. And here is where it says, right? Students resist correction of first impressions, that initial misapprehensions about textual features can become entrenched misreadings. This is what this study shows. So I work with this notion of construal that comes from cognitive linguistics. Construal means essentially how the speaker, the reader, is interpreting or what nuance of meaning, what associations, right? How that person is conceptualizing a word. So my approach to interpretation, I'll just go, I give an example from French and I talk about where I see the problem in the student's comment and I'll just go directly to the interpretation strategies. So I've written down strategies with the idea of helping people to become more intuitive. So the first is red flagging and this is when they do social reading, they can actually follow these steps as they do their reading. So red flagging means read the text with your gut. Whenever you experience a point of tension, surprise, annoyance, curiosity, disappointment, disapproval, et cetera, right? Flag the word or passage by highlighting it and write a comment that identifies your reaction and the source of tension that produced it. That is the one thing that on our own we have is our gut. What's telling you that there is something that's maybe not right? Other than that, we don't know, right? We can only feel, we can intuit. So the second then is questioning the text and assumptions. So read classmates' questions or comments, think about the points that you've read flagged, what sorts of assumptions might you be making, what questions can be raised, et cetera. Then finding evidence. Because this is online already, students have direct access to researching online. Oh, you have this question come up? Okay, let me do a little thinking further and get some so they can use the word cloud that might help. For example, what are the dominant forms of punctuation? What might this tell you about the tone of the text? What are the dominant verb tenses or time frames? What might this tell you about the character's perspectives or the genre, pronouns used and why, et cetera. So that word cloud can actually be used as clues to helping answer questions. Then I give them some references for where to look for possible glosses for things. Finally, formulating an evidence-based interpretation. When you have a working interpretation that's founded on evidence, meaning context clues plus supplemental information, linguistic, cultural, historical, social, share your interpretation, and that could be in the form of a definition, a sentiment and explanation with the group, citing sources where necessary. And here's where I say, do not make guesses about meaning based on what you feel or imagine. That's the pitfall. You say, oh yeah, but I think, no. No, get some evidence, right? I give them an interpretation strategy to work with and here is the rubric then. Red flagging, noticing tension points in the text, questioning the text and assumption your own and classmates, finding evidence, internet research, evidence from the text, formulating evidence-based interpretations. So you can read through their comments by student, you can see what each student is doing and you can assess this process that you're asking them to go through. And what I do is I have them, as I mentioned in this handout, I have them do three social reading activities over the course of the semester. The first time they do this, clearly they don't have a good handle on all of these steps. So I give them that feedback, you know, you say this, but this is a direct assumption on your part. You haven't substantiated in any way and I think maybe you can do a little bit of research here and oh, you see, say this, but actually in the next paragraph, whatever. So I give them that feedback. Next time they do social reading, they're often many of the students who are having problems the first time, a little bit better and by the third time I find that they're actually the classes functioning quite well on this level. So my final grade, it's 15% of the course grade is based on these social readings, but for me it's kind of an accumulative grade. What I really want to see is at the end of that whole process of the semester of doing this, can they effectively do social reading? So that's what I base my final 15% of the course grade on. So some students don't notice anything, notice tension points. I'm asking them try to read with their gut, try to make explicit where you're finding a point in the text where you think something else is going on and some students just don't do that. Now they're not feeling it, they're not demonstrating it, so therefore that's feedback that I can give them because I mean I always choose texts where I know that there are those tension points in the text, that there are things that they cannot know because they don't have the cultural background or maybe perhaps not the linguistic or whatever aspect of historical, whatever it is. So I know that, well like this word wedding, going back to this, so that would be if students were to red flag that word and say, huh, I'm seeing other information in this text that makes me think that weddings might be different in this context than what I know of a wedding. That would be an example of successful red flagging of a word, a concept, right? So if a student doesn't have any of that in their comments, then I would point that out. Didn't do any red flagging. I mean students, especially again within the context of the college, yeah, college context, they're learning to take an authoritative voice on things and to sound as if they know what they're talking about. And when they read, they'll often make a statement, just this outright, boom, this is this, this means this, wow, you know. So what I want them to do is to step back from that. So if they see maybe someone else who's not questioning what they're saying, they should be commenting on each other's, right? It's about this collaborative coming together. And I don't have an example to show you, so it's sounding very abstract. But once you start working with comments and you start thinking about these different areas, you do see it, so it's how I was able to develop it because I saw these issues in the student's comments. So finding evidence, sometimes students won't say, they'll say, oh, well, this means, you know, they'll go to, let's say, a dictionary, an online dictionary, and they'll come up with a definition of a word, which happens to be the wrong definition for that context. And they'll think, oh, I've satisfied this task of doing internet research. Yeah, okay, here's the meaning of this word. No, they didn't look at the context. So this is a template, again, and it's something that needs to be adapted for all sorts of reasons. So if you want to go the root of actually writing in that precision for each time, again, because it's a cumulative process, for me, the grade is a kind of general grade that they get on the first two, and by the time we get to, and I give them that feedback, they start to see, oh, I had problems here, I had problems here, by the time we get to the third reading, they do have a good sense of, and I have a good sense of who is, may still be having problems who doesn't. So again, it's working somewhat intuitively, so you may not be as comfortable with that. You can certainly always go into further specificity, but it will depend to a larger extent on the text. When you're grading, you're seeing what students are able to do as a class at that particular stage, the first reading, the second reading, the third reading, and because this is collaborative, right? So sometimes maybe a student didn't flag something, I mean, flagged it, didn't articulate it, and another student will come in and comment and say, oh, I feel something too here, this is what I'm getting from this, and maybe that student, first thing, will write back and say, oh yeah, you're right. So it's collaborative work too that's gonna ultimately come to this richer understanding of the text, it's a process. I'm trying to give them a set of reading skills that is different from standard reading skills, but that's going to help them, I think, become better readers, especially outside of a classroom context where they don't, and also just if you're traveling, this is these kinds of tension points, when you travel and you're constantly surrounded by things that you don't understand, you cannot understand, you can't be expected to understand, but if you develop those intuitions of, oh, there's something going on here, then you can ask for clarification, but if you're just going through kind of life-ly, mindlessly in that sense, or whatever, just okay, great, everything's fine, bam, you get yourself into trouble. I'm wondering, those who teach at lower levels, so at like a middle school, this is really kind of about self-awareness, my awareness as a reader, and I'm not sure how that would look for like an eighth grader or a seventh grader, so you're asking a college student to be aware of their disappointment or confusion or whatever, I mean, those of you who teach at other levels, is that adaptable, is that a possible, I really don't know, I'm asking kind of, is that asking too much, does that think it's doable? When there are two questions aren't there, there's is it doable, right, and is it doable and does it look like what tests that you're, if you have standardized tests like the tests that you have to speak to, does it match them in a sense? I keep thinking of my son, this is not a second language context, but he came home after doing a standardized test and said, oh, we had a question that was, why did the author write this? And my first thought was, he wrote it because it was fun to write it, but that wasn't the right answer, and I said, I think that's a really good answer, I mean, I imagine he did write it because it was fun to write it, but what they were actually asking was, what's the theme, what's the message of the text, and so there's this delightful tension sometimes in teaching between what we have to test and what we think the ideals are, and I think this is a place where I imagine the tests that we're hold accountable to might not be looking at that, but it still might be a desirable outcome that we want to foster. And I would imagine that red flagging as Joanna's describing it does not at all play a part of standardized testing because it's really about being aware of yourself as a reader. Being a better reader. To be a better reader, so yeah. However, when you take that standardized test, you're a better reader, you notice things more, and you pick up on meaning, even if it's that more direct comprehension question, fill in the gap kind of discreet point, question point, you should be able to, in the text, have a better sense of, okay, where is that answer, and it just gets you deep, as a reader, it makes you read more deeply. I mean, I don't, I'm not quite sure how else to say it. Hmm? Become more critical. Become more critical as you read, right? Not just take things face value, because when you take things at face value, you make very often very wrong interpretations. What really helps is that they already familiar with this type of work, so it's not like in French it's like, oh my gosh, it's so hard for them. You know, if you make those connections and if you refer to what they already doing in another subject, then that's gonna help. Even to this day, there are such a resistance on the part of institutions to go that route, but it's really what students need. And when it comes to language, you know, in their English classes, because of this divide between language and literature, their literature-focused classes aren't doing justice to attending to language very often in the way that even native speakers need, and vice versa, in the foreign language classroom, it's not doing justice for them to really be able to get into meaning. I work in the middle school right now, and so from the sixth to eighth graders, and then yes, just like she said, we know ELA class, they do exactly the same thing. They highlight it or underline it and annotate it, or if you don't know the meaning of the word, just write it down on the margin and then later on look up the dictionary, sometimes the teacher's gonna be encouraged them to kind of think about from the context and then try to think about a meaning first, then later on go into the dictionary. That process, I don't think they have a problem with it. The only thing is when you explain to those students what we expect them to do, must have some more specific explanation with the examples with it and how to break it down in their level of understanding. In that case, I don't think any problem incorporate those ideas. So you know what would be interesting is if someone wanted to share, to go further with this and adapt it to your context in a high school context or middle school context or elementary school context and send it to us so that we could include it, right? Because what's wonderful is for us to be able to have a range of these kinds of materials to work at different levels of education. So yeah, if you wanna pick up the challenge, please do. So that was reading and actually I wanted to touch then quickly on writing. And again, when we talked about rubrics, the rubric that I give you here, what I always start first with peer editing. So students write their draft and the peer editing tasks, because it is task-based. When you, again, research shows that when you leave students to their own motivations, let's say, when it comes to peer editing, what do they do? What do they look at? Grammar, spelling, right? Punctuation, maybe. So, hm? Content, too? Yeah, sometimes content, but if you don't give them any prompts, inevitably they go to accuracy. Grammar, spelling, maybe punctuation. So I tried to build into not only the writing tasks, the literary gets tied into that in various ways, how they're working with language. And then those writing tasks, sorry, the peer editing tasks are getting them to work with language of the text holistically. So in the Hemingway writing task that I had set up, this was the scene of a crime in a play script format. Imagine a funnier, weird, failed crime, et cetera, right? So here are the peer editing tasks. One, play script conventions. Did your partner incorporate the following conventions? Maybe this was present tense use for this setting, the scene, conventions for turn taking in the dialogue, accurate punctuation throughout the dialogue. So they're really looking at the genre. Then, topic development. Does the content of the dialogue in any of the stage directions used allow you to fully visualize what is happening in the scene? Are there any points of confusion for you? Can you see ways of helping to clarify the sequence events or the relationships between ideas, the characters' reactions to the events as they unfold? The style. It's asked for funny or weird. So would you say that the scene depicts a funny or weird crime? What makes it funny or strange? Can you think of ways to heighten the effect? And then use of pronouns in articles. So there was this built into it also in terms of what is known, what is not known, et cetera. So students do their peer editing work. And again, I find that in the college level, they, boy, they really are able to, once they're given those specific tasks, they go quite far in giving each other feedback. They pick up on a lot of things in their partner's writing. Then they do their rewrites so that what they submit to me, ultimately, is their best work. So I only grade once and I use then the rubric. I take all of those tasks, the peer editing tasks converted into a rubric. So I'm grading them on the same thing, on the same work that their peers have been, that they've been working on with their peers. So rubrics, yeah, are good tools, but then what you include in your rubric is quite the question. And that's something that, yeah, depending on your particular needs. But for a flight lesson, it really needs to go further than, you know, just the more standard. You wanna get them really looking at language in various ways. And again, in my textbook, at the end of every chapter, you'll see the peer editing tasks and you can see the range of kinds of things that, depending on the genre and the dimension of the literary that I've included in that, that you can see the different tasks that I propose.