 Welcome to this webinar session promoted by Eden and the network of academics and professionals that I represent here today and welcome so to this webinar on critical thinking and technology. Let me first of all introduce myself. I am Antonina Poce. I think you know me, most of you know me. I am from Roma Trai University where I teach experimental pedagogy and my research interests are based focused on especially on the use of technology in education and in higher education in particular. Today we are going to talk about critical thinking and the possibility to use the enhancement of critical thinking with technology. We are going to have the most interactive webinar weekend because it's important I think to have a discussion on these sort of issues that are really central in the scientific debate scientific community debate. So first thing I would like to do with you today is to warm up this meeting with some questions. I would like to know from you from those of you who are attending this webinar where I can't see your faces but I know you are there. What is your opinion? What do you think critical thinking skills are and what is your opinion regarding the possibility they need actually to enhance critical thinking skills within those that are called 21st century skills. Can you tell me something? What is your your view before starting before I mean before entering the issue in detail. I know you can type on the chat and I also know that if you raise your hand in that little box on the top of your of your video of your screen you can ask for for speaking. So Alistair welcome Alistair. Alistair Krillman is with us and is typing source criticism is the most important skill in all education today. Thank you Alistair especially in this post through society in fact this is absolutely true we are experiencing a very difficult time because we have lots lots of information available thanks to technology and this is the link we can already state. One of the links we can already state but if we don't use our our critical thinking skills if we are not able to distinguish which are the information that are reliable and on which we can elaborate our thoughts and the ones instead that are not we can't be truly active citizens. We can't make decisions and participate in in in the active society. Another definition critical thinking is thinking. If thinking is not critical it is not thinking this is again another interesting point you're right thinking should be critical and we'll see in the presentation I'm going to give to you today that this is an important point because we need to nourish to feed our thinking. Information competence is vital likewise ICT competence to navigate and use tools to handle large amount of data of course this is true again because we need to develop different skills and actually the one you are mentioning is one of is part of the skills the European Commission recommend as a mandatory in this time we are living and so technological ICT skills should be managed by everyone otherwise you can't access and you know make up your opinion on on things that are happening of course yes yes Abba being outside your comfort zone your comfort zone as well of course okay there's some discussion going there on the chat and I'm very glad about that so I'm I'll go on with what I wanted to say which is actually linked to what came out of this first warming up session and critical thinking skills are essential so you know we can state that we can state that critical thinking skills are drivers for creativity and innovation we said we already said that there are social and cultural reasons that make I think of critical thinking skills as drivers of creativity and innovation society needs this western of course western and industrialized society we live in in order to grow and to develop need creativity and innovation needs change needs the development of new possibilities for especially for for young young people and just if you have the possibility of practice and of exercising of exercising your your critical thinking skills you can reach the enhancement also of creativity and innovation without innovation we don't have improvement in any in any area in any sector of our lives in in any area we are based there are some data here which I wanted you to to reflect upon because which in a certain sense justify the reason justify why the European Union I'm talking about the European Union but there are lots of documents that there are lots of scientific articles dealing with this issue and dealing with the need to be active to something in order to face the difficult situation we are living with though that the crisis was over maybe in some in some areas it is but it is not actually if we have a look at this data that I'm showing you here so on over a seven-year period from 2007 to 2013 you can see that the proportion of young people need the so-called needs neither in education employment or training within the 28 countries composing the European Union increased significance so compared to 2013 we have a 4.8 points higher proportion than in 2007 and another uh important issue that I'm stating here and which is really impressive is that from 2007 to 2013 employed school early leavers decreased and unemployed early leavers increased so hmm we have a mandatory issue here we need to reflect on these data as educators first of all we need to understand what to do in order to face the situation and invert such such trends um already in 2010 this report from OECD investing in human and social capital was stating that we needed and we need actually the complex communication we have a strong demand for non-routine analytical skills problem solving inquiry based learning all different ways of approaching education that could help the development of certain skills which are so demanded in present society so we we are living a very particular uh and particular time where uh there's a strong demand for education in a very wide sense and there are so many so many young people living formal education so so if we uh need if we want to reflect on these issues what do you think do you think that in such a situation I tried to describe that's the possibility uh for teaching certain skills skills and do you have any experience on on that can you tell us something about your your your experience I really hope that I can voice some of you I don't know if I'm not uh able to do that but maybe the secretariat if you raise your hand can help you speaking otherwise you can of course chat your chat there are people typing so I don't want to hello hello hi everyone I'm Francesca Grusti from our department from the laboratory for experimental research is speaking yeah glad of it please Francesca the word to you as you know for sure and we are really involved in the critical thinking problem you know when you have to to speak about you have to speak about educational curricula uh critical thinking is like the chaser cat because it is hind in all the disciplines but sometimes it not appears as fully formed in no one of these disciplines as soon as you you push to to see it focus in focus it slips away but I think that critical thinking can be encouraged in many different ways and critical thinking is an important issue in education and our experience is that using questioning techniques the content becomes more relevant for every child or student in the classroom and our experience is about pushing creativity into critical thinking learning environment and we had this experience in Sicily and we have the students writing a narrative story in group in an online environment and we had this very very particular experience and I feel confident to say that it was really really encouraging to use critical thinking into a formal curricula educational curricula and I'm strongly believe that it needs to be integrated in every level at every level of classroom from key 12 and to university academies and whatever. Thank you. Thank you Francesco. I am seeing Alesser typed his comment and actually I would like to know from you Alesser because Alesser is typing about the need to integrate in every class these activities related to critical thinking development and he was suggesting to get the students involved in choosing course materials and this is a very interesting point Alesser can you can you argument more on that do you have any experience regarding this possibility to involve to involve students in in choosing their their materials the studying materials I think it's very very interesting issue to involve students yeah yeah are you there I think I think you can hear me now so yeah yeah yeah I can hear you very well I think it's I think it's a we've tried it it's not it's not easy to get them to do that I think they have to they have to learn they have to be given sort of templates and rubrics to be able to choose they need to practice quite a lot in assessing their credibility of the sources they find but I think we were not really doing them a favor by choosing everything for them and the more they're involved in I mean this is I mean all of you know I mean we're all working with a we are and with with student generated material that the more they're involved in producing course material the more they have to check sources the more they have to be involved in that I did read an article and I can't remember where it was but there was one article about some teachers who were who gave them they gave them texts that were full of false information yeah and their job was find the faults identify what was wrong there were false quotations there were false references there were factual fails failure factual sort of mistakes and they were they had the task of actually tracking it all down so they really had to dig into the the literature and look around and find other facts and they had to basically it was like marking a student paper that was full of mistakes it's one way of doing it see what other people think yeah yeah yeah that's that's a very very interesting experience and actually I you will see we'll be discussing this in in short time this idea of assessing credibility of the sources is one of the point that I consider most important because if you involve the students in choosing the sources but you make them understand that they have to think critically about the sources they are choosing according of course to certain indicators they really manage to acquire a very very important skill for their lifetime one according to which whatever they read should be analyzed and evaluated and put under discussion so this is something that they really need to learn so this sort of practice would be absolutely useful absolutely useful and if I go on trying to focus on what we we are discussing today and I need to make a link a connection to the labor market because as I was saying yes we have a strong need of certain 21st century skills and we are in such a huge need for education because of the crisis of the economic crisis of unemployment of what we were saying a minute ago but we are also in a paradoxical situation because we have this large demand of education but educational institutions are facing all over Europe I would say all over the world a big problem that is the one of seeing their resources reduced so they need also to look for resources in order to be able to after what people are asking actually and there are some countries where this kind of trend is being reduced and there are countries where there are policies addressed to improve equity in learning opportunities I'm referring to Belgium France and Germany and if you have a look at this article by this colleague Chien-Kueh dated a very recent article is issued this year you can have a look on very specific data regarding investment and the policies related to this this kind of skills development and I will give you some some references of course in the end if you are interested there are different links where you can find open resources on these issues and what I really would like to suggest is to have a look also to this paper by Laurie Lard I think you all know this scholar was been working a lot on technology and the use of technology in higher education she always and she frequently speaks of learning organizations and places where the organization itself should learn in a virtuous cycle from its possibilities from its quality indicators in order to improve its offer and in order to meet what the demand in education is so this is another point we should really think about because sometime we don't think that bottom-up activities and evaluation system evaluation activities could be helpful for the whole system and could give a very important benefit to the ones who are demanding education demanding innovation in education so this is another point I would like to discuss with you the other point is that we are asked to enter also the work in places if it's the the labor market asking for certain skill developments for certain knowledge development for specific knowledge development we should make this interaction more effective so formal education should enter the work in places and the other way round we should involve stakeholders more in our curriculum development and should ask what they think are the skills and the way we should enhance certain skills within our learning environment so that virtues cycle that I was mentioning before should be developed together with all the subjects all the actors that are involved in in this process this is a view we should think upon so according to the inputs I tried to suggest in this second block of slides can you again give me your point of view do you think do you believe that every organization every institution in which we are involved could be could become a truly learning organization and what do you think of the role again that technology in education could play in this and in this activity in this change that we could see really see in the way institutions could could be the role they could have some impressions from you and of course if you have idea of initiatives projects regarding critical thinking development critical thinking development in I would say dual world in a world west where the working the labor market is involved in higher education teaching and learning I'm referring to higher education in particular because it's the level of instruction where actually these sort of skills should be developed most even if of course also at primary level it would be important to have this kind of enhancement and development also is time students writing Wikipedia articles excellent way of real life practice this is again an interesting point because there they can be active subjects producing the resources we were talking about a minute ago and this is another way to enhance other skills such they like the writing skills which are very effective skills closely linked to critical thinking skills that's that's for sure and that's also Maria Rosario Re from Roma Tri University typing and Diana typing too waiting to see yes we are doing some activities as international virtual mobilities we are that's very interesting I wish you could tell us more about the virtual mobility actually we have also oh thank you Alistair Alistair typed the link where we can see the Wikipedia education program wow that's very very interesting I think I'll use it with my students so yes Diana that's very interesting the virtual mobility programs we have a pro a european project we are participating in a european project dealing with dual learning and virtual mobility in particular and we actually we are in the planning phase for for the piloting that we are going to have in in the spring and I hope we will have the possibility to have a webinar on virtual mobility that would be very very interesting and and as I see from from the chat different people are working on the same theme so it would be interesting to have a new webinar on virtual mobility and discuss the experiences together I have here another another comment so different experiences in these kind of exchanges of virtual exchanges Romania and USA that's very very interesting yes if you have links or or two links to produce we would be very much interested in seeing other other links very very glad about that I hope that we can also store this chat so that also for people who are not able to attend this webinar today can later on access the information and links okay Diana they work virtually in teams to produce multimedia artifacts we run this in every year since 2008 this year we focus on augmented reality projects great it would be again also Don, Don Alcott is commenting and saying we would be very interested in learning more about the US-Romania project I live in Krakow and teach in the US so yes I think that we should support this this idea of having another webinar on virtual mobility which is really I think a new team that we need and we wish it could be practice and evaluated more another link is coming because we need to use technology in a critical way and so using technology for mobility saving all the difficulties that traveling implies is really a successful way of using technology so if I go on great Alessar we have a webinar on virtual mobility back in September see the recording great so we can have a look at that as well very very interesting and useful for all the projects that are going on among us going on on what we were saying I'm going now to tell you about a research a research project we have been developing in our group in Roma University and at the Department of Education in particular we have been studying critical thinking abilities for some years now and we have been trying to study critical thinking abilities linked to the development of specific models specific teaching and learning models what we think is that there are some some issues some categories some some structures that need to be proposed that need to be designed carefully in order to be successful both in teaching both in getting good teaching and learning results but also in critical thinking development we were mentioning this issue at the beginning of our of our meeting we need to think if we think we need to think I would say properly but that is a wrong word if we want to develop our thinking we need to fit our thinking and what Paul and Alder state here I think is important because they reflect on the quality of thinking excellence of thoughts must be systematically and culturally nourished fat so what we need in order to have a high quality of thinking is culture that's why we have the reason why I think we have all these post-truth problems on the net is the lack of strong cultural basis so that's why we need to have this cultural issue really present in our curriculum development and in the educational proposals that we offer here you have here you have a summary of what we have been trying to do in our teaching and learning projects for some years now we try to focus on the teaching materials we provide online and we try to give these teaching and learning materials a strong cultural basis we organize guided discussions on dedicated virtual spaces where students work we try to analyze what they produce online their elaboration online regarding the teaching and learning materials we offer we have been analyzing these materials and we will see how through content analysis models but also through lexical metric tools we have been basing these analysis on an adapted critical thinking skills analysis model based on human web and cochlear model we have been making the students working on short essay test productions we have been trying to repeat this test course at the beginning and in the end of the teaching and learning activity in order to see if what they did could be measured here you have a synthesis diagram of what I was telling you about so pre and post test based on essay writing short essay writing online the activity online was based on cooperative writing and then evaluation of the activity their production students production was analyzed through content analysis as I was saying and lexical metric analysis then we had also other kind of analysis we carried out through online interviews and questionnaires but also their questionnaires and that's what I will be talking to you in a minute where structured according to this critical thinking skills model the model is really summarized synthesized here is made of 20 pairs of indicators and the assessment model is based as you can see in this formula on the presence or absence and the ratio is based on the presence or absence of the indicators in the written production made by the students the indicators are as you can see your 20 there are 20 pairs of indicators we tried to focus on the ones we considered most important to identify critical thinking levels relevance importance with understanding justification novelty and so on and in the data I'm going to present to you today we are focusing on what students from this course the course of primary education school teachers a course where more than 200 students participated I'm going to present you data regarding what they thought about the course they attended and in particular about what they thought regarding the open resources we put at their disposal during during the course in fact they had this online lecturing as part of the compulsory moderating educational research methodology and after taking part in the course they had to carry out this evaluation section and they had to fill in actually this evaluation form and this is linked to what we were talking about before regarding the need to reflect and to evaluate the resources they students have at their disposal either they they produce them or they get them from the teacher the lecturer the person who is in charge of their of their teaching and that's why we developed this tool which we consider innovative and you might you might say why do you consider innovative this evaluation tool first of all because it is based on the critical thinking model I was telling you about it's divided in four sessions and each one related to some indicators present in the model I was telling you about relevance and importance contains indicators linked to the formal characteristics of course content the second section regarding breadth of understanding augmentation and justification where we have a series of statements and students should indicate how far they agree with them critical evaluation third section where students are asked to critically assess content proposing issues for discussion and reflection the novelty section is an open-ended session and here the aim is to enable students to reflect and express their own opinion evaluation adding elements on already identified indicators I'm not going to show you you know all the results regarding the all four sessions I'm just giving you some some results regarding the first three sections that were the more quantitative sections where students had to to make their statement just and you know indicating one or the other position in in in the scale here you can see that they gave positive replies to what we were asking to reflect upon regarding comprehension memorization clarity of content and structure clarity of language and so on and so forth you can read of course the indicators what I want you to to to think about is the organization of the questions that were directly linked to the areas relevance and importance here with of comprehension augmentation justification solutions here regarding of course the content of the teaching and learning materials they had for their course critical evaluation so in this session we were asking the students to express their idea on on the course content to see if they agree or disagree regarding the possibility of enhancing their creativity innovation communication critical thinking proper solving memorization so on and so forth skills so they had to stop and reflect on what they learned on what they had available for their study and on the activities also they had to carry on to carry out online and to see if actually that activity was helpful regarding this kind of skills of course this model is still under investigation of course we were focusing on the relation between their ability the students ability of reasoning and a specific teaching and learning model we were offering them so it's difficult to make generalizations and we were basing our our study on again a specific hypothesis that linked a strong culturally based proposal students in that specific section were starting loose soul so we're starting a classical text and again you know it's very very specific we focus on technology has a facilitator we were thinking of technology as a facilitator of a certain specific model of instruction so again it can be a limitation and so it's difficult to say that even if we had a positive reaction this reaction can be considered positive and generalized what we we are doing actually is trying to promote a critical use of technology what we are doing is to replicate the use of such a model in a diachronic way on different years so in a longitudinal kind of study in order to see what we got from those analyses can be replicated and considered actually positive because data you know are stable in time so this was our experience this is our research this is the research we are trying to carry out and to promote and again because we have just 10 minutes left in this webinar of hours today before wishing you a good season break I wish we can have some more and reactions from you so I wish I could get your impressions on the experience I presented you and also to clarify some points if we have the time to otherwise you know where you can find me and if you have similar similar experience to discuss to present and and if you also can give us some suggestions on how to improve our research and our results please express yourself any suggestions done is typing now I know that these and if you raise your hands maybe you can also speak also I advise I'm sorry we need to give them need people to write so we need some time but I'm glad that we already some good suggestions for next uh next webinars this idea on the virtual mobility development I think it's very very interesting and as you can see is already on in different projects in different projects that we as a community have are developing in different settings so this is very important I can't see what Don and and Diana were typing I don't know if the experience I presented was sufficiently clear anyway I will give you some some references where you can find articles we actually we um published different articles on the experience we had and uh part of these results I presented today were actually uh presented at the Oxford the Oxford um conference the Eden research conference uh uh held in Oxford and the the article was published on and is published actually on uh urodil so you can get and uh access to it I think easily uh another another article uh related um related on what we were talking about today is available on academia so on the platform where you have open resources so if you access that you can you can download it I made it open so you can get it uh I don't see any questions or comments so um yes Alistair is saying that uh the webinar uh in you mentioned before can be can be downloaded so again that is uh um that is again an interesting uh source we can we can get we have five minutes to go but I can also these five minutes because I can see any questions or uh um comments I can um maybe clarify something regarding uh the lecturing uh the students had to attend online as I was telling you uh we had we have these very large uh we have large classes in our courses more than 200 students and these students have to study for their qualification as primary school teachers during this course of course they they they need to learn about educational research methodology the structures and issues related to methodology for research but they also have to attend part of this lecturing on um some of the tests taken by uh Russo work uh on Russo work and uh in uh Russo work uh they have to um to read original uh original part of his text uh the text we we identified uh was a meal and on this text they had to reflect on they had to make written comments on the um the passages we gave them for for reflection uh Emil or the education uh there uh we um of course collected their texts but not only actually they had to reflect on uh different inputs we gave them on the web on the platform where they had to uh work according to the lexio magistralis model according to the tutorial model what is uh uh uh still on in certain very high quality universities like Oxford and Cambridge where very small classes are held where students have to read original passages from very classical um authors and they are addressed to uh to reflect to to to write to uh produce their own their own um critical um comment uh so that's why I was saying that they had to work and to comment with our evaluation tool on a very structured um proposal uh very um on a proposal that in time in a long time proved to be successful and that's why I was telling you maybe this tool that we developed was to focus on a very specific uh teaching and learning proposal maybe on different kind of resources that uh tool uh could not prove to be so uh successful and effective this was the last thing I wanted to to tell you um I don't know if the secretariat can tell me uh uh something regarding the the the close of our meeting today we reached uh 4 30 so um the time that we were um entitled to to use I hope it was uh uh helpful at least it was the the occasion to reflect on our topic today I leave you with my email address so that if you want to make comments if you want to ask questions if you want more materials on what I presented today you can get it very easily um and I leave you also my uh twitter address so that you can also contact me there and also you can attend our Eden uh chance thank you thank you for being there and have a nice uh season break bye bye