 Okay, everything is good. Good morning everyone. I'm Adrian Solomona. I'm the deputy director of the Southeast European Research Center and I would like to welcome you to the first parallel session of today's feed for RRI online conference. It's a session on paving the way to institutional change via quadruple helix RRI embedment methodology. Just to give you an overview of what we will go through this hour, we will have a very brief introduction done by me on the feed for RRI co-creation methodology followed by actual showcases of applied RRI experiments. A bit of discussion on some comparative observation of the experiments that we had in feed for RRI and then at the end of the session we will open the floor for questions and answers from everyone. So if possible try to pin down all the questions, all the idea, all the discussions that you may have throughout all these presentations that will be done by various people as you will see now. Now because we talk about the experiment and institutional change and all these things I would like to just briefly provide an overview on what do we mean by experiment in feed for RRI and specifically for this feed for RRI experiment as you can see here on the slide is the actual action of engaging quadruple helix stakeholders in an observation and co-creation in an ongoing research towards identifying ways that this quadruple helix co-creation would enable RRI and open science embedment in the ongoing research and of course in the organizations that take part in this project. So basically this is in a nutshell the idea of the experiment that we had in feed for RRI and the goals basically of these experiments as you may have seen if you took part in the previous days by just as a reminder was primarily to foster the RRI and open science adoptions by institutions to build up skills that will facilitate the better implementation of RRI and open science and ultimately of course the end goal is to actually trigger and foster institutional change based on RRI and open science and all this was done by following a structured approach a unified methodology let's call it that you can see here on the slide and all the four experiments that have been organized in feed for RRI followed this approach in order to make sure that we can have comparable results and results that can actually lead to practical and policy implementations. So this specific methodology started with an appraisal stage in the sense that each experiment organizer was asked to map its internal and external stakeholders quadruple helix stakeholders for the external part and to identify for example their interests motivations expected outcomes from adopting RRI and open science and of course to select the most important RRI pillars to focus on as each experiment was usually selecting two or three RRI pillars from the pipe six that are available together with the open science concept in order to ensure that the experiments will have an applied and more tailored let's say approach rather than adopting all the actual pillars but as you will see in the comparative analysis side overall all the RRI pillars have been adopted throughout these experiments. Afterwards a design stage followed when together with the internal and external stakeholders the experiment organizers jointly developed the objectives or reassessed the objectives of the ongoing or the new research projects that they were doing the experimentation upon in order to facilitate this quadruple helix collaboration. Afterwards the implementation stage took place meaning the actual performing the actual research related to the research project or the initiative that you will see slowly in the case studies that we have displayed in the sense that they re-managed and they reassessed the way that research is being done in order to do it jointly with the internal and external stakeholders that were identified and of course afterwards after the implementation phase a follow-up phase was done in order to wrap up the lessons learned what can be done next what what measures can be taken in order to institutionalize the best practices. Throughout all these experimentation of course in order to facilitate learning across the experiments a mutual learning stage was done through which all the partner and all the experiment organizers were actually exchanging best practices and at the end we utilized specialized key performance indicators in order to assess the success and of course the effectiveness of each experiment and some of these indicators that you can see briefly here the quantitative examples for example the representation of the entire quadruple helix ecosystem interest in RRI of each internal and external actor at the beginning and at the end of the experiments in order to see if their interest was raised similarly for the awareness and the perceived usefulness of RRI. To understand it actually these experiments produced a positive impact in the desire of these actors to adopt RRI and open science and more quantitative ones such as the actual numbers of stakeholders involved pillars the quadruple helix consensus solutions and so on and so forth that you will see more in an applied manner from the three exemplification of the experiment. And of course we had also qualitative indicators in the sense that we try to observe in these efforts to implement RRI and open science in ongoing or new research projects what policy change recommendations appeared what organizational change recommendations are required in order to properly foster RRI and open science. If we receive any collaboration proposal from outside from interested parties yes yes we really enjoyed it we want to be involved in more RRI related initiatives and of course especially we paid a very big attention to proposals received from outside society when it comes to collaboration with researchers. And of course we looked into how this best practice modification of RRI and open science adoption has been done at the entire quadruple helix ecosystem. So in a nutshell this is the approach that we took in order to facilitate the organization of these experiments and now without any further due I would like to invite the first experiment organizer in order to in order to share with us and I'm sharing again the order of the presentations in order to share with us up to 10 minutes presentation along their experience and their outcomes from their experience. Rakel I will stop sharing now if possible please to share your screen. Yes hello everyone I will now start to share my screen you see yes yes okay I will talk about experiment one yes I will start by telling about who we are what our goals in the fit for RRI experiment were our main challenges lessons learned good practices and plans to further embed RRI in our work. First of all who are we we are a private company in Portugal big company with a lot of accredited laboratories and working around the world this to give a better idea of who we are. So the original goals of our experience were to anchor the experiment in a co-experience model to an ongoing project in the energy field which is a field in which we are very active in research and development of innovation and to also test previous outputs of the project on governance settings and sectoral viability with respect to RRI. The project that we anchored the experiment at the beginning was this project Moebius which was about modeling optimization of energy efficiency in buildings for urban sustainability but along the way we decided to have a new goal which was to build our own ISQ RRI model and what was and how why well I will tell you about how it was well the main challenges right at the start were that when fit for our project began RRI was a concept that was completely unknown to ISQ so we are a big company we are very active in the research and development of innovation very active in several R&D projects European projects in several different areas we are a house of engineering but we never had heard about RRI before so of course this was a big challenge from the start so along with it came the learning step of course because for us to implement an experiment on RRI we had to know what RRI was so the second main challenge that I can point is the learning step which was a very very heavy step but very very interesting so to understand what lies beneath the concept of RRI and open science the third main challenge for us was to get all ISQ researchers on board we are a big company with about 1400 employees but from them from this number only we are only about 50 researchers but we wanted to get everyone on board so this was again our this is also one of the main challenges for us so the first thing we did in our experiment was to start involving the people so after our first initial diagnosis then with some key people in the R&D activity in ISQ we started promoting these these events internal events for us to start reflecting together all the team of researchers reflecting together about what we are doing what is the the impact of the research that we develop so not mentioning right at the first the term RRI and then at some point of the events we would then summarize what the concept is about in 10 minutes and that was one of the challenges as well and then after presenting the concept of RRI we would start brainstorming and talking and reflecting together about what we are doing and how it was of course we organized after this first joint reflection sessions we started a training a training program let's say our training actions and for that we counted on some experts that were also involved in the projects for instance as University of Mino they are experts in open science so we promoted several training actions about what is this of the components what is this of science education what is this open open science open access so several events were promoted then and again we tried to involve all researchers and that's a big challenge meanwhile during the project duration a new ISQ R&D policy department was created because we had these units individual units of research that were spread around the company working together with the operating areas but then we decided that it would be better so the company decided that it would be better to put every researcher together so the research department was created during the experiment and one of the the advantages we had was that well the new director of the department used to be one of the people involved in the Moebius project so the one from the beginning of the project so he was more or less aware and after participating in all in all the sessions that we organized he was quite aware of what we were doing and what this concept was about and so he was very keen of embedding RRI in the new department division vision and mission and this was of course an advantage for the success of our experience what did we learn with this very briefly so we learned that the joint session with joint reflection sessions were key to get everyone get everyone involved these these are really in interest important because if we want to implement an institutional change of course everyone needs to be on board especially everyone when we are talking about with responsible research and innovation everyone that deals with research and innovation so we we we think that this was key to promote this joint reflection sessions with everyone then another lesson learned is that researchers during these sessions researches were faced with issues and ideas that are not usually on the table in their professional lives and work routines and again I want to stress that we are a private company so yes everyone in other companies or in other type of organizations like universities and I know the majority of the audience is coming from the academic part of the quadruple ux but yes people don't talk that much to each other we we are always so busy in our business in our activities that we made most of the times or sometimes well in our case a lot of the times people didn't talk to each other so there was no space for this reflection and discussions and what are we doing where are we going this kind of philosophical discussions and this was an opportunity for people to to talk and exchange ideas about what they were doing inside the same company and inside the same activity of the company so yes and when faced with some of the questions that we launched for discussion of course the researchers were faced with some issues that they don't think about in their normal day yes the tailor-made training sessions constituted an important awareness raising action and being a company and having all the resources that yesterday we talked a lot we talked a lot about from the foster portal and from the fit for our iPod together well with all the resources being a company we felt the need to tailor our sessions and tailor our contents in most of the cases because we found that some some of the resources that are available are very tailored for an academic or an university context sorry yes going and another lesson learned is that RRI is a flexible model not one size fits all and this is very very important if we are talking about implementing RRI in a company at least because for our for our right to be implemented in a company it has to be to have proven benefits well good practices and I know that my time is almost over good practices so this the selection of the components that we would give priority to was done based on the concerns that were the concern of the researchers so the concerns that we heard that the researchers had while promoting these joint reflexion sessions also yes that yeah thermal tailor-made training sessions on the selected components to accommodate the organization's need this is very important and also to plant the seed because yes we really think this is about planting the seed there is no immediate change that we can see in mindsets and everything so this is this has to be a long-term work yes and plans for us to embed RRI in our work so we are building our own RRI implementation roadmap we want to engage more with the society because given the turn of the experiment we ended up not having so much time to do everything we wanted so yeah the society part of the quadruple elix we didn't get there that much yes to set and measure IQ RRI model implementation indicators maybe also aligned with the more more he indicators continue this RRI training make it in couple sorry for each new researcher and to spread the words thank you thank you thank you very much Raquel for this excellent presentation I believe it was very spot-on and it really shows the substance that the fit for RRI experiments have achieved and this is a true impact case study so congratulations for this now since you have to leave I opened the floor for questions now I just I'm breaking the rules for just for you and I will just a five-minute window for questions I have already received in private a message with the question that I will ask now but in case anyone else is can think of any questions just after Raquel answers this first question please feel free to type it in the chat or to ask now just the first question and then we go to Giovanni now the first question since you are of course your business of course like okay I know you don't like to call yourself industry but you are business see any clear link between RRI and return on investment or how does it support the organization on the long term let's say with any return on investment strategy well looking at from a business perspective of course the anticipation part or dimension of RRI and the meeting the expectations of society of course this is if we look at from a business perspective this is working for the market so of course I think I see in my perspective this this is perfectly aligned with the doing what the business wants to do so yeah I don't I see I see it as an advantage of course it takes a lot of work in the time to to put this mindset because of course we want to meet the expectations of society and not harm society and not harm our world right and sometimes the expectations of society are good products that the market wants are not exactly good for the society itself but yeah I think it's perfectly possible to align both excellent super thank you now I think Giovanni wanted to say something I'm not sure is this in the case Giovanni, yeah, please unmute thank you very much for a very interesting presentation I would like you if possible to articulate spell out a little more what can count as a proven advantage of RRI for a company so it's on the same line of Adrian's previous question so what are the things that are perceived as a meaningful advantage for a company I would say that it's something that is good for business putting it in a simple in simple words of course and what is good for business it's good for business that researchers feel that they are doing something meaningful for instance and of course this goes in line with RRI if we think of it in all the depthness of each component and the whole the whole dimension of the concept I think and also as I said to Adrian as I told to Adrian seeing seeing that the innovation that we work for in fact are expected from my society so by the market and this of course is good for business and yeah I have been working with some other organizations because I was I really I'm a converted person you know yesterday we were talking about the converted ones I'm a converted one of course so I have been spreading the word in other companies from into the engineering field and also in other other universities and sometimes companies see it as a marketing advantage but I think this is the first the first view before getting in depth about on what RRI is about and what are the dimensions of RRI I think I don't know if I answered your question okay thank you very much now there's a question from Gordon please go ahead my name is Gordon Doulton I'm the coordinator of the Ring and Grip projects the Ring project is a global project in RRI and we had a work package that is dedicated to the competitive advantage of adopting RRI and there was a global survey conducted with interviews and the deliverable is now available under work package five under the Ring project and we're both the financial the branding and the marketing value of adopting RRI mostly to industry is being explored and second the grip project then is like people RRI it's going to implement five case studies in the Ring and Marathon where we're creating action plans for five case studies but we're we like to think that we're going you know sort of all the way where we're creating we're implementing change management and a very large number of interventions so of course training is one intervention but successful changing of strategies and change management's permanent strategies then is how we're going to measure our benchmark on how successful we are so I was interested to hear that you didn't succeed in engaging with quadruple helix and I'm wondering how successful did you get top management buy-in or institutional change well thank you for your question and for your hints I got very interested in your hints so it was very useful thank you I will check this specially Ring project available from work package five it will be useful for me and for the work I mean I'm doing with other other companies and the other organizations that are starting to work with RRI your second question as to your second question well how can I say it there's a momentum I think sometimes for change to happen there's a momentum and there was a good momentum during the Fit for RRI project for our change for our institutional change which was the momentum of creating a new department and aggregating all research activity that was that being done in the company for years of course so that momentum was a good momentum that facilitated to get the top level management top management level support but of course there's the momentum and I have to admit that with this COVID situation I believe that it got a little bit suspended and I hope that it's something that has to do with this complex situation that we are living I haven't been in the office since the 11th of March although some colleagues are but I haven't and I'm for the strategy that we planned and that we designed it's not exactly the same to be together with everyone and and of course we are dealing with exceptional situations so I hope that once everything goes back to normal we can continue and still count on top level top management level support I believe we will but I'm not sure okay thank you very much Raquel this was very very informative now just to be cautious with the time I would like to proceed so thank you very much any further question of course we can take by email in the chat and we can facilitate the answer later on I would like to invite Andrea Ricci. Hi good morning to everybody we start sharing my screen okay it should be in presentation mood so I try to be quick to save some time for the questions. Here it is what we did in Sapienza for our co-operation experiment. First of all our challenge we aimed at establishing and which were our haze the idea was it implementing through an ongoing Sapienza project named Sabirin Co a responsible governance framework capable of strengthening the university capacity for generating social impact and value and of course we want to convey some specific message to this experiment among the sustainability because our main assumption that we are trying keeping nowadays also in our policies at Sapienza is that responsibility should be seen as a proper and effective governance dimension to be considered in the management organization together with the most traditional one that are mainly efficiency, efficacy and cost effectiveness. So more concretely which were our objectives first of all as we are a very complex organization like 100 thousands more than 100,000 students and 4,000 researchers and so forth to test in a pilot scale a multi-actor co-creating the responsible governance in a complex environment in order to like see in a small context limits and opportunities for a big organization at Sapienza. To involve different stakeholders also based on a quadruple ethics approach in an open debate about responsible research and open science focusing also on sustainability circular economy and biomaterials with the attempt of shaping a common agenda to engage families and children in order to support them in experience through hands-on experiments the role of sustainability in their daily lives. As a result we select some pillars to run the experiments as you can easily imagine the main pillar of Bata Rai was governance because we were working in implementing a governance framework but we also consider two other pillars that are those of science education and public engagement. So mixing the idea of having like an internal governance framework even if supported by stakeholders together with a completely external dimension of science education public engagement towards our community. Well let's start seeing how we worked for the governance perspective which was the most complex one for sure. First of all we put together about 15 internal and external stakeholders so represent them from the industry world from policymaking from civil society together with our researcher and we ask them to work in a co-creation according to a co-creation approach to design the elements of a desired at least desired RRI based research center and we use the theory of change by current wise so we ask them to work according a logic framework with inputs activities and as I don't know who was there yesterday for the keynotes trying to differentiate as cannot say in the complicated relationship between outputs so the immediate results of erection then outcomes had impacted and really measured the real change you can have in your organization. So the results were encouraging even if not absolutely clear. By the way we were able to like come out with some outcomes that were for sure that the setup of a responsible governance in research funding and performing organization needs that scientific matters are informed by RRI for example touching teams such as research integrity, gender balance, open science and open access. In addition it was important that all the stakeholders were sure that responsible governance in research organization is by definition open to society and to external stakeholders in a logic of continuous feedback. Another important thing was the need of establishment of a common language platform and procedures because well they perceived the between for example the public and the private world for the private world but also between hard science and soft times some differences in languages because some sometimes we also dress it as somehow implicit dimension of RRI especially in the teams of art sciences. And again the most important things which is sometimes which is somehow linked to the previous one is that to implement real institutional change there was the need to bridge profit and profit research and innovation actions because that usually characterize some matters and in addition responsible governance should be based on a shared set of values that should have been reflected and we're working on that on guidelines with proper indicators and in the stressing the social value of research and innovation. So based on these feedbacks some months later in 2019 we tried to formalize our governance perspective in order to make more feasible a concrete implementation of our responsible governance and again we had to choose our work to work because the were the same people that we had in the previous meeting so internal and external stakeholder and in this case we decide to work through the balance scorecard so trying to define main objectives and then some objectives indicators induction with related targets aimed at focusing on the implementation of our responsible governance according to form in dimension the financial dimension which is also I would say resources dimension so also in terms of human capital or infrastructure that you have then the user's perspective so the external perspective but also then focusing on the internal processes so how we can change our internal processes and then the perspective of growth and innovation how can we improve our work and how can we innovate through responsibility the formal results was this balance scorecard with a general objective which is implement our right based research center and the sub-objective for each of the four dimensions cited before and this was further implemented we will not go in detail on each of these tables then you will see them but you can of course have access to the slide through tables based on indicators target induction for each dimension just to see an example here in terms of like financial perspective we say realize new investment on responsible and sustainable action and this was like linked with three indicators with a proper target and with a set of related actions as well so I will go quickly on this then now we can move to the education and engagement perspective that took place in May 2019 in order to cope with the idea of working on sense and vision and public engagement within this research center named Sapienko we organized two hands-on workshops for kids and their family and these workshops were linked with other public initial engagement initiatives already in place in Sapienza that deals with the opening of our like University Museum during the month of May the two workshops were dedicated to sustainability and more in detail to bioplastics preparation with the ingredients so for example organic waste or coconut flour and to the creation of super heroes but with 3D printers using organic materials what we wanted to achieve also apart from giving the idea that sustainability it's in our daily life was the idea that science education and public engagement represents a responsible action itself that show our national aspects are connected to all the right pillars and you have like some pictures that's the bioplastics and the kids preparing bioplastics and these are our SAPI heroes and the kids working on this as far as it concerns our quadruplex approach we can say that cooperation amongst the holders was really favored by the experiments venue because Sapienko was funded to be an interdisciplinary with the after-hub devoted to procreation and prospectivization by the way there is a common difficulty that is grounded in the academia in establishing a quadruplex approach because somehow the university system at least in our national context is still reluctant to some forms of prospectivization when there is not a proper national strategy or an incident structure to foster and favor it institutional change while we use a teaser which is not exactly the one that we are talking about so our arrival in science but we use termination which is somehow a trending topic now in university and that means the capability of the academy of implementing trusted mutual and socially fruitful science and societal relation and in this way by using like the anchor of third mission we were able to start a process laid the groundwork for some degree of institutional change but the real outcome that we are trying to achieve is make researchers aware that responsibility and openness are not administrative issues to talk about but they represent another value in their daily research work. Well for doing so this some policy support is required at least internally and we started working on this creating a working group both with research and administrative staff in order to work together in the definition of indicators for the third mission and we also launched a successful internal portfolio funding for implementing science and societal relation and well we had some slowing due to COVID-19 but we have like more than 20 projects approved and we are like going to see how they will work in this. Next goal, keep working on responsible governance keeping this collaborative approach as well implement engagement and science and educational activities within Sapergrinco and even within the entire university on a regular base trying to involve not only civil society but also other political actors and trying to connect these experiments with long-term initiatives. In this case we are working creating a connection what we did with series project which was the one funded under the European University's call Erasmus Call where we're establishing open labs for cooperation. That's it. Thank you. Thank you very much. This was very very good. As I wrote down in the chat we will keep the Q&A session at the end just to better manage the time therefore if write down on you know digital your questions just not forget them and I would like to invite Nancy if possible to share her experiment. Hello, good morning. Can you hear me? Yeah. I'm going to start setting my monitor now. Does this work alright? Personally I cannot see but I'm not sure if... It works Nancy. Okay. So I am here to present the use case from the Open University and what we did at the Open University is that we took a little bit of different direction so we did not plan a use case study which was related to responsible research and innovation and open science and the application of it but our topic was a little bit different. It talked about text and data mining of big scholarly data but we tried to do the whole process in an RRI way and this was the first time for us and we learned a lot of new information with the guidance of the project partners and we also have drawn very many important and useful conclusions that we plan to apply in any future project that we are going to run. So in reality the problem that we were trying to solve is the fact that right now there are a lot of outputs out there that students and researchers are able to access via the university subscriptions but this access is only for people's eyes to read and it's not for machine to machine communication and the latter point is the one that prohibits text and data mining. So we wanted to see how we could text and data mine content and it's not only open access because right now there is a large corpus of open access content available on the internet but at the same time we wanted to take advantage of a recent UK change in the copyright law which enables people to perform text and data mining for non-commercial purposes so we wanted to see how this could be made possible via this research study. We gave our research project a name and we called the CDUTDM because we got inspired from edurom and I suppose that the vast majority of you know that edurom is just an internet connection where someone who is affiliated with the university no matter where they go whether their affiliation is in the UK but they visit another university in Italy, Spain or some other place they can get access to the internet so that way we envisioned that edutdm would work for a machine to machine access for content that it's not only open access but via different work it would be made available to those who want to perform text and data mining. This was a very difficult task to begin with and we did not know the outcome of these results and therefore what we wanted to do is that we wanted to create a group to discuss the idea around it and develop a conceptual solution where this working group that we created would agree upon. So we wanted to have a lot of stakeholders from various places understanding the main concept of this idea and then seeing how this idea could be made possible. The outcome of this is a white paper the link of which is included in this slide and one of the problems that we met and you will also see that of the white paper is that we did not cover some ideas that were difficult to solve and discuss during this short period that this case study was about to like for the timeline of this study therefore we did not intentionally cover the ownership the development and the management of the tool. We just wanted to see in general as an idea what kind of attention this is going to receive how people are going to see it whether they're going to have a positive or a negative response and we wanted just to bring more attention to the matter rather than discuss important things such as ownership and development management but those are questions that we could not solve at the moment and require further discussions. So to get a little bit more in detail we created a working group which had members from the vast from wide variety of stakeholders and examples are the public systems for example experts on text and data mining, policy makers, other organizations that were in position to create recommendations and spread good practices and the industry. Of course we understand that even though we wanted to include the many stakeholders in the group we left others outside. The reason that we did that it's not because we did not foresee that their contribution was needed but again this was a limitation in the study because we wanted to do this we had to do this within a time frame and we had to be reasonable and pragmatic as to what we could end up having in the end. Therefore for example we did not involve the plain users or we did not involve any researchers from a UK academic institution and we don't involve any researchers from a non-UK academic institution so that we know approximately what is this that may be common or not or uncommon between their responses and way of thinking and needs but nonetheless we tried to be to have a wide variety of experts for example in the publisher systems to see whether there are changes between those publishers who offer closed access articles only or hybrid publishers and open access publishers for example. So what we did with this working group is that we had six sessions and then we had created a mailing list where we exchanged also a series of emails. So the white paper was the outcome of these six sessions and the email exchanging. Everyone had to agree on everything that we concluded and we wrote down at the white paper in the end but generally speaking we focused most on the technical challenges and as I said earlier not so much on the organizational challenges so for example what it was the conclusion of our investigation was with regards to the technical challenges but there is a lack of the common standards or adoption of standards to make this idea happen and that publishers use a variety of formats and of course there are issues with regards to the systems interoperability used between the publishers platforms and or university platforms and all the tools that are being used by text and data miners and what is this that for example organizations that promote openness and relate to recommendations and best practices would like to promote and then with regards to the organizational challenges even though we did not touch on them and we did not dig deeper deeper into them we identified that there were issues such as ownership incentives especially for publishers who would have to change and amend things and the cost of course and then who is going to what kind of agreements are going to be with regards to standards formats how is this going tool is this going to be delivered and what are the risks and who in reality is going to get benefited from that and what about those who don't see the actual clear benefit right at that time in front of them and then to proceed a little bit further with regards to the technical section we created a survey to establish the position of the stakeholders where we asked them a variety of questions that related more to their needs and what kind of TDM queries they may be running or how do they see themselves using this service and then we created the requirement analysis where we created stories for users and stories for publishers so we took some hypotheses and we said that as a researcher I would like to be able to do the x thing from a publisher system for example and to collect full text or to collect the data set or to get this type of information and with this information I would like to be able to do an analysis on the x topic and then while looking at the users we also looked also at the publisher's point of view and then we also asked the publishers what is this that they would like to see from this tool and then what they came back to us saying is that they would like to be able to evaluate the usage of this tool and they would like to know who had access to the tool and what kind of information this person got from the tool so it was more about knowing their audience and their customers and how the information is being used so that they can monitor it and they can use it for their future needs as a publishing house and then we developed a conceptual design which is presented here and in reality to make it very simple this graph over here it has a user who does a call on the API of the UTDM tool and then it requests some content from a publisher and when then what the tool is doing is that it goes to a variety of publisher and it tries to bring to the user back the results that the user is asking and then hidden behind this user action there are some extra actions such as the authentication of the user the authorization the aggregation of the content and then some other optional services and then we also defined the services that are required for to implement the UTDM tool and what we wanted to have is to create a core service of the UTDM tool that it's going to have all the things that we discussed earlier such as the validation the URL redirection and the aggregation but also some extras which would going to be beneficial for the user the publisher but also the medium like the university that would sit in the middle and this would be for example the casting so that it would not produce a lot of traffic to the universities and publishers load or the searching of the aggregated content or how these usage could be monitored either by the publisher or by the institution that sits in the middle which is the university and then we created also a table where we tried to identify the other most important organizations or publishers that provide the content for text and data mining and we tried to make it in a nice visual way give the benefits of the UTDM tool and while we were doing this research in an RRI way we in the beginning we were very much frustrated and we were not quite sure whether the things we are doing we are doing them in an RRI way and we are doing in a way that it's more inclusive and we had a lot of challenges but we also learned a lot of things there so one of the challenges that we identified is that researchers must have a very good understanding of the stakeholders that are related to the research question and it took us a couple of meetings of brainstorming to find out who is going to be affected by this research proposal and who would be interested from this in this research proposal and sometimes you just realize that it takes a lot of thinking to look around in a spherical view and try to see all the stakeholders that are being related and then we also wanted to see that funding for following the RRI process is always a challenge because right now this project has stopped because we cannot receive a funding and doing it in an RRI way means that it takes more time than the one that we could do if we were not doing it in a RRI way and then we also found out that we involving all the stakeholders from the beginning of the process it was very very interesting but then we also realized that at some point we because the discussions varied from the first discussion to the last one that at some points when the discussions were more technical we would lose the interest of those stakeholders who were not as technical or their role were not so technical and then we were when we were talking more about the organizational aspect of it or like about best practices then perhaps we would lose other stakeholders who either did not have the interest or did not have the capacity to participate into the conversation but nonetheless we were very happy that we did this based on the quadruple helix actors even though we did not include some of their main components and stakeholders and we wanted to we are very happy that we did it in the first for the first time via project because we did not had the pressure of doing this like in real life actually participating in the in the RRI component in real life so now in the future we would be able to perform in a RRI way a project because we have learned all this knowledge from this fit for RRI project which was whose aim was to teach us how to do this in a RRI way and there were some surprises for example in the beginning we thought that we should not involve too many publishers and then it turned out that some publishers found out about what we were doing from a word of mouth so they contacted us and they asked to be involved into these conversations so this was very surprising for us and then there were also others who were interested about the experiment and about the aims of the experiment for example someone from within the open university found out about what we are doing and they were they contacted us again to find out more information about this so those are very good things on promoting the project but also on the impact that this may have perspective in impact that a project may have on others who may not be very much familiar with what we are doing and at the same time as a researcher we did not think that they would be interested in what we are doing so this is everything for me and i hope that i did not take a lot of your opinion. Thank you very much Nancy this was really really excellent very detailed and thank you also you did part of my job because you summed up some of the key points that i wanted to say and thank you very much for that we can proceed directly to the questions also do you know with the time constraints i already received some questions but in the meantime please feel free to raise hands you know write a note or do something let us know if you want to ask questions i'll just proceed to some of the questions that i already received in private messages from you from the audience there's a question actually for for all experiments was there any clear mindset change towards RRI in terms of adopting RRI inside the organization any anyone who wants to answer from the experiment yeah well clear mindset i would say that is exaggerated for sure there's a new sensitivity for the topic and it is entering in our overall agenda of the institution so we are start thinking about responsibility and openness we approved policies on these so there's improving attention now we really need a general cultural jump probably this happened in some just some part of the organization but not in the entire organization thank you very much Nancy in your case as i said earlier our project was not so much to change people's mindsets within the organization within the institution but we also realized while we were having the discussions with the variety of stakeholders that concepts that were presented from one point of view let's say the publicers were not concepts that were thought from another point of view let's say the text and data miners or those who promote best practices and especially with regards to openness and using open tools so in our group this has happened but not at an institutional change yeah and it's actually very interesting to notice this and it's a very good case study and it was actually on purpose that we had in the consortium and as part of the experiment the open university because it is a completely decentralized organization as compared to the traditional higher education institutions in Europe and this showed actually how RRI can be embedded in these two alternative types of institutions thank you very much Raquel if you're online mindset change well as i said before i think it's more like we planted the seed and yes i think in in time things will start at least the city is there and people are starting to think about it so i think it's not clear but it takes time to see the change yes thank you very much and the second question that i received as a private message from the audience relates to involvement of society did it work or why it didn't work Andrea you can start again if you want okay oh well yeah it worked i mean that's probably even though i say that we are a little bit reluctant to cross virtualization i understand that there is well a mutual need of cooperating with civil society and also the society is asking for reference point to to cooperate with academia so i think that this is like as Raquel said in terms of mindset it's an interesting seed that we've planted and we're working on that thank you very much uh well in the case of finance of course it's a different situation so it did not apply that much ken well in our case as i said also during our presentation that was the part that was a little bit skipped because of our activity we work a lot of public authorities and we work a lot with the industry which are our main clients and partners and we but at least we know that yes engaging with the public public engagement it's something that we really need to improve and there are plans for that already so yes i think it's a matter of time thank you very much are there any other questions that someone wants to ask live besides the written ones i just want to add something maybe yes but at least in our case it's it's noticeable that researchers are quite resistant to involving the public in the research so yes that's also something that we need to work on with the researchers it's themselves excellent okay if there are no further questions i would like to thank you very much for joining specifically this parallel session it was a very engaging and actually applied discussion and hopefully you got the answers that you were looking for everyone that attended and for us it was a very good learning experience as always we can close the session now and as a reminder in 12 minutes the last session of today's uh strand of it for our i will start please connect using the appropriate link it it is going to be fully interactive and it is actually going to capitalize on everything that happened until now during this so thank you very much bye bye thank you bye