 So let's go right back to Kleena because the first one where are you there Kleena? so Ahmed texted in a Great question, and he asked was there any internal Cultural resistance to the data-driven change that the Irish Times has has been going through and if so what how did it overcome? How did you overcome it? I saw that question there in Twitter actually and it is something that we do get asked quite a lot and Absolutely, there was some resistance to the change What I would say in that is it's a constant exercise So this isn't something that happened overnight and it takes it can take years to implement and I would say it's still it's still ongoing So a couple of things on it I would say one is looking at others in the industry So we would look very closely to the New York Times to the Guardian to the FT and see what they're doing And it helps our case when we're saying we want to bring in more of this or showing reporters the performance of their content It helps when we know that the Guardian have done this or the New York Times have done that and seeing it that way And the other thing I'd say is I mentioned about the quick wins I'm a big believer in finding those quick wins. So So when we started out with the analytics function and the company was about to embark on a digital subscription shift And so the team was set up a year before that up to that point We didn't have a relationship a direct relationship with our readers. They went into a news agent They handed over their money they bought the paper and we didn't know about them what they're reading what they weren't reading So it was imperative for us when we were setting up that function to know about our customer So we were able to go to senior management and others and kind of say you know what here's the number of articles people are reading Here's where we should set a meter or not Here's the impact that we'll have on our traffic and our advertising So showing a couple of quick wins and bringing people over and again going back to the communication piece really really important All about relationships and it is about talking to people and the reputation of the team and the people you hire in that team are Representing operation. Yeah, it sounds like the the other teams are nearly hungry for that data now and they're looking forward to seeing their stats Yeah, and that's the that's the important shift because we could have bulldozed in and then you have the walls will go up And you let you have that fear of it. So we we had to treat it in that way. So you good and I was really fascinated and critical about the the Disney Journey and the customer experience Now there's a few people over from America And I think Orlando is like the new Taito Park with people visiting it as as often as they can but I was just wondering I Think Disney's a great example, but could you bring a more concrete example of how that applies to Davey and you know Your customers and your business opera opportunities and and how that relates to your business a little bit Yeah, so we're we're probably in one of the early stages. So we're mapping out personas at the moment. We're trying to understand So we're we're in the how do we put it? We're in the gathering slash connecting phase in the implementation side, but we've had a good strategy session We've got the senior leaders on board And I think it's it's at that stage where we're trying to see the data that we have The personas that we think are the target audience and do they actually match up or are we do we understand our customers? Well enough to even Define our target audience in that space because I think wealth management as an industry is not really recognized in Ireland It's a big deal in the UK in the US, but I think to understand wealth management It's it's still you know still bucketed with you know stockpoking and private banking And so I think it's there's a there's a growth in the brand awareness that has to happen in the industry awareness And I think as part of that we will probably reap the benefits or at least under help Bring that market up to speed but yeah, so in Davie at the moment We're we have I hope there's someone from my team here. Hello, but the guys in my team are at the minute working with the strategy and Cx team to help build those personas at the minute so and At the end of the day, you know senior management just want to sell more or increase their margins and you know retain their customers What kind of metrics and how does your work? You know feed into that? Can you prove the ROI of the work? You're doing or was that to come down the line? Yeah, no, I think that the agreement on the metrics happened So that that is that is definitely clear It's it's always hard with marketing campaigns to prove ROI that is probably the trickiest bit Especially when you're always going with last attribution, you're always going to attribute success off something the last action Someone took so even though the journey might be long It's probably the very last download that they did that probably is going to get the benefit Or this or the attribution of the success So I think that's where we have to crack that a bit more and understand how we can Prove ROI and marketing campaigns, but otherwise I think The other metrics we are working through but it's mostly NPS So it's very much customer satisfaction and engagement driven that we want to try and bring through and get senior management to You know agree on that. Yeah, not just business benefits. So yeah, absolutely I think sometimes you have to play the long game like Disney would do, you know delight the customer You know, sometimes you don't see it in the metrics straight away But over time that builds up a huge loyalty and a kind of fan base nearly for your brand, right? Yeah, so that's what you're trying to do. Does anyone else any thoughts that maybe you do similar UX customer experience work in the Accenture. Do you have a similar view of that? Yeah, and I guess in the Docker multidisciplinary and I guess like the user experience is key So we have a whole we bought the design agency fewer a couple years ago So we have a fewer design agency and the teams that we put together to solve what might be seen as AI or analytics problems Always have a designer on them to make sure we're understanding properly the user experience I think it is the the realization that great products and services people have so little patience for a proud product I love the idea that the baby like try to swipe the magazine and you know if technology doesn't work They just have no time for it. Obviously the magazine doesn't swipe, but it should Will someday. So yeah, they're just ahead of their time So we're gonna move into a few more Slightly tech interesting question came in and this is probably generally for anybody who wants to take it, but it's quite Modeling focused Elpida Bantra who we know well from creme global and she has this very interesting question And I think you can interpret it a few different ways Do smart people build smart algorithms or do they collect and feed their models smart data? Maybe maybe I'll Can't say yes or no. I mean, it's yeah, and I mean It's it's I can't say you don't need smart people to build smart models. That wouldn't be a good thing to say, but the models are as Good and as limited as as they're built to be based on their algorithms. The data is absolutely crucial data will always beat Okay, I better not say I take out the word always data generally beats a better algorithm So when you're trying to get a good model or a good predictive model, for example It's the data that you're using in other words what you're training on that will make the model good So when you're trying to get the right set of algorithm or classifier or whatever try different ones But the difference will be in the data that you're feeding into it and this and to come back to the smart comment You can't blindly use data in a company and I know that sounds very trivial, but you have to have people Especially if you're using sort of structured quantitative data who understand the data That's absolutely crucial because sometimes the domain experts are kind of left out of it But it's very important that they're part of it, but data generally trumps Algorithms generally Question and it came up seasoning your speech around the Cedar model about the deep learning algorithm that anyone use I mean and this is an open question But is there a danger if you make algorithms possible for anyone to use that people start using them that aren't thinking about the data That aren't thinking about biases and you end up with models out there where Experts who don't know the things that you need to take into account haven't been consulted Yeah, completely and I think that becomes even more of a risk with deep learning because then you don't necessarily even need to work closely with the data before you feed it in because deep learning will Actually go and figure out what the data is about to so yeah that that is a danger and it's it's it's I mean That comes back to explainable AI and it also comes back to your model will only be as good To go back to it again as the data that you feed it in if you've got bad data It doesn't matter how clever this algorithm is you feed it into you won't get good decisions out Because it doesn't know about them It cannot make good decisions based on bad data if it doesn't have the right information and to know whether your data Is good enough or not you have to have people in your company who understand what data you're gathering what it's about How it was and and to clean it and to be able to sensibly understand what each part of it is about It's absolutely crucial. Yeah, then try it in a few models. Absolutely. Yeah, it makes sense If there's weird things going on revisit the data. I think that's really crucial So Keanu Matney had the same question may have just asked it. So Keanu consider your question asked And onwards then to another question from Darrow Darin who's here to speak from from Boston actually And this follows on from that key issue of data and he said how do you then convince leadership to invest the time and the money and The bandwidth in gathering and structuring and cleaning that data The latest expected to be ready to go That's that's that's the 80-20 thing that takes your time, right? So any any thoughts on that how to get your organization to invest in that and spend time any thoughts I think from from an Irish Times perspective. It was a case of starting small We use a lot of open source tools at the start Yeah, we didn't go in and say we need this much investment and this much resource in order to deliver this And we very much started small started with the information that we had did a bit of cleaning on it And again showing those kind of results and those quick wins that we have in it Yeah showing the results on it so from an Irish Times perspective We publish over 200 articles a day it's a huge number of articles and content is is what our product is and So what we've done is really to make that date make that content work harder for us So we're publishing that money anyway, and we're publishing them at a certain time of the day So by staggering the publishing the promotion of them and looking at what people want We've we have exponentially grown the audience for that for that content So that's showing value back and that gives us leeway then to kind of say well We want to do this next and we want to invest in this and because we've shown that because we now have a Subscription product as well. We're able to show what our subscribers are reading what's converting subscribers or content is or isn't So that really helps us now that we have a product that we are selling and it helps make that make that case a little bit easier So it's great to have that number of subscribers so you can see those trends now I'm buying the paper copy, you know You might have a few people shoulder and trying to see what they're reading I always do that. I'm very annoying. So I'm trying some looking at what people are reading But yeah, hi there you can measure that so yeah I think I think another thing is I guess if you can find examples in the media of people that didn't have good data Where they get called out and make sure that people near a company know okay You found you have good examples cuz you were able to get good data But actually this is what happens when you don't the other side yet Metrics as well is very important if you if you agree that you're gonna show improvement and something and you actually prove that Then you can say I can I can only show it to a certain point because of the bad data Clean is there is a risk of Media I maybe even the Irish times becoming too data-driven You know and optimizing everything to chase those eyeballs and advertising revenue that comes with them Yeah, it's it isn't it's very important for us And that balance in terms of the ethos of the Irish times and it's very very core And again going back to how we didn't want to pull those in and say that everybody has to use this And use this data if we were only driven by the data and our journalists are only driven by the data We'd be writing by Justin Bieber every day and chasing clicks in that way And we'd never be writing about something that's happening in Syria that's important for our readers to know So we're very core on that and I think what and there would have been reluctance in that in that sense of it and it's really case that we have to trust our journalists and trust that they and Their ethos of what they are working there for and again go back to the subscriptions being a help It is not about the biggest traffic drivers. What is the biggest readership? We do look at other metrics like engagement time and we have a long read and particularly hard-hitting Pete's Resita Boland one of our journalists wrote a few months ago about Anne Lovett and The death of Anne Lovett and something like that that had you know people reading that for say 13 minutes Whatever reading the article that's much more important than and the pages on that and again having the the feedback that she would get We talked about comments earlier the feedback on on Twitter and other social channels And and also again that subscription piece that we need to attract subscribers in People to pay for our content and we need to retain them We're not going to retain them if we are writing clickbait content in that sense And I suppose going back to that idea of the 200 articles It's not about changing the content as such It's doesn't have the best possible headline to explain what it is for people finding what they what they want to find Does it have the best hit the best? Image in it and are we making it easy for for people to understand that way? So it's it's it's not aiming towards dumbing down the content, but making it work harder Sometimes I noticed the headline changes on the Irish times You know you've seen an article and all of a sudden it has a new headline and you go in and realize Oh, I've already read that article. Okay, you come up with a good headline like that We do it there's a skill in that and there's also there's differences in terms of the headline can be for From a Google search point of view the everything you need to know about the Pope's visit was the headline that was needed for that And we would have looked at Google trends to see our people searching for people visit for Pope's visit It's a Dublin is it Ireland? What is it and to get those right keywords? We do use headline testing tool as well But there can be differences in how you promote something on Facebook versus you know how it is on on Twitter in that sense But there are key things like if you're talking about a sport a match or something like you you have to have the team names You have to you know these kind of things But then when we do kind of first-person personal stories things like with a question mark We notice that headlines with question marks in them work better than than those that don't so there's There's trial and error But the thing because we're looking at data in real time is that we can see we can change a headline We can test it and keep going as opposed to saying if we'd known that yesterday we would have we would have changed it So very good Just moving slightly then over to the skills And you know a lot of people here would have young kids growing up at various ages and various stages of development in their education system And it's always interesting to think about what they're learning in school and what what the work Work world of work will actually require from them in the future So what are your thoughts on what we should be teaching our kids? Beyond maybe above and beyond the core curriculum of you know basic education skills that they will get for them to prosper in this brave new world So I'd like to start with that one I Mean I guess what we're seeing in in the dark, and I think it's probably across the board I mean Experimentation and curiosity are key. So with things changing and things are going to keep changing just learning Learning a set amount of stuff like obviously it's important that you know the basics of your skills for data scientists You know about algorithms But actually how knowing how to to learn continuously and being interested in learning continuously is really important So I think in the Irish education system. We're quite good at it in primary school Then as soon as kids hit secondary It turns into rote learning for exams and I was actually my son's going into first year next year And I was at a school Presentation and the principal was talking about and these are the subjects you do and for in for Interest that I was gonna say for for a leaving search and then she and then she got to fourth year And she said actually your kids long after they forget what they do in the leaving search They'll remember what they did in fourth year exactly for that reason because it is you go back to the curiosity and experimentation I think things like Kota dojo are brilliant as well And I think it's to integrate more of not necessarily pure coding skills But having that mindset of actually wanting to crack something open and you know the curiosity bit But trying to build something from scratch is such so empowering to kids and building an application or building a mobile app So integrating something like that Yeah, I think learning by doing is much better than learning by sitting and listening So I think we're just about out of time So we will adjourn for our coffee break in the experience zone So let's thank once again all our speakers are excellent this morning