 I'd like to introduce our three panelists. We have our very own Usheen Boydell from right here in Ireland. He's a data scientist with CDAR and UCD and he spoke earlier this afternoon. So Usheen's background is in data analytics, machine learning, personalisation and recommender systems and he spoke about machine learning, AI and differentiating between the hype and reality. We have Kim Nielsen from the CEO and founder of PIVIGO based in the UK, am I correct, yep, and with a PhD in astrophysics and an MBA from Cranfield School of Management. She has given a talk as well today and her talk was on from data dinosaurs to data stars in five weeks, lessons from completing 120 data science projects so we'll talk a little bit about that too. PIVIGO is a data science hub, our marketplace based in London, maybe you could tell us a little bit more about that. John Elder is no stranger to us here in predict having spoken at the previous three events so welcome back again. John is of course based in the US and he leads the US largest and most established data mining consulting firm Elder Research with offices up and down the east coast. John is running his workshop tomorrow, core machine learning and data science techniques which will be hosted in the Herbert Park Hotel right beside us here. So still tickets available, I highly recommend it, I'm going myself so looking forward to that. So without further ado, I have some questions here from the curated information that we've gathered throughout the day from Twitter and from interesting talks. So these are open to whoever. So starting out, I guess what struck you, anything unusual or new in predict 2018 that struck you that you might have thought unusual or you didn't expect anything you'd like to comment on? Who'd like to take that? Sure. Well one of the things I noticed was that there was four talks today with the word hype in the title including my own. So I think you know one of the things that I noticed is you know this use of the word hype around AI and data science and there's a sort of need I think from people to kind of understand what is the hype and you know what is the reality and that was definitely a consistent theme that I noticed across a number of talks today. And that's what predict is all about, isn't it? Trying to demystify and debunk the hype and really get down to the real practical applications and I think we saw some great ones just a moment ago with the biomonitoring and the devices that design partners are creating really practical and working solutions that are delivering. Any other thoughts from Kim or John? I was particularly happy in this very last talk as well to hear about the word empathy used. So I think that is going to be it's not yet very much talked about in our data science community but I think it's a topic we should talk a lot more about is how we can become more empathetic with whoever is going to be the end user of the algorithms that we create and so on. So I think that is going to be a topic to look out for in the future and to discuss more. John, any comments or thoughts? There were a lot of good talks and I like the ones that combined the business issues along with the technological. Kim's was particularly good in that, done a lot of successful projects and was outlining the path you have to take and the stakeholders involved very efficiently. I love that talk. Because we've learned over the years that obviously the technology is extremely important but the main killer of a project is the resistance to change. It's people related, the biggest danger is other people and not the technology risk. Yeah and I think that's where the whole concept of design has really tried to play into that and I love the way Cormac said you know you have all this data and at the end of the day the system just says great job well done and all us data scientists are thinking but look at all the data you could have presented in graphs and tables and charts but all they want to know is good you've done well today. I had to get rid of my Fitbit. It was in charge of me instead of the other way around you know all that gamification and metrics. It's too attractive. So another topic that came up today that wasn't actually prominent in previous ones was the concept of dark data or people call it black data so that was new to me. I don't know if you guys would have any thoughts on that or what that might be from an organization's ability to do data science. Any thoughts on that one? I must have missed that talk. I'm not quite sure what the definition of it is but of course every organization will have large pools of hidden data. It exists out there and actually one of the points that I was trying to make was that often an organization starts thinking they have data but when it comes down to it where is it? Who owns it? Who's got the keys to that locker essentially and are they willing to hand it over and that's a big challenge. And there are always surprises in data. I think one definition I made up for big data is it's data nobody's looked at. So you know they have data they have myths about their data and then you finally look at it and you discover artifacts that sometimes are more important than the models themselves about how their business maybe a flaw in their business or something. So there's a lot that we can add to a company even before we model or forecast just understanding what they have. I think in the context of unstructured data as well and huge silos of opaque data where there isn't really much kind of understanding to start to kind of get an edge on it that that's a perennial kind of issue. Yeah I think data is key and I think earlier this morning we talked about is it about smart models or smart people getting smart data into their models and I think these days there's models all over the place there's data robot that can run a hundred models against your data set so you have to be very careful with with the data you choose and that's where the real thinking comes in and spotting those small anomalies that might lead you to a big insight or else it just might be bad data that is meaningless and that's the interpretation of data is key. To know it's an anomaly you have to have some domain knowledge you have to be in close cooperation with the folks who have the business which isn't possible if they see you as a threat or you know so there's all these human factors that you have to take into account to really have success. And in previous years John you talked about delivering projects that were technically successful to large companies often that had good actionable insights but unfortunately they sat on the shelf maybe you could tell us why and have you experienced that becoming less of a problem since the few years ago when you talked about it? That's the biggest problem we've ever faced in our first decade 90 percent of our projects worked technically but only two-thirds of them were implemented and our second decade is closer to a hundred percent because we choose our clients better and and then about 90 percent are implemented so we've really come a long way I think in the industry it's more like only a third are implemented I remember when Microsoft they were talking with me and they were astonished that I had two-thirds of them implemented because internally they were only getting a third of them actually implemented which is astounding you think about this expensive meal the chef is prepared customized for a patron who likes it pays for it and then you find it scraped off into the trash at the end of the night you know what's going on so it's these human factors of lack of trust and you lack of trust in the model change fear to what's going to happen to them and these cognitive this cognitive biases are something that are extremely important probably more important than the technology once you reach a certain level of confidence in the technology you really need people who speak human yeah absolutely any other thoughts on have you any either of you experience that same thing with a really good technical project you're delighted with the output but all of a sudden the engagement with the end client or whatever for some reason it doesn't engage I guess Cedar sees that as well with all these demonstrators and trying to get them out there into the marketplace that's right so we see that from a number of different perspectives and in some scenarios as well a company might want to do a particular demonstrator project of some sort of quite out there idea that they have which is you know really exciting for them to work on and work on their data and show these kind of possibilities but it might be something that really aligns with their current business at the moment and that's it could be kind of too far ahead of their current business roadmap that's one of the challenges with the Cedar model is that you know Cedar are available for companies to go oh I'd like to try that out you know I don't have the skills I don't have the capacity will you try that out and that's great and you can try things out but then it's really getting that back into the core business because it is by its nature out there and so it's interesting to see now Kim's done a hundred and fifty projects coming to that five months so how do you how do you do that well in five weeks five weeks sorry yeah well it's similar it's similar style projects where it's often about companies doing a proof of concept and trying something out and and yes absolutely horror stories unfortunately of companies that where we have delivered algorithms that we have shown can save the company millions of pounds or increase revenue by millions of pounds and you check back a couple months later and nothing has happened and one particular horror story where the whole company including the CEO and the board were super excited about it but the CIO blocked it because the CIO just said no doesn't fit with our systems doesn't work and and and again in my talk I talked about find the skeptics in the organization and try to convince them bring them on board early because what happened here was that no one had talked to the CIO they'd gone ahead done the project and then sort of dumped it in his lab and said get it to work and and there was that that cognitive bias you know I don't want to do that I want to be part of it so yeah good um so um also on that point um in terms of deep learning have you seen adoption of that in the become more mainstream or how do you engage the progress level of deep learning in industry um are there real projects being executed successfully are there still in the early days of deep learning any thoughts on that yeah well I would say I mean I think deep learning is it's like it's hugely on the radar at the moment and I mean I think there's a lot of tasks that I can do that just aren't really viable with more machine more traditional machine learning techniques so I think you know that we have seen a lot of interest in that with with cedars members companies as well and particularly some of the the benefits that deep learning can bring in terms of maybe negating the need for so much low-level feature engineering which then allows companies to apply these with less sort of very specific domain or or data science expertise if you build the right tools around that which is something of interest yeah do you see that as well over in the US deep learning is by far the most hot topic yet I've not touched an actual application yet but it's inevitable because it is actually powerful I mean as as anytime you have neural nets the hype outweighs the reality but the reality is still very good and and there will be applications very soon I'm sure and did you focus on that in your and if you're 120 projects one or two one or two but that's typically the startups that are working on the on the absolute absolute cutting edge that that are trying to do something completely new with any of our more established partners no not yet very good and keep your questions coming in on on Twitter if you have any questions at this stage we do do have a little bit of time so hash predict conf if you want to tweet something in I'll keep an eye as best I can and still have some more here so so we talked about data driven culture in organizations and what the how do you think what what steps can be taken to help companies to become more data driven in their culture and can they get to data driven and lose some kind of spark of creativity within their staff or workforce that that could be detrimental and he thoughts on on on that that's an interesting question get to day driven we're always pushing people to be more maybe maybe we can go too far certainly can go too far and getting them excited about analytics and then they think oh my gosh you can do everything you're like wait wait no wait come back you have to manage expectations a little bit but the the sign that they're getting more data driven is when they start thinking about investments that they want to make in terms of doing things just to get the data to see how it turns out later you know and that's that's definitely down the line but that's exciting when you start to see thoughts along those those realms and of course you're not going to get there unless you have those those pilot projects that get that early win and and get that return on investment with a measurable goal usually doing something that they're already doing just doing it better faster you know more uh accurately um and then and and trying not to do a moonshot trying not to do boil the ocean or use whatever analogy you want to try not to do the big projects that typically have a very hard time succeeding but do those small ones and then momentum will build yeah yeah I think we had a horror story of an organization that went too far in in the UK in the last six months and that's Cambridge Analytica yeah and I think that's that's actually one of the theories about going too far is when you have an organization that is full of super smart super focused people who just want to see want to push the boundaries of what they can do and they don't consider the consequences of what they what they do they just want to solve the problem and they don't care why they're solving it or for who and and they obviously went too far and it brings back the question of how do we maintain empathy in our industry and how do we maintain ethical use of data etc and I think everything comes down to the culture of the organization obviously that was a toxic culture in that organization and in that in turn comes down from the leadership and the communication throughout the organization and it just needs to be on point whether it needs to become more and more data driven or less it's about making sure that you communicate from the top what you want to do with your data I think ethics is becoming more understood now that the dangers of you know being totally data focused and whether you can do something doesn't mean you should do it so I think having that ethics committee and universities do this you know when you're applying for grants and there's any kind of human studies or interactions within those grants there's usually an ethics committee that has to sign off so I think companies are a little bit behind in that the techies can be do things or the management maybe doesn't know exactly what they're doing or what data sets they're using are those data sets diverse enough are they biased and what is the impact of this work so I think kind of a maturity in AI and modeling needs to arrive where people can have explainable AI and we talked about explainable AI this morning as well and then the decision makers can really understand what are they doing and should they be doing it and then have a kind of diverse group who can look at these things from an ethics point of view I think that's going to be crucial do you have to get your stuff by ethics committees and UCD sometimes uh machine yeah I mean yeah of course and I mean with you know GDPR and and these regulations you know it's that's a measure of example of how important it is and I mean I think yeah the explainable AI is well going back to the accountability of models that you know there's just thinking about that now whereas maybe a number of years ago that that wasn't really thought about so much I think deep learning and explainable AI to me seem a bit counter what you had to call it orthogonal I suppose because essentially that's the whole point of deep learning is like you just throw the data at it it figures out all the probabilities and it's really just high powered statistics with high powered computers figuring out all these probabilities that you could never really explain but I guess you can explain the process and the data and your assumptions and then the models and how you train them yeah and and I think it's a lot to do with the context of where these models are used as well some decisions that you want a model to make you just want the most higher accuracy as possible it doesn't matter they don't have to be explained as such they might have kind of direct impacts on people for example whereas others you do so it's it's a matter of using the right tools for the job there as well and I understand that the sometimes the explainable models are less powerful than the non potentially explainable models so then you have the dilemma whether you should be using the best model for this particular diagnosis of something or other or should you use the explainable one which you don't trust it as much but you can explain it so therefore is there an ethical challenge between those two ideas and there's no harm in running them both and then if they agree you're both happy right maybe that's a choice in the future when you go to your db it's like do you would you like the answer that is most likely to not be wrong but it'll be less likely to correctly diagnose you that's the patient you want the right answer or you want something I can explain it's depends on what the question is I guess right I really like the session this morning on the customer journey analytics and Disney as customer experience was raised as a an example of doing it right and I think again using data to drive that and pushing some of that data to the front lines and the people who know the customer so they can interact more more powerfully with or you know more effectively with the customer is a great example of how think companies are thinking now and even Disney was thinking like that so this comes to kind of the the collaboration again between user experience design and data science and how how can those two things really coexist are there conflicts there sometimes or are there challenges in getting those two things are they mostly in your experience well aligned and you know reinforce each each other any thoughts on the model that's going to be consistent but it's doesn't have any common sense can't look at the situation I was a year or two ago united airlines ended up like handcuffing somebody and taking them off the plane when you know if they just raised the amount they were willing to offer volunteers they wouldn't have lost millions of dollars in stock value after this horrific episode you know and and yet the the employees were actually following the manual they were following the algorithm of what to do because that company had put a lot of emphasis on standardization of processes and so forth and no one felt armed to make a judgment call at that point so if your people are really good you want to give them as much power as possible and where you have less capable people you need the you need the things but it's it's definitely it's definitely a judgment call all the time yeah combining those two things the computer and the human at the who are is at the cutting edge and has really good decision making capability like you said in that example you know it just makes sense so but unfortunately the computer says no is a kind of a phenomenon these days if you're at the checkout at one minute past 10 and that you're trying to buy a bottle of wine and the computer says no simply can't pay for it because the check it says no so any other thoughts on that coexistence of user experience design and data science and in your experience Kim and in the projects you've been doing see it as important yeah so most of our projects sort of end before well at the algorithm stage before it actually goes out to a user and I thought it again it was very interesting from the very last presentation now about the thing that maybe you just have to tell people you're doing all right that's all they need to know and I think it is there's there's something we need to balance there because on one hand we have to make the outcome the insight easily accessible to the users to the end user but the same time we also have an educational prerogative I think to to make people understand how their data is being used and why it's being used and how they can influence it themselves and therefore I think we should also try to educate that audience and explain or at least give them the option to understand how the data was used any thoughts or yeah well I mean I think you know as well going back to that and it's kind of related to explainability explainable AI in a way as well that you know for in the context of some task for some people they just need a sort of yes no answer at the end of the day you're doing fine and then in other cases you need some kind of explanation or the end user might want to know that and they might need that information so it's very kind of context specifically and I suppose a really well designed system will tell you you're doing fine but if you really want to drill down and only three percent of people might but they can drill down and say okay why are you saying that you know what's doing what's good and what's bad or what could be better so having that design I am an outlier here I'm sure because I actually think interpretability is a grave danger I think more harm has been done by paying attention to interpretability than by saying okay I'm going to make sure I have really good science around my black box but the people really want insight they really want interpretability and that innate human need I think has to be paid attention to whenever possible so there are these interesting technologies out there now for helping people do locally linear models at least so they can explain what the sensitivity is at the point where you are on the model you know and there are laws in the U.S. that if you're to turn down for credit you have to be told the five major reasons under your control that have anything to do with that and so these are very helpful forms of forcing some at least some sort of interpretability on but in most of my experience a lot of data modeling problems come from paying too much attention to interpretability because people fall in love with an interpretation they want and then they stop listening to the data okay once they have once they find something that they like they they pay attention to that and it's way worse than the model they could build if they would just let it go yeah yeah so you can fall in love with the the concept and start biasing your thinking bias everything the data yeah I agree I agree that that's true and two more quick questions one is about future skills needs you know we talked a little bit about this this morning but with with all of this power of technology and modeling coming on AI even coming into force what do you think are critical skills we should be teaching our students and how do they cope in this in this world of data and AI any thoughts or she's well I think I mean there's a lot of discussion around you know Python or R or these these kind of technical skills which is important obviously but I think underlying that particularly in data science is a curiosity and a sense of curiosity and questioning and to sort of foster that that kind of approach because I mean that leads to wanting to ask questions about the world and then asking questions of data to find out like Python and oral come and go you know we've been through different technologies and languages but having that ability to continuously learn but how do you instill curiosity then is it is the next question I'd I'd yeah well I mean I think that probably has to come from kind of quite early stages in in education so you know young children should you know have the curiosity in in that age group and then that builds a yeah their life yeah any other thoughts Kim I would like to add as the flip side of curiosity we also need skepticism okay in the sense that we can be curious and we can try models and and we want to explore as much as we can but then when something comes out we have to be able to look at it and go is that correct have I been biased somehow is there correlation when there's is there causation when there's correlation and be skeptical and really then try to twist our model around and see have I made a mistake anywhere so we need to we need to have a curiosity to try things out but then also take a step back and in that scientific approach really tests the outcomes critical thinking and I think that's a skill those are vital and you can't do it alone because if you the thought of a weakness in your model you it wouldn't be a weakness you have to have other people critique your model and that means you have to have a culture where that's a good thing where you're not attacking the person when you attack their model you know you have I teach occasionally at the university and every now and then you get a student who's thankful for the marks that you give on their papers or their work mostly they come up and negotiate in America I don't know if that's the way it is here but in America they want those extra two points you know but but every now and then the student realizes oh my gosh an expert has carefully read everything and has responded to things so that's that's a great sign you want to hire that person you know the the groups if you have groups where people are free no matter the rank of the person with the idea to say well what about this have you thought about that did you turn the machine on you know various things and it's not taken as an insult but it's it's a it's welcomed as a way to make truth be found and success go forward because reality is going to kill your model if you can't you have to you have to work really hard to kill it in the lab to avoid having it die out in reality it's a great point actually yeah and I think that peer programming and kind of philosophy are you know agile and scrum where third people work in teams and they code review and they check you know it's very powerful learning and I think that is the future so lastly and then briefly maybe we can last year we talked about the singularity and I love to talk about the singularity let's finish on that the prospect of AI posing a threat to humanity by becoming even more powerful and self-replicating intelligence that even humans cannot understand yeah the panel was largely skeptical last year good skeptics as Kim you know good thinking obviously but there is a risk and whether it's 10 20 100 or 200 years away and so I think we should just revisit that briefly any signs the singularities around the corner are becoming more of a threat than it was this time last year John well if you remember the panel last year split and actually in the audience talking later but completely by age old like me said no way younger people said of course you know there are things not dreamed of in your philosophy you know so I haven't changed my mind but I am I am surprised I mean the AI which by the way when they say AI has solved go and poker and driving they're using data science so using inductive methods to do that they're not using the traditional AI methods so at least it's our side that's winning but anyway I'm on record is saying never gonna happen okay sticking with it Kim she just wants to go in with the youth camp there yeah I read so much so far and I love it and of course it's gonna happen but actually but but but my take on the singularity is that I actually think it can be a positive thing because I think we can then learn from them and it can help humanity move forward quicker than we could have otherwise I mean imagine that they start to build huge spaceships and and we can sort of hop onto those spaceships like little rats we can be the rats of the next Columbus mission to the next stars and we can get to go on these voyages and and I think it can help humanity takes the pressure off us doing all the thinking right that's surprisingly good have you cracked the artificial general intelligence down in UCD yet or she we keep trying one of the things that I've kind of noticed is maybe a bit more of a realization oh more recently that kind of AI is sort of bigger and broader than maybe we thought or people might have thought so you know there's areas that like a lot of the modern machine learning I mean it's essentially correlations pattern matching that's like one really small piece of the big AI picture yeah like causal inference cause and effect and and these kind of areas I think there's been much more sort of information and and there's a number of books out in the last year actually looking at that these areas which have sort of been ignored and suddenly people realize hang on this AI things actually it's massive and it's there's much more to it maybe than than we've been talking about recently so all right but up or down what's that we're gonna go yes or no it's gonna happen or maybe maybe okay so thank you very much to our panel we'll leave it at that and I'll just say a few closing remarks so thank you very much all right thank you panel thanks