 Hello, and welcome. My name is Shannon Kemp, and I'm the Chief Digital Manager of DataVersity. We'd like to thank you for joining today's webinar, Trends and Predictions for 2009, brought to you in partnership with First San Francisco partners. A couple of points to get us started, due to the large number of people that attend these sessions, you will be muted during the webinar. To open the familiar chat and Q&A panel, just go to the bottom middle to find those icons. And the Q&A can be found by clicking the icon that looks like three little dots. For questions, we will be collecting them via the Q&A in the bottom right-hand corner of your screen. Or if you'd like to tweet, we encourage you to share highlights or questions via Twitter using hashtag DI Analytics. If you'd like to chat with us or with each other, we certainly encourage you to do so. Just click the chat icon in the bottom middle of your screen for that feature. And as always, we will send a follow-up email within two business days containing links to the slides, the recording of the session, and additional information requested throughout the webinar. Now, let me introduce to you our speakers for today, Kelly O'Neill and John Ladly. John is a business technology thought leader and recognized enterprise information management authority. His 30 years of experience including planning, project management, implementing information systems, and improving IT functions. John writes and speaks on a variety of topics that enjoys sharing his expertise on strategic planning, data governance, and practical technology applications that solve business problems. Kelly is the founder and CEO of First Hand Francisco Partners, an information management consulting firm. She is a veteran industry leader, speaker, author, and trainer. Kelly is passionate about helping companies leverage the value of data, empowering them to derive insights that inform decision-making and improve results. And with that, I will turn it over to Kelly and John to get today's webinar started. Hello and welcome. Thank you, Shannon. Hello. Good morning. Good evening, I guess. Excellent. Well, John and I are really excited to get this webinar going. This is one of our favorites because we get to think about what happened in the 11 months previously and also pontificate around what we think is going to happen in 2019. So it's super fun to build this presentation and we're looking forward to sharing it with you. Pontificating is my favorite part. That's right. Absolutely. We will do our best not to do soap boxing, but definitely there will be some pontificating, anyway. So we will do a little bit of a review of what we saw in 2018 and what are some of the things that are influencing the trends and predictions for 2019. Hopefully this will resonate with many of you. And we've divided our predictions into people, process, data, and technology. Now, obviously we know in this world of insights and analytics that there's not a black and white delineation between each of these different four categories, so some of these obviously influence each other. But that's how we're grouping it and then we'll wrap it up with some key takeaways. We are planning on having more time at the end of this call than on previous webinars. So we're going to stop the presentation and open it up for questions at about quarter to so that we can go through all of the questions and share any ideas that people have before the end of the webinar. Okay, great. All right, so what did we learn in 2018? A couple of things. One of the things that is really interesting is that we've seen that there's been this shift from how to a what. So capability is really what needs to be done. And so we're seeing this shift from being worried about how you do data governance, how you do analytics, how you do big data, to what do we really need to get done and what are those capabilities that we need to develop in an organization to be sustainable and to create some sustainable practices around analytics, big data, and data management. So that was one of the first things that we identified. Governance is evolving to become more collaborative, user-empowered, and user-supported, and this is augmented by some of these technologies. So we really see governance moving as close as possible to the decision point. And therefore this is impacting operating models. This is impacting the way that companies are organizing around governance so that the service can be provided as far or as close as possible to that user and to that data decision as can be. We're also seeing this shift from process-centric approaches to be more data-centric approaches. Again, some of this is being fueled by new technologies that have been developed. But what we're seeing is that it's not just about optimizing processes around data anymore, but it is truly about optimizing data about the data. So I don't know, maybe we could call this the year of – the 2018 was the year of the metadata. And then demand for technology enablement. This is, I think, going to continue to influence what we've seen in 2018, 2019, and forward. The demands are becoming higher. The pace of change is becoming dramatic. There's more work to do. There's less time to do it. More business units are interested in governance. There's more regulation happening. Frankly, we've just got more data, right? So there's this demand for technology to help automate and enable, whereas before we were probably muddling through with this process approach. So again, the shift from process-centric to data-centric. And then, of course, across the enterprise, everybody's becoming more data-literate. And so the delineation between someone who's in data management and someone who uses and consumes the data from a capabilities and a knowledge perspective, there's not such a knowledge gap between the two. So again, these are the citizen data scientists, the citizen data governors, et cetera. John, anything you wanted to add to the learnings for this year? Just a couple of reinforcement things for folks that are maybe – what do you mean by data governance is more collaborative or user-empowered. Some parties are saying that that is data governance becoming decentralized versus centralized. I even read that in one of the big analyst firms here recently. I don't like that choice of words. This choice of words here is meaning that data governance is becoming more spread out. Remember, at the end of the day, if you're going to do big data and analytics and be data-driven, essential behaviors around data need to change. Well, and that means everyone's behavior. So if everyone starts to think about it, become more empowered, become collaborative, it looks decentralized, but it's really the behavior changes which are the important thing there. And then the comment on metadata. Kelly, I agree totally this has been the year of metadata. In fact, some of our clients and some other things that we've just observed is that the metadata and the results from the output of, say, a glossary or a lineage effort or a GDPR-inspired or whatever thing are starting to drive some other initiatives, some other data governance, in fact become an instigator versus a deliverable, which is really cool. So yeah, just a couple of things I wanted to add to show that it's been a really significant year, I think. 2018, as we look back, will maybe a bellwether year. I don't know if it's going to be a tipping point year, but it's a bellwether year. Yeah, that's great. I appreciate that. All right, so as we move into 2019, so what are we seeing from a people perspective? Well, the rise of the Chief Analytics Officer. So originally, I think the original webinar, John, the series that you were doing was the Chief Data Officer series a couple of years ago, and then we morphed into data and analytics. Well, now we're really seeing this roll. When I was young. Right, weren't we all? And we had our discussion earlier this year on a previous webinar where we talked about these role categorizations, and the Chief Analytics Officer is becoming much more common in organizations over the past year. So just keep an eye out for that, and we see that continuing in 2019. Data literacy will of course continue to mature, and organizations are truly becoming more data-driven because it's not just the technologists or the data people that are worried about data anymore. It's actually the people who are using the data. They want their data to be trusted. They want to make sure that it's verified and accurate. They want to use their data to drive additional capabilities. And so that data literacy is of course going to continue, which influences the next trend, which is that of the citizen data scientists. So as the data literacy matures, there will continue to be more roles that are data-focused and to support these data-driven enterprise. The citizen data scientists is one of them. And really, I think the fundamental aspect of this is that non-data people are asking data questions. So that's really the biggest change that we're seeing, or what we're seeing as the trend to go into 2019. So how do we – sorry, John, did you have something to add? No, you were just ready to say there. You were headed there. Go ahead. Great. Okay. You want to talk a little bit about what to do about it? Any advice? Well, yeah. So what I was going to say is the CAO aspect – I'll start with that one when we talk about riding the trends. What I was going to say there, more appropriate on this slide, is that CAOs are actually starting to reach out now as initiators of data management and data governance. Whereas maybe three, four years ago, right, Kelly, we saw they came in and they were kind of like the Uber data scientists, right? And doing the cool things with predictive and descriptive analytics and stuff. But now they're actually – in fact, I think a lot of CAOs might be really kind of chief data officers with a different title and stuff like that. The point from all of that is that you're going to have to revise your AC matrix. Accountabilities are going to shift next year and in years to come. The takeaway for someone is when you do predictions, right, and Kelly chiming in. When you do a prediction like this or something like that, we're just trying to get you light on your feet. And what we're seeing is that the organization and operating frameworks are going to be a little bit more dynamic here as things get solidified as things mature. So the analytic officer per se will fit in there and hopefully will add value. So it's really important now that you might have an analytics officer whose analytics function is how do you add the ongoing value, all right? Will this person add value? Well, not that person specifically, but the role. Now on top of that, so to be very, very aware next year and years after that is this growth of the citizen data scientist in conjunction with the power of analytics and the sense of being data-driven will mean that you're going to have to have guardrails there. You're going to hear compliance. You're going to hear the word ethics a few more times in our talk today. And this is where it starts to enter into it. It's the old Spider-Man quote, with great power comes great responsibility, right? CAO knows a lot. The users, the business users are much smarter. Next year, you're going to have to keep an eye on that nobody gets into trouble. So that's how to write it out. Absolutely. No, I think that that's great advice and to make sure that also your organizations are continuing to be efficient. John, did you want to start the next question? Yeah, yeah. So process-wise, and we broke that into, first of all, kind of the highlight functional process area here, data governance, 3.0, right? Expanding data governance to be defensive. So that means keep you out of trouble, all right? You've got the capabilities I talked about on the prior slide. Self-service, machine learning, automated responses to situations. Data governance will step up in the coming year to allow organizations to better leverage that without getting in to trouble. Analytics is going to become more intrusive today. Today, business is because of robotics, because of business rules, because of AI. And you're just going to have to get used to that, which means, again, whereas a lot of our focus might have been around analytics with a downstream model with some downstream advice, you are now dealing with governing and managing a capability that has real low latency that wants to go from idea to implementation in less than a business day. That's some significant process changes for some individuals. Augmenting analytics, that's the automatic intervention in business processes. It's the old problem of automation, really. When you automate a process, what do you do with the manual things, what do you do with the controls? Yeah, there's a sociological thing of automation. Set that aside for a second. What's just the nuts and bolts? You can automate something, but are the controls in place or are the controls necessary or do you need additional controls? Lastly, the digital twins and other types of virtualization technology or augmented reality, virtual reality, whatever, sitting there in a chair wearing goggles. That's much more interaction with the data, much more interactive responsiveness. Boy, that data better be right and you better know what it means or you're going to get in a lot of trouble in a big hurry and not even know you're there. So there's some stuff that we have to be able to do about that and I'll turn it over to Kelly here for how do you deal with all this? Great, absolutely. So lots of challenges happening there. So I think the first thing is making sure that some of those real fundamentals are in place around access and ensuring that as people demand data literacy or people want to have data literacy that that's being enabled in a way that doesn't just become an expansion of lack of understanding that there's a way to make sure that they are actually becoming data literate and it's not just fake news, if you will, right? So making sure that that's a coordinated process. It doesn't need to be necessarily controlled, but it's a coordinated process. And then update what that means from an oversight perspective and recognize that it's okay to a certain extent a move to decentralization but I wouldn't really think about it that way. It's more of a move to the empowerment and enabling the business to continue to operate and optimize with the way that you also track your data and track how that data is being used. Some of this can be done via existing efforts like a data quality effort, a glossary effort, other sorts of things that may already be in place. So take a look at what's already happening so that you can then possibly support the new processes and capabilities. John, anything that you wanted to add to that? Yeah, as soon as I unmute, sorry about that. Just a quick touch on data literacy. You should probably plan next year for formal education in your organizations, even if you are well underway with analytics, well underway with big data, well underway with some data governance. We can almost assure you that the awareness of what this means and the literacy about what you can do with it is not deep enough. So if you want the process oversight, if you want to do any verification or all these other, you know, how to write out these trends recommendations here, you're going to have to get people up to snuff with what data literacy is. And that means just some basic knowledge about big data analytics, what are the capabilities, what can you do, what can't you do, things like that. So a little insertion there. And back to you, Kelly. Great, okay. All right, so let's think about what is our next trend. So data trends and predictions. All right, so let's look at some of the actual data trends. Well, this first one is probably one that's on a lot of people's minds, privacy ethics and regulations. So this was the year of GDPR, right? And so organizations were either ready or they weren't. GDPR happened anyway. And next year, folks are anticipating that the privacy regulations are finally going to be solidified, communicated, and in fact there's going to be enforcement dates that will be identified. So that's coming up. There's the California Consumer Protection Act that was legislated this past year. Apparently Colorado is working on one. And those are just the legislative aspects of privacy. Then we have ethics, right? So this is a huge category of work that's happening here. And in fact, there was a book that was written not in 2018, actually, but in 2016 about data ethics, the new competitive advantage. So this is really a place where I think people are going to be spending a lot of time in 2019 and leveraging how their data ethics policy is going to enable them to market that to their customers and their marketing trust and protection of information and that sort of thing. So that's a huge trend I see. Data acquisition, as companies are acquiring data from a variety of different resources, sometimes trusted and sometimes not, and data is being generated significantly via some of these new technologies that are on the downstream, like on the edge, the virtual reality, edge computing, et cetera. That sort of influx of data and how it's managed is going to be a priority for a lot of companies in 2019. So tying to the last one is that you can buy just about, or actually you can acquire, I don't want to say buy necessarily, just about any sort of data set on any category that you're looking for. And sometimes there's a fee associated with it and sometimes there's not. But there's all kinds of data providers out there now that are categorizing data for a specific use case, whether that's an industry, whether that's a business process within the industry, whether it's a demographic. But there's a huge growth in the amount of data that's available to people. So as people are building out their analytics communities and their data lakes, they're looking at rather than how do I slice and dice my internal data, what if I just acquired a whole bunch of data externally? What if I use some of these data sets that have been vetted by other people and that are produced particularly for my own use case? So that's something in 2019 that we're going to see as a huge trend. John, anything to add? Yeah. We're going to get to this on the next slide. You want me to move on to the next slide? Yeah, go ahead. We'll do that because we're moving along rather smartly here, but what the listener hasn't known is this particular slide here we're going to hammer on for a little bit. The data trends and we break this down first with ethics and privacy. All you have to do is study what's going on in England right now and Facebook. I don't know if anyone saw it, but yesterday Parliament released all the dirty little ugly secrets and emails out of their things. And when you examine those, I took some time to read those. When you examine those you see what a, it's not necessarily that there were errors of commission going on, but nobody understood that they were acting unethically because it just never enters our minds now. There are so many elements around being data driven. It's a lot more complicated than just a good password and privacy and access policy. It's whether you should do stuff with data or not. It's not, and it goes way above an individual access. It's an organizational capability whether the board of directors should permit something like this or not. So wherever you have data management or data governance in terms of analytics and big data, and there is this temptation to do really, quote-unquote, seemingly neat things, you have to apply an ethical lens to that or a regulatory lens. The statement that, well, there's no law against it yet so we should go ahead is probably legally correct, and we've all heard that, well, it's not illegal, so right. But the fact that ethics has come into this and ethics usually become legislation as we're seeing, as Kelly mentioned around the country, that there are some serious considerations that have to be done here. So do not take this lightly. Leveraging governance to drive data ethics then is really important. Many of us, if we take this scenario of some analytic environment or big data, we'll have the data scientist come to a data person and say, we have a data quality issue, let's fix the data quality, governance gets involved, blah, blah, blah, blah, okay, we've all talked about that. Let's go back, go back to planning. Let's go back to the board meeting where you're determining to approve a new line of business that's going to monetize your data. There has to be a consideration of ethical use there. Governance is wading into not only just annual planning, but into strategic planning or even the very nature of a business organization. So maybe you'll get a chance to deal with that next year. Maybe not either way beyond the lookout. Now when we talk about acquisition and external data sets, we're also talking about the technology to deal with all of that and all of them. And honestly, we said last year, 2018 is the year of metadata. Some of the products that have come out are amazing. The products that are out there are scrambling to be equally amazing. It's really easy to get caught up in amazing stuff on top of all this really cool analytical stuff. But again, are you doing it? Are you okay to do it? You really have to, we usually don't have to do things like that in technological pursuits. And then lastly, I was noticing some of the comments and things on our chat here. There's the year of metadata. The other ugly stepchild of data management has been data movement and integration. It's always been everything from just FTP of the file to run a thumb drive down the hall, which we used to call sneaker net any old days. I don't know if anyone uses sneaker net anymore. Most SISOs are pretty hard on that now. But anyway, we're also in a period of time where the architecture around data movement and data sharing agreements are going to get much more rigid. You're going to see some real discipline pop up next year and in the years to come around data integration. A lot of the tools are working much better. Virtualization is pretty much works. We don't need to do a proof of concept. I mean, all this stuff, how you move stuff around and get it and how people use it will be a lot more than just an annoying logistical aspect of your data architecture. So on that, let's moving on forward to... Well, Kelly, anything to add to that before we jump ahead? I'm sorry. That's okay. You know what? There is actually a question that it might be good to take right now in the chat. Oh, sure. I'll go ahead and read it. Is there a standard data ethics framework that you see emerging that you would suggest we look into? So again, when we're... As companies are determining how to make these suggestions here a reality, I think that's a great question. There are data ethics frameworks out there on the market. I mean, you can just Google data ethics frameworks. And there are government entities that have put them together for their own purposes. Some organizations actually have data ethics frameworks that they use. I mentioned the book, Data Ethics, the New Competitive Advantage. They talk a little bit about that there. So I wouldn't consider there to be an industry standard one. But I think there's a couple of things to consider from this perspective. A data ethics framework generally has those guidelines and principles that you want people to adhere to within your company as they make decisions around data. You can create a data ethics policy, but there's a principle between this sort of policy-driven or compelled ethical framework versus something that I would consider to be more where you want to create a data ethics culture and you want people to buy into the ethical constructs of data. And this is why that second bullet point of leverage governance to drive a data ethics culture, a lot of governance has worked in that way, such that they have created not just policies that compel people to do things, but also mechanisms to start to change the awareness around data. So use all of those things that you've done over the years. It's not just writing the policy, it's actually getting it to be used and adopted. What are all of your organizational change management techniques that you've been using over the years that you can repurpose here? So things like getting an executive sponsor. Who's going to be that executive sponsor that drives your data ethics program? How are you going to roll out this concept of data ethics? How are you going to get feedback on whether people understand the data ethics? How are you going to create awareness and get feedback on whether people actually are internalizing that awareness? Does it make sense to create a change team? So all of these things that over the years you've learned as far as governance, all of these things can be applied to creating a data ethics culture. Did you want to add anything? Well, in terms of just some nuts and bolts guides, healthcare in the United States has dabbled heavily in personal information for a long time and we have HIPAA. And that is actually kind of an ethical framework. It is manifested in regulation, but it is based on preserving private information. There are a lot of writings on it. Also, if you just kind of do the Google thing or look at a particular website that started out selling lots of books and still managed to sell a few books. In 2018, they released 10 new titles with ethics and data in them. And all of those books have some type of guidance as to how to get going. Although, Kelly, as to a formal framework, I haven't seen anything from the EDM Council or anything like that yet, but I do know they're noodling on all those things. Yep, got it. Okey-dokey. Okay. What's next here? What's next? Technology. Oh, technology. Well, machine learning and AI. Boy, if we didn't put this on, we would look like a bunch of sleepers. Yeah. I don't know about you. I'm getting tired of hearing about it. Anyway, this is going to dominate data management conversations at leadership. Rightly or wrongly, the big data and analytics was the hot topic in the boardroom a couple of years ago. It's now machine learning and AI and doing really cool things and all of that. It's also becoming a topic among social pundits as well as to whether it's a smart move or not. Either way, you're going to have a lot of talk about machine learning and AI. I think the number one bellwether that this is really important is a lot of people are talking about it that don't know anything about it. And trying to do stuff with it. And then they go, oops, that wasn't exactly what we thought it was. So, we'll get to how do you handle this in a little bit. Data virtualization. I mentioned this earlier as part of integration. A very powerful tool now in the technology toolbox in most organizations. You can go back three or four years and you'd have to try it out and see how it worked in your particular thing. With advances in the technology and a lot more people doing it, it is a very, very valid approach to gathering and presenting data to your constituents. Data from autonomous things means IOT type stuff. Obviously, edge computing and all that goes with that important quantum computing. Nothing will slow down Moore's law, I'm afraid. It looks like we're in for another five to ten years of Moore's law still holding steady here. So, it means a lot more and a lot faster. It's one thing to govern something that's going to make an impact once a month. It's another thing to govern and oversee something that will make an impact 64 times a second. Some of the conversations will obviously change. The last one is when you have a period that we're going through and a lot of you will talk to us about how's this vendor and how's that vendor and should I try them or try those? The fact is we're going to enter a period in the next year maybe two where you're going to need a scorecard to keep track of who's around and who's not around and who's bought who and who's merged who. And that is perfectly normal. It's gone on at least in my career two times, maybe three times across various upheavals in technology and it will continue. I'll just let Kelly chime in here and then she can then lead on and tell you what to do about all this stuff. I think that those are great trends, absolutely. There's always this reality check that we want to encourage people to always balance the AI initiatives with common sense considerations. The other aspect of this is that if we don't consider the data aspect of machine learning and artificial intelligence we can't learn and they don't become intelligence without the data that's fed into them. So we need to make sure that as we set up these new capability areas that we consider that they're learning on appropriate data and that they're learning on high quality data. This also actually ties into an ethical framework. You don't want to inherently bias artificial intelligence and machine learning based on the data set that you gather to feed into one of these intelligent engines, if you will. So, again, balance AI with common sense. And then, you know, one of the things, actually, John, you might be able to talk about this a little bit more, is educating the organization around data virtualization and how that fits into the technology toolbox. And in fact, the question came in, which I think it would be great for you to answer, to hear more about this than I do. So the question from Chris was, we'd love to hear more about data virtualization. For what part of the data pipeline is it most useful? So maybe if I could just expand on that. So what parts of the data pipeline and then also architecturally where do you think that it should fit in? Oh, really good question. Okay, so the best way to answer this would be to everyone, let me go up to the whiteboard and draw it. Exactly. And where's the whiteboarding capability? I don't know how to use that. I know it's here, unfortunately. But I think we can do, is everyone out there listening, imagine that you're looking at the whiteboard and we have one of our classic data architectures on the board. So source stuff is on the left and happy, happy analysts and users are on the right. And in between are a few layers of things or a few columns of things that happen. And there is always that from the source to some type of initial area, which could be called a staging area or a holding area or whatever you want to call it. Virtualization has a key role at that part in the pipeline because, well, you're dealing with a lot of different sources and virtualization tools have connectors or services that can go out and grab disparate data and bring it in. Now, that's pretty good right there and if you're dumping it into a standardization layer, say in Hadoop, so now we're kind of at the middle of our picture. It can just go in there. If you're going to be classic Hadoop schema on read, just let the data scientist well at it. There you go. And it's kind of no must, no fuss and the tools keep track of where they got stuff and some of them are coming out with some good lineage and some good metadata. So that's the next part. Now, the other part where the tools really got started, is farther to the right where we have an analyst who's trying to get some data and they have the data warehouse to get some and then they maybe have some stuff that's sitting there in Hadoop and they buy some external data and they still don't have the complete picture and then someone says, well, there's the 50-year-old operational system over here that has that data in it and that's another spot where virtualization can come into play where we can actually launch a query or some simple hypothetical model to bring some stuff together and try a few things out. So to summarize my virtual whiteboarding here, two places primarily in the data supply chain. One is sourcing and coming up with a fairly efficient way to source into some standardization layer. The second way is the classic use of disparate sources of data to satisfy some business intelligence or reporting needs in lieu of a wonderfully integrated great infrastructure which a few of us have. So that would be the two places you would use that. Great. Thank you, John. So our next response to how to ride the technology trend, so when we're thinking about all of the technology that's available to us and enabling decisions and enabling governance as close to the use of data as possible, this will impact the way that you create your governance and data management operating models. And so I think that it is an opportune time to start revisiting the way that those are structured. And if, you know, maybe you're in a phase of data governance where you might be losing some momentum or what have you, take this as an opportunity to revisit the way that you can govern data as close to data consumption as possible. And can you actually put in place some, I guess, awareness or some ways to make the usage of that data at point of consumption easier? Can you apply knowledge when people are actually building a report and that sort of thing? So that would be something to think about as well. And then our prediction for 2019 is that there is going to be some change in the vendor landscape. So whether there's going to be some consolidation, which we do anticipate that to happen, there's also a lot of tools that are coming onto the market because this drive for governance closer to the point of consumption is actually creating a market that technology companies are trying to fill. So as you consider that, think about how you want to balance this high volume of tools out there that are available to help in different categories of your analytics process. So, John, anything to add on that? The, oh, where was it? The federated governance, right, the planning. That kind of goes back to a point we made earlier about decentralized versus centralized. Maybe I can see what someone means by that, but it might not be the right words to use, but definitely since from the get-go, all governance is somewhat federated. Really 2019 is the year, especially, you know, if someone's been up and running for a few years, they're going to be able to revisit your operating model, for sure. So stuff is happening out there probably under your radar, or if you're just dealing with analytics and things like that, some things are moving so fast that you're going to have to revisit that. So that would be the one, I just wanted to emphasize that point. Okay. I think we just have one more slide here. Oh, there we go, yeah. And I think you might be able to actually meet our kind of internally imposed deadline and take some additional questions. Okay. So key takeaways, you know, again, pace of change is becoming a lot, everything is becoming faster. Change in technologies are becoming faster. The expectation on output is becoming higher. So this pace of change is going to impact a lot of the way that you'll go into 2019 with your planning. So think about how that, how you're going to not plan just for 2019, but how you're planning for 2020 and beyond, and consider what capabilities you'll need to develop in order to get to that longer-term goal. And think about how ethics plays into this, because your organization might not be talking about it yet, but your clients, especially if you're in a consumer-based business, are probably talking about it. So start to think about how you would leverage work that's already been done in a governance perspective into data ethics and consideration of how data ethics comes into play at point of decision, but also how it comes into play in all those different data management steps that lead up to making the data available to that analytics or to that citizen data scientist. And then the whizbang stuff, that's a ladleism, will always take the top headline. Sorry. I love it. But think about how this can be, you can take into consideration the fact that there is demand for new technologies and be realistic in the way that you build out your architecture to take advantage of these new technologies, recognizing that there might be some change in the actual technology vendor landscape in 2019. John, do you want to add? Actually, we're at our right at it, and we have a bunch of questions. So why don't we dive into those? Why don't I read a couple here, Kelly, and then you can take a shot and we'll work away. We've got at least four or five here to deal with. Let's see here. So first one, question related to programming and avoiding the development of discriminatory algorithms. This is especially concerning with collaboration and development of cognitive neuroscience, the use of AI and ML with deep learning. Who is responsible? Kelly, you want to protect first out that one? Sure. The accountability question. Oh, yeah? Yeah. So this is something where I would see that this needs to be driven, the accountability and the way that data is going to be used and algorithms are going to be written to leverage AI and ML does need to come from the top down. And I do think that this is why considering a data ethics framework is going to be important, because if you can create that ethical framework and you have the sponsorship from the folks that drive the organization, then you will be able to ultimately filter down this consistent decision making that influences how you source the data that feeds into the algorithms of what the algorithms do in order to create an output. But I would see that the responsibility for that structure at least is at the highest points of the organization. Yeah. Well, I would just add like so practical. So you're out there listening to this and it's going on and the CAO says we are now going to go into some deep learning work here and it's going to do X and Y and Z. The question you should ask is, has the board approved that? Is this an executive thing or are we experimenting? And then if there's an ethical consideration, most organizations have a policy where if you think something is not right, you're allowed to say, I don't know about that. We need to take a look at that. I would definitely in this coming year familiarize yourself with that and be prepared to deploy it if you're just down there working here doing your data thing and you see this go by you. I think you're going to have some challenges along that line. Next question. Actually, you know, can I just add something? Oh, sure, sure. I do think that there's a role from a data perspective and from an analytics perspective to provide examples to test the ethical framework, right? So one of the things that I think we could do as kind of the executors or the worker bees with our organizations is we'll be able to say, hey, here's a great example of either where an ethical decision has been placed in front of us or where there's a consideration that an ethical decision might be placed. And so surfacing up where there could be discriminatory algorithms, where there could be essentially outputs from machine learning that are biased based on the inputs, surfacing what those are is a big responsibility for us as part of the data management organization. Yeah, I agree. Next one. Agile data governance. How realistic is that? Well, it is. It's very realistic. That's my answer, Kelly. Oh, yeah. Absolutely. Oh, yeah. Yeah, yeah. No, you go. You go first. Okay. It's very realistic. Look, Agile, oh, boy, I'm going to try to limit this to about a minute here. The broad way to answer that is step back and look what Agile truly is at its true academic definition, which is fielding something that people can look at and say, yes, this is good. I can live with this and then moving on and refining it. It is not just doing stuff really quick little bursts. Okay. That is a manifestation of Agile, but that is not the philosophy. So apply the philosophy to data governance, which means what can we do to govern something and see if people like it? Well, that is a classic implementation of the work that Kelly and I do have done separately over the years and have done together over the years. We always look for agile deployment of data governance. That is fine. First of all, agile projects that you can govern. Second, find business use cases that we can implement immediately and then see how it works and then make adjustments in your operating model accordingly. So that mindset of Agile tends to apply to coding and to standing up services, but it can also apply to any process of any sort. So very, very, I go to Agile data governance first as an approach before I did anything else. Back to you, Kelly. I would absolutely agree. If you're not taking an agile approach, then you will end up with a lag in your ability to execute. So one of the things around the journey from a governance perspective and this is part of kind of an organizational change perspective is that generally most governance programs get kicked off with a ton of excitement and there's this increase along the cycle of both momentum and interest and quick wins are hit and everybody thinks governance is the best thing ever. And then when you're starting to tackle some of those bigger challenges and you're starting to get more business units involved in that sort of thing, there tends to be this fall into what we call the trough of disillusionment. And that trough of disillusionment can be long and can be deep. If you take a more agile approach and you look at the incremental way to improve the way the data is governed and you break down these bigger projects into more incremental deliverables, you'll be able to make that trough a lot shallower and you'll be able to get through it more easily because you'll see more regular outputs and regular success rather than trying to start with easy things and then take on a super big hard one. Well, that super big hard challenge can be broken down into also easier things. So I would like you, John, agile first. Yeah. A couple more here coming in. Someone asked a question about our mention of capabilities. Say more about the capability shift. I think what that means is we're talking about thinking about capabilities or the what versus the how. And why is that important? I think it's a sign of maturity because you start to worry about less how is this going to work and you start to worry more about, well, this is something I think we need to do. Do we need to do this something? And then we'll figure out how to do it. But I do think that it's more of organizations trying to get more mature quickly and get a bigger picture of what it looks like and what it is they have to do around managing the analytics and the big data and being data driven. Kelly, any comments on that one? I would agree with that. I think that the capabilities approach is that, you know, there aren't such a delineation between roles anymore and that people that use the data are very interested in the process up to consumption and up to analytics. And so taking a more capabilities approach considers the fact that there is a blend between expectations and skill sets in this citizen data scientist world. And then you're recognizing how to develop those capabilities across an organization so that when you move from, you know, not just project to project, but from program to essentially operationalization of these things like, you know, governance and, you know, analytics and things like that, that it is truly embedded within your organization. Yep. Let's see. We've got five minutes left here. Do you have some examples of data ethical challenges? Oh, yeah. I've got a list. Go ahead. Okay. And then I did see another question came up in chat. So I'll make sure that we leave time for that. Okay. So here's some good ones. There was, so I'm going to give a shout out to one of our colleagues, Deborah Henderson. So she told me a story about where she was actually at a utility. And so I'm going to present these as a series of questions and you guys get to kind of make your own decisions. So this utility is a monopoly and they are looking at analyzing the customer base that they have and for those customers that are constantly delinquent on paying their bills, they want to be able to identify and ultimately be able to say, look, if you don't pay your bill, we're going to terminate services. And so then they reach out to the data management organization and say, hey, data management, you know, we need to get our customer data. We want to be able to segment it, et cetera, et cetera. And so then once that comes to the point of decision, the question is, if you terminate customers that don't pay their bills, is that a problem? Right? So if you are a CFO or somebody looking after profitability, well, you know, probably not. You need to pay your bill in order to get your service. Well, what if you remember that it's the monopoly utility provider? There's nobody else that provides the service. And what if we remember that Deborah lives in Toronto and they have really harsh winters, right? So that is an ethical decision around are you really able to terminate the service of these potentially low-income households because there's no other way for them to get their electricity? So that's one example. And then another example is there was an article in Ink Magazine that was about all of the patent applications that have been submitted by Facebook. And I'm just going to give you a couple of examples because I think that they're priceless. One is there's a patent out there to be able to capture your expression when you look at your screen on your phone so that they can gauge your sentiment to what is occurring on your phone. So that's one patent. Imagine how that information could be used. Here's another one. A patent to be able to listen via your microphone on your phone to the background noise that's happening as you're having a phone call. Supposedly to be able to provide this information to the television networks, right? Who else would they provide that information to? So I mean, these are some serious considerations. Is that an ethical use of the data? Yep. Anyway. Well, and as I mentioned, if you actually, if you follow me on Twitter, I posted the link to that document that the Parliament of England released yesterday. It's all over the place. It's about 200 pages, but it is, whether you agree with it or not or whatever, but it just shows you how important it is to think about this stuff before you do it because things were done in the heat of the moment to, you know, through this mythical provide someone a better experience thing that were just blatantly incorrect, at least from my view. I'll just talk about my opinion here. So yeah, I mean, these are terrific. We have another one. You want to take a crack on another one, Kelly, here in the next minute? Yeah. So there's two more questions that came in. Go ahead then. Yeah, one that I'm seeing. I think that this would be good for you. How about trends connected to prescriptive analytics? Do we want to comment on any of those trends for 2019? Well, a lot of them kind of tie into, you know, the word prescriptive means, you know, do it this way. And way back in our series, remember we talked about the difference between descriptive and prescriptive and what everyone thinks initially was analyzed was really descriptive. You know, how did we get here and correlating and all of that. Prescriptive is looking forward. And that's really not much different than AI. Maybe you might want to call it a dumb version of AI. I don't know. But there isn't a lot different with those trends as there are with AI. And that is, first of all, do you understand business process aspects? Are you using the right data? There was a question at there is a talk in the chat here about the discriminatory algorithms. You know, are you truly objective in that? And of course that kind of ties back into data quality. All of this that we've already talked about are trends that apply to prescriptive and all analytics. I wouldn't segregate any flavor of this from any other flavor at this point in time. It all is what it is. So John, do you think we could do 30 seconds for one more answer here? I can talk fast. How would you deal with enterprise data in Agile software development? 30 seconds. I wouldn't let the Agile software developers anywhere near the data. I would have another department, another department set up a list of data services and business services only called data services. Nobody writes a native data service without permission. Thank you. I'm done. Sweet. Over to you, Shannon. Thank you, John. Thank you, John and Kelly, for this great presentation. We really appreciate it. And thanks to our attendees for being so engaged in everything we do. Just a reminder, I was in a follow-up email by end of day Monday for this presentation with links to the slides and links to the recording to all registrants. And thanks, everybody. Happy holidays. I hope you all have a great new year. Thanks, all. Thanks, everyone. Bye-bye. Bye.