 Thanks everyone for joining and just before I get into the content I do want to say it's just kind of on behalf of Microsoft We're real happy to be able to sponsor Europe Python We try and do this every year we can I think we're on quite a few years running at the moment This is a sponsored session. This is one of the benefits that we get in exchange for that. So thank you for coming along regardless Hopefully you've learned some stuff and you'll see some exciting stuff Anyway, so don't take that sponsored session as like it's just a sales pitch. I mean, okay I'm gonna show you our cool stuff, right? But I hope you're excited by that. We're excited to be here We do have a booth set up if you go and chat with them You may notice especially if you've chatted with us before that our usual kind of team of product people and marketing people have Not made it over this year. We we've got locals Prague actually has quite a big office for us here with a lot of developers there and so we rounded up the ones who are most excited to come to Europe Python largely for the first time and That's who we've got it up with so if you want to head over there and say hi to them They're just as keen to meet you and learn what you're doing as as they will be to tell you about what we're doing So go and say hi there. They would love to meet you some of the new features that are Available now or coming for visual studio code. I'm gonna be honest. This is not my personal work I don't work on the team that's been doing this and just in learning about what they've been doing my mind gets blown So so first warning as far as I'm aware and everyone keeps insisting to me. None of this is fake My first reaction when I saw some of it was why are you sending me fake stuff? I'm gonna get in trouble for this It's it's genuinely not this is this is all real. It's just it's not all released yet There's a couple of leaks, but but it is some cool stuff. So the basis of The the kind of Microsoft platform for Python developers these days is visual studio code Now if you're not currently looking at it on your own laptop in front of you chances are the person next to you is So give me a few minutes and look up from VS code But it is our open source cross-platform lightweight IDE and of course it has full Python support and it's where we're putting kind of a Large portion of our energy in trying to make sure Python developers are well supported have a great development experience Not not just across all the Microsoft's products, but but anywhere it's like at our at our core We're a developer tool company and and largely always have been So this is what we do we find developers where we can and enable we like to use the term empower But I enable encourage support developers everywhere. So this is what VS code is for and The the numbers numbers of users and the feedback that we get show that hey, we're doing a pretty good job This session is mostly going to be demos of new things and they're largely data science focused I still think they're pretty cool. Even if you're not a full-time data scientist because it's just a really cool stop in there But because because it's demo and we're doing data science we need data Here's the data set we'll be using because I need to show you this for completeness It's the Department of Transport airline data set. You may have come across it before it's available on Kaggle and it's basically a whole lot of rows of It's 200,000 rows in fact, I believe it's been cut short flights in the US Whether they are on time whether they were delayed one of those categories is bigger than the other and a whole lot of information about where they were going some interesting stuff to analyze and And and yeah, you'll see this is what we're working with okay Put on the hat of your new data scientists just arrived at a new job and you're going to start analyzing some airline data Currently the data stored on github. So we're going to clone this and this is just from the github interface I'm not sure if that's playing or not. I don't think it is there we go And we're going to go and clone it. So we go to this code button and expand that There's going to be an SSH URL in there for our get clone We're running on Mac here. So that's just fine. The HTTP would also work jump over in the VS code and clone that get repository pasted in and That's that's our starting point The clone time happens if you've used VS code, you've probably done this already This is core functionality, but it gets us a repository with the code and you'll see a couple of files show up there in the left There's our CSVs Now we don't just want CSVs we want a notebook and so we're going to make a new file and just call it Dot IP YNB and that has opened up a Jupyter notebook Chances are the first time you do this you'll get a little prompt saying you need to install an extension But once you've done that you have a Jupyter notebook and and you have that Analytics workspace that you're probably used to I'll talk more about it later on but this is real Jupyter. This is a front-end over Jupyter We haven't built some different thing Exposing the tools that you used to already Okay, so it's time to start coding Except we're too lazy, so I'm gonna go to the context menu and start a code chat with co-pilot That pops up a little thing right inside the cell where I can say hey Give me the code to load the flights.csv data set and that's gonna run off and just generate the code for me I can accept that and immediately I've written my first cell Correctly use read CSV without actually having to think too hard about it at all Can hit run on that cell And I believe that's just a shift enter so the usual keyboard shortcut right there And we can now see the live data set and inspect that Couple of things you will have noticed there. Hopefully Right now you can see the big tool tip telling us all about that function while we were typing syntax highlighting was taking effect immediately There were pop-up completions There were suggestions before it even popped up because this is the full VS code editing experience inside of Jupyter We haven't just kind of passed text back and forth. We've got the full editor there So all the things you used to having in the regular Python editor in VS code You also have inside Python cells inside a notebook inside VS code Now that's done. Let's more closely inspect what's actually going on We can see the data frame printed there with the head But if we want to see what's live, there's this variables Button up here, which is going to open up a window at the bottom that shows all the active variables Normally it'd be scrolling up and down to see which things you've set up and what they are We just have a live list here and every cell you run is going to update that We also have debugging support if you need to see how that came about you can go to any cell any line of code set a Breakpoint and then step through that cell line by line or that will run up until the breakpoint and Have the debugging experience of VS code but inside a cell for your notebook So you need to track down how something happened or why something is not happening correctly You've got the the tools built in right integrated directly into it to do that Okay, let's start on some actual analysis But we're going to come over here to to our co-pilot chat, which is one of the new features Because I don't remember how to merge two data frames. So I'm just going to ask it And I'll tell it that it's that it's in pandas and There we go. It's it's run off and grabbed a bit of information description everything that I might have found from I Don't maybe a minute or two or three of regular search engine searching But it's found it's so much quicker and it's in context. It's right there. I can ask a follow-up question To adjust that slightly and it's got the history. It's got the awareness of what documents I have in my workspace It's got the awareness of what code I've been working on and is able to factor that in to find relevant results chances Are I didn't even have to tell it to use pandas there because it it knows that pandas is in the environment It knows that I'm working in python. It can figure those out for itself However, you'll see here that the code that it's generated that isn't actually aware of The the variables that we're using or any of the actual data So it's not really going to be that helpful for specific cleaning up in our context I still need to do all the work and have all the understanding of what needs to be done to make this data ready for use So to make my life easier for that we're going to go to data wrangler The data set here will just look at the head again. We've got kind of the basic Pandas output headers a few rows of data And this button at the top that says launch data wrangler So data wrangler is a tool built into VS code or one of the extensions that is interactive data cleaning So we open this up. We now have 200,000 rows, which is the entire data set, but it's a sample Some summary information at the top of each column. We can see everything we can scroll across And have an overview of all the data Cleverly selected information at the top of each column There's a bit of a summary to the right so that that was the list of columns missing values and over the left We've got all of these operations that we can perform on the data to change things So we're going to infer a column by example here. We're going to take this date column and Add another column that's going to be the day of the month and we're going to do that using this infer by example So we create the column it adds it there immediately and we go and we type an example so in this one the the day the day of the month is 9 so we type 9 in as the example and Data wrangler goes off and figures out how to do that for the entire data set And so it can fill in the entire column calculate the entire column. We haven't had to write the code to do that It's been able to infer from our examples. This is the code that's needed to do that transform You'll see down the bottom that we have a list of cleaning steps each step is remembered It's not applied and then forgotten about the transformers remembered and we can go back and edit it later We can remove steps. We can reorder steps and see the changes as they as they are applied We can add a filter to that you can see it's real click around interactive You can see all the things that you might do So rather than kind of in the complete absence of anything you have to go, you know Column equals to would be no you can see it there and you can see oh, maybe begins with would be better So we can add a filter and say only where this column is this value and it shows which ones are going to be removed the highlighted and As soon as we apply that step they actually disappear from the view that we have here But now if we go back to edit that step They all come back and we can actually see the code that was created for that and even change it now It becomes a custom operation and you can see that a different set of rows are going to be removed because they're applied to a different different value and Our step now removes everything that is not the 31st of the month There's a really wide range of operations here A lot of inspiration if you're unsure what you need to be doing to your data Then even just browsing through is a great way to find what's available one of my favorites is one hot encoding Which is where you can take a categorical value column and Convert it into Boolean columns and so we've just done that for one of the the airline information fields here and we have a Set of columns where only one is true at any one point This is often far more useful for machine learning input than a categorical value And we can just do that and now we scroll down here and we see the code Data Wrangler is code first as I said it doesn't apply the transforms. It generates the code to apply the transforms Which means at the end of the day you can export it to CSV or you can click this back to notebook button Which is going to jump us back to where we started Copy in that code that was just generated and actually provide us a function that will apply that cleaning Transform repeatedly to any data that starts with the same schema And you can take that code and do whatever you want. It's your code. It's readable Python code It's idiomatic pandas as as much as we possibly can it's gonna if you do something. That's not idiomatic You'll get what you get And you run that and we can see that the result here is the filtered code. It has the additional columns And and that's that's your code So if you need to productionize that transform you can you've got the code for it if you need to modify it You've got the code right there for it. It's Not just a cleaning step. It's a preparing to clean forever step. It's actually setting up your pipeline Okay, so now we've done let's assume. This was our first job and we need now need to save it back to the repository If we come over to the the source control panel again, this is core vs. Code functionality We know that we've added a notebook, but you'll see that the diff here It's not it's not super interesting because it wasn't there before But normally if you differ notebook you get some Jason Unreadable thing we do a rich diff We'll actually extract each cell show the difference between the cell before and the cell after the metadata and the output are also Compared they collapse by default, but the full comparison is available And then you can commit that and synchronize it back to that get repository The metadata there has changed. That's the red. I think that's just formatting Okay, that was quite a lot. Let's I know a lot of us prefer slides with dot points to demo I Do myself so let's see slides with dot points rather than demo so vs code notebooks We created a notebook just by creating a file with the right extension this this works in vs code You don't have to choose from a huge list of templates If you know the file you want you just make it and it will Generate what needs to be there and then you can view edit and run of course you can open existing notebooks That's not a problem at all debugging is in there Viewing filtering values all all the things that would happen as I said, it's real Jupiter You connect to it. You can connect to a remote Jupiter server. You don't have to run it locally The key bindings are all the same and as we saw integrated with co-pilot And we'll see that again in a little bit Data Wrangler is one of my favorite things that we've done code centric data cleaning and preparation tool It's interactive. You don't just have to kind of imagine what the data will look like and then write some code You can see it You can click the buttons to see what the result will be and then get the code from that do whatever in you need to with That code it's built directly into notebooks one click to get into data Wrangler from a data frame one click to get back Really quick insights shown across the top of the columns Just to give you that sense of what the data looks like and a whole range of built-in transformations to make Coming up with the right choice a lot easier for you and of course good hub co-pilot The the big hot thing right now all the all the large language models Fundamentally changing how we interact with software very exciting natural language to describe what you want to do The first thing you're going to think of when you decide to do something is probably in natural language Most of us don't solve a problem directly into code We solve it into a description of what the code will do and then translate that into code You can skip that second step now or you can at least start from that first thought of what you'd like and get some more inspiration for it and Co-pilot does a whole lot more than we showed just here. It can convert comments to code It can generate comments for code. It can explain code to you. It can fix code May it may be you trust it to do that It can automatically write tests and it is getting better over time. It does Keep getting more training more Tuning to produce better results as more people use it and as we get a better sense of what code generates well and what doesn't So that's some of the big revolutionary things, but I have two more cool things that I want to show because Can't just leave it there. So we're back in GitHub and The the pop-up is still up from where we left off like it's we haven't gone anywhere really But we're going to refresh the page and we'll see that that notebook is there now and Let's imagine where someone else on the team someone who doesn't have VS code installed But still wants to see this notebook if we come to this code button Instead of looking under local we can go to code spaces now code spaces is a hosted development environment attached to your GitHub if we start one of these what you'll see happen is We get back into VS code, but we haven't actually left the browser this is entirely hosted in the browser running running locally and Connecting to a remote code space hosted in GitHub that has Whatever definition your project specifies actually it's it's based on dev containers, which means you can put Specification in your repository to say hey when you start a code space put these files in it install these extensions pre-run these commands and We have in the browser a live Python environment that we can interact with And execute code without having to install anything locally at all and you know, it can be different operating system can be Different set of packages whatever you need for that project really really neat way to quickly start interacting with a project Incidentally the the CPython repository recently got support for this and so for those contributing to CPython That that github.com slash Python slash CPython is enabled for code spaces You get in there and it's already cloned to the code. It's already built the code once Which means incremental updates as you change anything. It's really really great way to get started quickly And the last thing I want to show is a bit of a leak bit of a leak This is not released yet. It may change may change before it actually comes out But it was too cool to not show you all We're going back to co-pilot and we're gonna ask for just some general help because I've got this data set. I'm not I kind of know what I want to do But we're just gonna say hey, hey co-pilot if if I want if I want to predict flight delays because It'll probably just say one but you know if I want to predict flight delays which psychic loan model would you recommend and You can ask a general question like this and it'll go off and come back with recommendations And it's going to suggest linear aggression is is the easiest way to do it and you can say show me code to do to do that and it will go often and Find us here's some sample code Tells us what to import gives us a bit of an overview how to start it And we can go click this button and it will insert it directly into our notebook as a new cell Now as I pointed out before it's not actually aware of any of the other code that we have in here so we don't have a variable x or a variable y and It's not going to predict anything because it just doesn't it's it's just not aware. It doesn't know our data So what's new and exciting is at the moment at the moment it's shaped like this it may change We have this co-pilot button up the top that pops up a little list of our live data And we can pass that into co-pilot That tells co-pilot here's our actual data set and we can just say something like tell me about my data set and it looks at all the column names and it finds the information and it's able to describe it back to me in In human terms in in English in prose It can suggest some questions. What's the average departure delay for each airline? That's the thought Then you go through the processing to write the code or Now the co-pilot knows about the data It's able to know that to group the data frame by the airline column calculate the mean of departure delay Can show you the code to do that in pandas using the right variable names and the right column names And then it can just go off and calculate it and show you the results I'll be honest. I saw this and I was like you can't you can't get me to do a fake demo It's not Which airline is F9 We're just diving deeper into it with the kind of question you might ask You know the senior developer who hasn't written any documentation But co-pilot's going off and so here I know I know who F9 is here's all the details And yet the airline is actually frontier airlines. It's it's got access to the information. It's got the ability to search the internet It's even told you that searched online for it And we can even ask it to plot it has enough information to say plot the average arrival delay We'll go off and think it'll plan out how to do that and In this case, it's going to actually immediately insert the code into the notebook and run it and give us the plot of the average Delay by airline as well as are on the side of the description and the code snippet of how to do it You know, I've been working here a day and I quit right? It's like I'm not needed anymore this AI looks like Such hype Really doesn't it? All the all the chat GPT and everything and we've all seen that, you know The the pointless kind of Twitter posts and discussions like oh, I made it remember this thing I made it say this thing. This is operating at such a different level from that because it's taking your Written language and converting it into a plan and then executing that plan It's not looking at its memory It's not looking at something that it found on the internet 10 years ago and got you know trained into the model It's looking at the meaning of your words turning that into something that it can do and then bringing it back. This is Such this is the way forward. This is the way that large language models don't free us from knowledge They don't freeze from writing stuff and putting it out there They don't freeze from actually thinking about stuff, but they freeze from having to type it They freeze from having to search for it They freeze from having to break down a high level concept into a series of steps and execute on those steps So this this for me, I think is where it gets really really exciting Okay, what next where do you get it? As I said chances are you're looking at VS code in front of you right now. Thank you. We appreciate you That you can find more information at any of these links. They're all ak.ms slash VS code that's Jupiter slash data wrangler or VS code dash co-pilot Easy enough to search online for these things or find or choose your favorite AI search engine and type it into there And I'm sure I can tell you all about it And with that I'm going to leave this slide up. Thank you very much for coming I hope you're enjoying your Python and Feel free to come and speak to us at the booth whenever whenever you like or if you find me floating around I'm happy to chat with you about anything at all. Thank you. Thank you very much. That was an impressive demo We have time for questions. I want to remind you that In the back, there's a microphone So if you have a question here in the room, please go to the microphone If you're a remote participant or you just don't want to go to the microphone You can also ask questions in the discord channel of this room So please start Hi, so first any ideas for a new career for a data scientist But more seriously in the Copilot output there was These evaluation results of this code. It was just plain text So either any way to know for sure that like this Data we see in the co-pilot chart. It is actually executed our code and not hallucinating the some answers I don't know that I can give kind of a Definite answer to make you feel confident about that because I Like I can't show you how it's implemented, right? So I can can tell you that When it gives you the plan of what it's going to do and shows you the code that is going to execute That is actually part of part of the the integration side of things So those final results those calculated results are not because the entire data frame has been sent to co-pilot And it sent back the results it sent back the code and the code has been executed So the the actual results that it's showing there if you took that code and stuck it into the notebook and ran it You'd see the same results as what it was printing out in the co-pilot chat We're just saving that step because maybe you want to see the results of it and not necessarily keep it in the entire notebook history, so those ones towards the end are being executed locally and they are going to be The right the right data because it's using it's running the code that you see there It's not hallucinating those results. Maybe hallucinating the code in which case you you know, you you get what you get but you can see the code and You won't be out of a job as a data scientist You'll simply have more time to relax while your code is generating Since with data scientists have never had to wait for it to compile, right? It's like you know We wait for it to install and for model training and training. Yeah Yeah, so so your job becomes waiting for model training waiting for the AI to generate code for you and and you know Plenty of time to relax and then check the results, but that's that one's never going to go away We're always going to have to be checking the results that come back Yeah, as long as the bosses don't know about how much waiting I do. Thank you Thank you So hello, thank you for a nice representation and demo We at our company are very enthusiastic about going to use co-pilot But we have some concerns about how can we use it at our customers? So can you tell us a little bit more about what happens to the data that we send to co-pilot the data or the code? I can only really tell you what's publicly available on the site because I Don't work directly on co-pilot. So I don't really know anything more besides what's discussed Publicly, there's a really good FAQ on on the github documentation about what happens with the data that fundamentally it gets encrypted in transit processed and Stored in accordance with whatever, you know, data privacy regulations apply It's definitely very restricted. It's like I I I have absolutely no way to get to that data at all And I probably get fired three times if I try before I get anywhere near it because it's just so far cut off from anyone who's not Directly needed to touch it. So it says it's as restricted as safe as we can possibly make it That's about as much as I can tell you because I I honestly don't know I will say that there is an email address in that FAQ which Feel free to email them and and ask specific questions if they're not answered in there. So thank you I'll squeeze in a remote question Joardo Aldino is asking. Thank you for the talk our data wrangler and co-pilot tightly coupled over Would I be able to use data wrangler without co-pilot? You can absolutely use data wrangler without co-pilot the the inferring by example is Slightly older technology that we've had for a while if you've used flash fill and excel It's based on the same thing as that which all kind of predates co-pilot Doesn't require a network connection doesn't send anything out of your computer at all. It's all running locally Here is a rudimentary question from the beginning of the talk You open up the Jupiter notebook inside of VS code and you said this is an editor window My question is when you're editing text inside of those little windows inside of the cells of your Jupiter notebook Whose key bindings do you see are you seeing this codes? keybindings or jupiter's key bindings or the web browser's key bindings I Inside the ISO if you're inside the web browser, then all of those key bindings get overridden Okay, which so it's a hosted web page inside of the studio, and it's not an actual editor window per se It's so it's going to it's going to be a sensible hybrid of Which of which shortcuts you get the one the ones that for kind of manipulating notebooks Going to operate in much the same way the when you're inside the editor actually editing The the shortcuts that Jupiter uses are not the same as the VS code ones and I believe again I didn't I didn't personally do the work on this to figure it out But the the team will have gone okay this shortcut from VS code is more It's better for this one to be consistent with the normal editor versus better for this one to be consistent with the Jupiter Experience so if I set control w to delete word backwards like the Unix shell That does work or that doesn't every single shortcut is overriding and if you override it it can do exactly what you want Okay, great. Thank you. Thank you. Please give another round of applause for Steve