 Okay Welcome everyone really nice to see you very very happy to be back at slush, especially this time with Arthur My name is Paul Murphy. I'm a partner at Lightspeed based in London Just a real quick bit about lightspeed for those that don't know We have actually we're a Silicon Valley based fund, but we've been investing Europe since 2007. We have over 30 companies now in Europe and Yeah, we're investing in pretty much every sector In every stage We're talking about AI today and I think it's important to put some context around that from our perspective We actually have been investing in AI for nearly a decade. We have about 50 companies And I've invested over a billion dollars into the category and that context is relevant because When we met Arthur and his co-founders We thank you. We immediately fell in love with the vision of mistral And so I thought that the best place to start would be To ask you Arthur to tell us a little bit about what you're building at Mistral Well, thank you very much slush for the invitation. Thank you. Paul as well. Um, so yeah, we started mistral Six months ago with Guillaume and Timote and our vision was that we Wanted to make the foundational models a bit differently from the other companies We've been in the field for a most a decade now And we've seen it go from a cat and dog detector to something Is very close to being human like intelligent or at least looks like it And we knew that we've a very dedicated team. We could develop state of the art models very very quickly And we could actually take the field Into something that is that would be more open where we would give more access to developers So that they could specialize the models make them their own make them as small as possible to solve their task And for us the good way of doing it and the good way of starting that was to Sheep the best open source models Create models that would be very easy to to use by individual developers and from then on build onto an enterprise play to sell the platform that allows developers to take large language models and to make Them their own to to create some differentiation on the application they're making And that's a differentiation which is currently hard to do when you only access apis of a couple of providers But if you have a deep access to the models, you can create things that are much more interesting And this is what we want to enable So when we we led your seed round It wasn't that long ago You told us that your first thing you're going to do is to build your 7b model And then I think it was it was like three months from when we signed the docs on that round We got our message Saying hey, we're ready It's ready and it was faster than we had expected. It was already incredibly ambitious I'm just I think everyone's probably wondering how you did how you did that so quickly Well, I think the secret is to have a good team So we were joined by our first employees Dozens of them at the beginning of june and nobody took holidays We recreated the wall what we call the machine learning ops system. So That's actually very simple. You you need to create a very good training code base you need to create a very good inference code base to To deploy the models you need to be able to evaluate the models And the one thing you do need the most and where we actually dedicated 80 percent of the team on For three months is to have some very good data sets We we went to the open web to public domain knowledge Created it so that we could just get the best of it Filled it did everything to get something very good Did some work around how to better optimize the models and combine all of this And then train the model to get the 7b and we continue doing it With the new models will be soon announcing When you say it's fairly easy I think maybe some people would disagree with you on that But you definitely made it look easy. I think that's true So i'm curious The community you know has been very engaged with 7b models since you released it I think it was you know trending on hugging phase for multiple days top, you know top top models What kinds of things have you seen that have been interesting so far from the community? So we've seen I think thousands of derivative work. So Uh Developers that took mistral 7b and fine tuned it on their task or on their data sets to make it special So we've seen new capabilities to like longer context Better instruction following capacities. We've seen uh Like new topics. So we've seen like occult specialized models able to talk about Post-test experience and the like much better than what mistral 7b was able to do before So many kind of different applications, uh, some of them Useful some of them are just funny Um, we've seen integration in a lot of llm open source projects. So the open source world around generative is pretty It's pretty involved already. So you have retrieval augmentation systems You have projects that allow to deploy the models on your laptop You have all of these things and they adopted mistral 7b very quickly And I think it was the field was really missing an actor that would produce the best open source models and actively engage with the community And that's what we we uh, we're we are enabling. Okay. And so now the 7b is out there. What comes next? So we have Nothing announced yet, but we we do have things in house that we'll be announcing before the end of the year new models, uh, new techniques, uh, and obviously The beginning of a platform. So we're actively working on the product We'll be soon offering hosting capacities for our models. Uh, we've very fast influence Capabilities and yeah, that's for very soon. Okay. Oh watch the space. Um, so You're also while you're doing all this incredibly What I think most people would think of is quite challenging technical work. You're also building a company And I know that's not easy. I haven't done it myself before What's keeping you up at night right now? What's your biggest headache? Um, so hiring is obviously a very big challenge I think the only reason why we got there so fast is because we hired the best engineers and the best scientists in the world It's a very competitive landscape Europe is full of talent, especially the junior ones. Uh, and so we We are uh, this is some like a very big preoccupation for us. I'm constantly working on it So that's one thing. Um, the other thing is like creating the community engaging with it So we started with them with mistral 7b, but we really need to Yeah, well facilitate the life of our users Have them engage have facilitate upstream contribution Facilitate the emergence of ideas that we could help enable. So that's another thing We have a lot of, um, I guess policy matters That we did not expect but obviously this is an agenda that you don't select Um, there's We so there's there's different tracks you have in the u.s. You have any you um, we've been uh Vocal about the fact that we wanted to have hard regulation on the product side because it's very important And we see ourselves as the provider of tools and a big enabler of compliance for the application makers So we've been saying that, uh constantly and and and we've seen like the debate, uh progress on these topics And so this is something that yeah, we're very keen on trying to enable from a technical perspective because it's important that you have Technical founders that participate in that discussion. Uh, and so that that has kept me up at night For a while and I think you know the ambition is certainly to be able to build something that could rival other large companies like open ai And i'm just curious What do you view as a differentiating philosophy or approach to companies like open ai? I think a differentiating philosophy is that we really target the developer space and we really think that When you're making an application that you want to put into production You do want to have several specialized models that are as many Chips you should you should see them as chips that you assemble in an application And it's actually not easy to make a very good chip for the use case you want So you can start with a very big model with thousands of billion well with hundreds of billions of parameters It's going to solve your task maybe but you could actually have something which is a hundred times smaller And when you make a production application that goes at scale and target a lot of users You want to make the choices that lowers the latency lower the costs And leverage the actual proprietary data that you may have and this is something that I think that that's not the the the topic of our competitors that are really targeting Like multi usage very large models agi We'll take a very much much more pragmatic approach in enabling super useful application today That would be cost efficient that would be very low latency and that would enable strong differentiation through proprietary data Okay, and you've talked I think another key difference. You've talked a lot about open source as being a corporator your dna And I think question I sort of wanted to ask Arthur by the way wouldn't look at these questions beforehand. So he wasn't expecting this one, but I understand the concept of open source software. I think we all do we see the code you kind of can take it modified And use it But in the world of ai and and models The concept of open source just feels like it's maybe a bit different because actually some things you do keep for yourself Or you have to What does open source mean in the context of llms and ai? So we don't really call them open source The models we provide are open weights. I think it's important to like keep a good distinction between the Like the terminology we were using for software and the terminology we were using for models If you provide the weights of a models you're enabling modification. You're not necessarily enabling like Full understanding of what's going on But even if you do provide full transparency on the datasets and shining you don't know what's going on because it's it's a bit Opact by design so it's an empirical science when you create a model The only way to verify that the model is doing what you expect is to measure it with with evaluation This is something we'll be enabling and then it's to modify it with some signal coming from either humans Or maybe machines to to modify the model. So really the modification part is super important for differentiation And we are taking this approach There's a full open source approach which I think is very valid as well for science In which you'd expose your datasets to disclose everything That I think that's that's something that we would strive toward at some point But obviously it's super competitive and the dataset part is very hard to to obtain It's also very capital intensive. You need a lot of gpus. So right now we're taking a balanced approach in between what we opens what the the open weights will provide the things we keep for ourselves to to get a competitive edge And this is going to be a dynamic play and we expect it to to evolve with time and with technology Okay, and then does the there's the open weight approach help with other challenges like biases and control Yeah, so it helps with better key two things. The first thing is that you can Modify the biases you can have like a strong and fine Modification capabilities on the editorial tone on the orientation And alignment of the model. So we allow alignment of your own models to your own values and those can slightly differs Like fine control of biases goes through fine Deep access to models. So that's the first thing the second thing it allows and we've seen it We've active engagement of the AI safety community in particular around open source models It allows to have better interpretability because you can see the inner activations of the of the models And and that tells you things about what's happening About why the model is taking a decision and not another. So why is it outputting a word and not another? And so in the interpretability world, it's also super useful It's also and I guess the last thing is that it's very useful to do red teaming Because you have a deep access to to the model and so you can try to verify the the part of it Which are a bit failing or behaving unexpectedly and these are things that you can then correct very similarly to What we've been doing in the open source software for security cyber security. Okay, and then what I mean What is sort of what do you as at stake here? You know, why is this is this in other words? Is this the business advantage from Israel? Or is it something more fundamental that you see as almost a responsibility? So it's both a business advantage because we allow further customization and differentiations And it's a very mature market and we expect that on the application space The one actors the actors that are going to survive and create some value are the one that will be able to strongly differentiate themselves And so they would need deep access to models. So that's a business differentiator Then there's a bit of an ideological differentiators in the sense that I've been Contributing to open source for 10 years, give mentality as well We really think that AI has been accelerated by open science by the circulation of knowledge And that's how we went in 10 years from something very very Well interesting that would just detect boats and so something that actually Will speak the human language so This has been allowed because you had big tech labs you had the academia as well that was all of them were communicating at conferences every every year and and Information would circulate and that accelerated things and suddenly In 2020 openly I decided to stop publishing and it was followed by its competitors Very closely after and so ever since 2022 We haven't seen like major advances in LLM publicly announced. And so we've seen Currently, there's like new architectures that are used internally by our competitors and that are not available out there This is something we will correct very soon. Okay, great So I want to shift focus now talk about something you mentioned earlier, which is regulation And it's a topic you kind of can't avoid. I think if you're thinking about AI A lot of focus within europe and in the uk And I think you're at the safety summit in the uk the ai safety summit last month there's a lot of ideas out there and I think You know curious to hear your view as to what should be the priority how should regulation be prioritized and instrumented Yeah, so I think it's quite interesting. It's a very interesting topic for me and and we've been Yeah, we've been contributing ideas The one thing that I would start with is that we've been talking about regulation and safety And mixing concepts very heavily. So there's a matter of product safety, which is Answering the question of you deploy a diagnosis assistant in the hospital You want it to be Safe you want to be able to measure whether the decision it's making is actually sound is actually correct So that's that's what we call product safety. That's something you have when you buy a car You have product safety of your car and it should very much be similar for applications That's one thing and ai to some extent creates new problems because you have Models that are not deterministic and so they behave in a potentially unexpected way So it's useful to refine the hard laws that we have around product safety regulation Now there's another topic that came up which is national security So the question of whether the llms that we're training the lm that everyone is training is spreading too much knowledge So when you have access to llm, you're effectively able to educate yourself on many topics And this is something that is a concern For different actors because you could have like small groups that are deemed bad that could use this knowledge to do bad things So this is this has been at the essential topic especially in the us We're still lacking a lot of this absolutely no public evidence that llms are facilitating anything. So We're really we've been advocating for for some empirical grounding of the discussion And this is something that's currently very much lacking And then there's a third thing which is kind of mixed with all this with with the two first which is existential risks So knowing whether the technology we're making is effectively on an Unbounded exponential that will end up destroying us because as every exponential it kind of Breaks the limits at some point and and that's well it becomes ill-defined as we say in mathematics So this is something that for us it is very much science fiction. That's empirical evidences So what we've been saying is that We should really focus on the first topic which is imminent It's something that is we do need to have product safety on ai because it's it's going to Otherwise it's going to break trust in the technology we're making and so we want to enable that On the second part. We are lacking empirical evidence, but I think this is something that we should monitor closely knowledge Historically knowledge the spreading of knowledge has always had more benefits than than than drawbacks and we are ai is not different in that respect But still it's something that that could do with monitoring because it's really a new technology on the third aspect of agi and And the like and the fact that you could have an autonomous system that would go out of control This is something that we are not that is discussing because we really Think that as scientists we are lacking evidence of any existential risk We think that it pollutes the discussion on the first aspect which is super important Yeah, and so if I just kind of make sure I understand this right the view is that The application layer is probably the one that has the most responsibility in terms of safety at least to Consumers or end users, whoever that is businesses But that perhaps the models could provide that as a feature or functionality But it's not the responsibility of the model to ensure that the ultimate data transmitted is itself safe Exactly. So we think that the correct way of putting some pressure on the model providers Like us is to effectively say that any application which is deployed and that includes the application that we deploy Should be should meet a certain number of safety standards. So they should Do what they're expected to do and if you do that then that means that the application providers We'll be looking at model providers that are controllable enough that can give some form of guarantees That can give some evaluation tools around the fact that they're controllable and that they do What they're expected to do. So you have some form of second order pressure that is put You put pressure on the application layer and that puts a market pressure on the foundational model developers and that's the correct way Of making a healthy competition in making the most controllable models and making the best evaluation tools making the best guard railing tools And we think that it's a much better way of doing it Than applying directly a pressure on the foundational model layer because if you do that Well, you're you're in a ill-defined territory because you're trying to Control something which is by design super multi-purpose very akin to a programming language So you can't really regulate the programming language because you can do anything with it And so really there's a problem of definition And then there's an operational problem of the fact that if you put some heavy pressure on that layer you're effectively Um favoring the big actors that have a lot of compliance capabilities and you're You're making it harder for startups with innovative ideas to to come up and compete and so This like foundational models is a bad proxy for market capture. And so we believe that Applying the regulation pressure on the application layer is the one thing to do Because that's going to foster competition and provide a safer world Do you think that there's a role for an organ, you know an IAEA kind of like organization to exist that helps to enforce or provide this guidance regulation So yes, I think um This kind of regulation if we need to Monitor I think we do need to have empirical evidence of what's happening in the space And we need to monitor the product side safety and one way of doing it is to enforce that We have very very independent Organisms that actually monitor these things and when I say independent, I mean That we should be very very cautious of of preventing pressure and regulatory capture of these things So setting standards but ensuring that No big actor is basically writing the standards themselves So what that means that if we are if we if we need to have this this form of organisms They need to be very well funded probably state funded and being Completely screened from pressure from the industry. Okay. So now I want to shift, you know, I think the regulation debate is largely Many of the debates in AI are Tend to sort of skew somewhat negative. So Let's dream for a second. Like how can AI make our lives better? What do you see as the utopian future with AI So I think the there's so there's many verticals in which AI and like interacting With machines with natural language carry a lot of value So healthcare is going to be completely changed by AI because you you will be able to interact With empathic beings that are actually super well grounded on statistics and that's really what You are expecting from medicine. So we expect that AI is going to empower Physicians to be much better at what they're doing and to make better decisions Education is also a super interesting topic Personalization of education we know that it's super important to Take the most potential of human beings and having some like your individual teacher being an assistant This is going to change a lot of things Especially in the global south So that's two things generally speaking. This is going to change the way we work So it's a way the fact that it can interact with infrastructure knowledge and that it can do Well act as if a bit imitate the boring task of of your daily life This is going to enable more space for creative thinking So we will be able to think more creatively and that's going to unleash I think a new society very soon And if you think about some of the more existential risks we face in the world like climate change Do you think that something like that can be addressed or at least improved? Yes, so I think this is a frontier which which hasn't been completely addressed yet But this is really a promise of having better models The fact that if you if you have some ways of reasoning around a pool of science Well, you can enable scientists to come up with new ideas. You can potentially unlock very precise things like Like in chemistry in accelerating Chemical reactions so that you emit less co2 for instance these things like material science chemistry nuclear fusion as well all of these locks that we have and that are That we basically need to break in order to address climate change Well, I mean, that's one of the way you can address climate change obviously the one way is also to reduce consumption but the These things we think that AI is going to be an enabler of Breaking these locks. It's not going to be an easy task There's still many things to invent and we think that going through the open science part fostering The the keeping fostering the AI community that drove the field forward for 10 years is great is super important to break these locks Okay, that's great. So I think I want to come back to to europe sort of for our last question. It's we're out of time How I mean, I think The fact that the company is being built in europe is very important to you It was obvious to you and your co-founders when we invested How important do you think it is for the industry that we have a european champion Emerge in the field of AI so the technology is AI Generative AI is is really a wave you can it's going to change society quite significantly And in europe we have a choice of either being on top of the wave and driving the technology forward or just Looking at it happening in the u.s. And in china And we think that in order to shape the technology to our values and to the way we think about democracy about society We need to have very strong technological actors that are able to drive the field forward make proposals Both in terms of policy and in terms of technology and so that's why we believe it's super important that actors That we have strong actors in europe great. Thank you so much arthur really appreciate it amazing you