 Hello everyone. I'm very happy to have you here. Welcome to the last session before the closing ceremony of wikimaniya 2022 I'm super excited to be here. Super happy to have you all here Even though I have no idea who you are because I don't see anything face to the virtual event Um, we want to talk about wiki functions I will give a very short introductions into wiki functions before I let my team talk about it. I have a few fantastic speakers who will talk about different aspects of wiki functions And then we'll go to a Q&A sessions at the end What is wiki functions? wiki functions will be a new project of the wikimaniya movement the first one in almost a decade to start We it's a wiki for functions and functions can do things like Calculate things Add information to existing data and so on so To give examples of functions is for example given a date of birth you can calculate The age of a person or given Population and an area you can calculate density of the Of the place and so on and all these things can then also be used in the wikipedia And kept up to date for example based on data and wiki data So we can draw the population directly from wiki data Make a calculation on it and display that and the wikipedia This way we can keep more information more data up to date And share it also across the different language editions of wikipedia So um you can call those functions instead from the wiki you will be able to call those functions instead of the wikipedia So the other wikimaniya projects you will call those be able to call these functions in other modalities like just wiki functions directly itself Our plan is to keep working on it. We just launched our beta version We're not there yet to really Kick off and launch wiki functions there. We'll do it when we're ready. I hope this will happen in the not too far future but I don't want to commit on a date, but Just make sure that we are getting a suitable point where we are Satisfying with the products that you can launch. We will start launch with a small product just as we did with wiki data A decade ago. So it won't have all the functionalities yet But we rather will grow then over the months and years to add more functionalities and to allow for more Possibilities of wiki functions, so we will start with a very minimal project It won't be integratable into wikipedia yet it um and all of this but It will show it will show you the first direction of where we're going to go One of the main goals of wiki functions is to also provide functions that generate natural language text that natural language text is then um Meant to be a Foundation a baseline of knowledge that will be available in all the different languages of Of the wikipedia's so that we can write an article in an abstract rotation this is better with abstract wikipedia comes from abstracting from a natural language And translating it into natural languages furos functions wiki functions and make it available in the 300 or more languages that wikipedia supports So this is the rough idea. This is still at the go. Um, I'm very happy now to Let the team members speak on certain aspects of it functions. I'll see you there after that for q&a If you can roll the video, please. Thank you Hi, everyone. I'm james forester. I'm the lead engineer for abstract wikipedia team working on wiki functions I've been a staff member for 10 years and a volunteer for 20 years on the wikipedia projects And I've been working on abstract wikipedia since we started in 2020 so An overview of the technology is that we've kind of split the code into two parts There is the bit that you interact with directly that Displays your functions your test cases your implementations and your discussions about whether this is Hi, everyone. I'm james forester. I'm the lead engineer for abstract wikipedia team working on wiki functions I've been a staff member for 10 years and a volunteer for 20 years on the wiki media projects And I've been working on abstract wikipedia since we started in 2020 so An overview of the technology is that we've kind of split the code into two parts There is the bit that you interact with directly that displays your functions your test cases your implementations and your discussions about whether this is a good idea Your community forum all that stuff all of that's in media wiki all of that is Hopefully as familiar as possible within the confines of it being a totally new form of content for community members to be working on And then on the back end we have services that actually take your code and run it based on requests so the front end is media wiki extension wiki lambda which was The first kind of code name for the the project before wiki functions was community picked as as the project name The wiki lambda code is media wiki extension So the the content is stored as jason inside media wiki. You can view edit interact with the content, you know look at diffs write abuse filters All that on media wiki in the normal way The actual kind of front end is written in view js Which gives you a very rich experience for doing complicated things like creating a new input of a particular type that is live created every time you input a content and setting a label for it in 12 different languages out of the 500 or 2000 that we potentially support Then on the back end We have two services One of them the function orchestrator, which is kind of user facing potentially and uh, it's where you can request hey, I'd like the Function to use this piece of data from i'm james farrester. I'm the lead engineer for abstract liquidity working on wiki functions I've been Runs through that to that wall to have some serializers and deserializers in in certain cases for depending on your content type And then pass that all the way back up stack that way you You're not running user written code on the same service that have access to the databases and things like that Um, so there's kind of separation of concerns, but also that's kind of our general set I'm a designer on the abstract team currently working on wiki functions And today I would like to share with you a small update on how we're approaching design functions And for the purpose of this update We're gonna look at some of the objects that are currently available on wiki functions And how the display and how the visualization of this object is going to change Following a new design language so for example, we're going to look at this mushroom object and from wiki functions objects they usually have A custom way to be visualized But it can happen that certain objects don't have like a custom component To be visualized so they're going to fold back to a default component So everything that you see here in this dash line is a default component Sorry Visualized so they're going to fold back to a default component So everything that you see here in this dash line Is a default component because in this case this object doesn't have So I'm trying to screen share again now instead of link But it doesn't allow me to custom component to be used for its display And so we can see that this object mushroom object has type has an identity Has different keys or you know, we could call them attributes or properties Like it has family it has a species it has a genus And also the creator of this object decided also to add a list of common names And then here on the right. We see another object, which is This mushroom type of mushroom Caldor Vista subsculpta And here you can see that the creator used the previously the previously defined object to visualize Or to define what is the family what are the species and what's the genus and what are some common names and I apologize for what you see here It's currently about we're working on it, but imagine that you can see a list of common names And so the way we we are approaching this Is the following so what we're trying to do is to reduce the amount of information content available When you land on an object page to its essence and basically display Just an overview of the information and then let editors, you know go deeper only if they want to So the concept that we're trying to leverage is the progressive disclosure of information through interaction So basically every action of an editor is going to disclose information Their own request and here for example, you could see our object mushroom And so at a glance you can see the type the identity The validator and you can see that also has like four items for properties Here you see this Calvorista mushroom using the previously defined object and you can immediately see at a glance What's its family the species the genus some of its common names and here on the right You could see another object that might use another type in this case is a type calming and it's using And it's defining the title and the outdoor in this case And so to go a little bit deeper on what we mean with progressive disclosure on information So it's basically this What you see here is a potential flow for navigating this information So let's say you go to this mushroom object page and you touch or you click if you're on a mouse input or a drop by input device this Row keys it would open and we will display what are the keys that are currently available in this object And then if you would touch or click on a specific key again, you would go a little deeper and you could navigate You know, what's the belly tie? What's the kitty? What's the genus? And as you can see there's also the language icon And that's because you could access Potentially available translations for the specific keys and in the scenario where A key is not available in the same language as the language you are that you have set You're going to see a full-back language based on your own preferences or you know navigation history with a dedicated language tag Signaling that is a full-back language. So this is how we're approaching design and How we are thinking about, you know, organizing information with functions We'd love to hear your feedbacks on your phone and thank you very much for your attention Hi, my name is Hanoja Galata and I'm a staff software engineer in the AFSA Wikipedia team My colleague James already gave you a nice introduction on the architecture of the functions But I wanted to go a little bit in depth into the persistent flavor and the content model I want to do this because this ultimately will allow the functions to be a wiki and to collaborate edit create Objects, but it also is what adds Very big vulnerability. Ultimately the functions are user contributed code that will be run in our service. So that makes it very important Now, I'm sure you're familiar with the high default content model of Wikipedia, which is called wiki text In the case of the functions the content model is from said object So media wiki offers Rates tools and abstractions to handle this kind of difference We do this just as a very very brief technical overview by Setting the configuration to say which is the class that is going to be handling this content And this class is in charge of creating object content But but it's also in charge of Making transformation from one one way to the other. So for example When when we're reading stuff from the database and we're sending it to the browser Who will modify that content in order to make it readable to the user and create the UI but also when we Send data from the front and also from the API is to to save it in the values We'll do validations and transformations and make sure that this this data is secure So what is called content objects from the point of view of the function model is what we call persistent object or set to Now as an object content is just the json file, but this is really heavy structured And this is an example of it Persistent object. It just adds a layer a wrapper around the actual object that we're saving So that we can save metadata. This metadata is for example An internal identifier or a page name. This identifier is extremely Important to keep unedited. So this needs to be remained completely locked and secured once it is created I would say that this is our current state right now when saving as an object the id cannot be changed It's completely forbidden. Uh, and also there's a set of additional validity checks Like for example, it should be a valid json or it should be a valid zero But there is a bunch of other things that need to be checks while saving for example this other Third piece of metadata, which is multilingual identifiers labels and aliases This is free to anyone to edit to add new labels in new languages But the actual content as it took it to Which is what contains the or the function or the type this Should be protected in certain aspects some pieces of it Should not be edited and some pieces of it are open for edit Now to illustrate what we are going to be working on in the next Month I wanted to create this really silly example for hyper eyes This is a function that transforms asha Wikipedia into a hacker language word It's very simple Now when I when I do an edit what what I'm sending to the back end is this whole json file with number of changes All of these changes will be done using a UI so that the user doesn't have to confront this kind of z object language Um, but what the back end is receiving is this this json file and the back end needs to understand what's going on What changes are happening and whether those changes are allowed or not So as I said before changing the object ID is completely forbidden. This is fine Uh, and if I add a new label to the function, this is completely allowed that anyone is welcome to add Labels and aliases to this function now. There's four other edits going on on the content block What happens with them? There's changes like for example changing the argument key that should also be completely forbidden. So if I change The way that an argument is internally referenced every function Call that is calling this function will be broken. So this is something that needs to be detected and completely forbidden but then there's other changes like adding a new label to the to the Arguments instead of like only having one english label will have to That is open to every user to contribute There's other changes for example, like adding a function tester or adding a function Function allowed, but they're only allowed to Privilege users uses that know which testers and which implementations are safe and are okay after reviewing um, so all this intricate understanding of the object that is being represented Finally, which changes are allowed and finding which changes require certain types of privileges Is something that we are going to be working in the next four months. I'm very excited about it I hope you are too. Um, and keep reading our newsletter because I'm sure we'll be talking a lot about this Thanks a lot We can be dating and I'm going to talk a little bit about wiki functions and how it makes use of a tool called codex So hopefully you've seen some of the videos from my colleagues that talk a little bit about what wiki functions is and how it works Uh, but for the purposes of this it is basically, uh, the ui or the front end That allows you to interact with functions in the wiki media world So codex, uh, is a toolkit for building user interfaces within the wiki media design system I'm reading this from the screenshot for their documentation. Um, and so it basically contains both the design pieces of You know, this is how things should generally look and feel. This is how interactivity should generally work but also, um relevant for, uh, for developing the front end it has an actual view three components Uh, and so you can see on the left hand side where this red box is a list of some of the components that Codex provides for us And so I like to think of this as things that I might see on any web page That's not super relevant to the functionality of that specific web page So a button a checkbox an icon things like that And so it's really nice about using this library from a developer's perspective It means that we don't need to write and maintain the code for something that's going to look nearly identical to that of um Another wiki media extension or something like that. Um, and it also means from the wiki media perspective That all of the different wiki media products will look and feel very similar So even if you're new to wiki functions, you don't really know a ton about it. There should be a lot of um Interactions and visuals that that are familiar to anyone who has used something like wiki media for example So this is an example, uh from wiki functions So what we're looking at here on the left side is a screenshot from the function page So I'm viewing a function that's called echo. Um, and what I'm viewing on the right side is a developer tool, which is a view extension to the current browser and so The part that I've highlighted in red is a codex component called tabs Tab container and then the about is one tab and the details is another tab And so just to kind of reiterate uh, what I said earlier What's nice about this is I can grab that component from codex And then I actually only have to worry about the parts below which are very specific to wiki functions Whereas the way tabs work is not necessarily Um, kind of what makes wiki functions special. And so that part I can sort of abstract away from my application um, and so what I'm looking out on the right hand side is Uh, sort of like an under the covers. Here's how that actually breaks down into individual view components Uh for anyone who's familiar with view Uh, so what I have here, uh, the cdx stands for codex. So you can see I have this function viewer That's my component that I'm serving. Uh, and then that pulls in the cdx tabs component cdx button component, uh, and then the individual tab And then from there I can say uh to to the codex component. Hey, I want these tabs to be called about in details And I just want you to route me when when they click about I want them to route me here And when they click details, I want I want you to route me here and Codex sort of out of the box will say okay. This is you know, this is the line that we have This is the color that it changes to be all of that cool stuff And so this uh is just Even one layer more under the covers a little bit about what the actual code base looks like This is part of the wiki functions code base, which is available online Um, and so this piece here on the left hand side is html And so um on the right hand side in this little box You can see how you might actually use codex in in an application You set it up as a dependency and then you do something like require quote at wicked media slash codex Dot name of the component you want so we just looked at tabs and so I have tabs which is like I want to have a place where lots of tabs will be and then the individual tab And so then once I've done this, uh, I can go back to the left hand side and I can use it Like I would any other html component It has particular props that are relevant to it like the names of the tabs Um and stuff like that all of which is available on the documentation page But for my purposes it works just like any kind of out of the box html component So that's a little look at, uh, how wiki functions is leveraging some of the cool stuff that, um, the foundation has to kind of offer and uh Yes, hopefully you enjoy Hi, my name is Cory Massarro. I'm part of the epshode Wikipedia team Uh, I'm here because I'm interested in language and like probably all of us the democratization knowledge I'd like to present a little bit on a topic that's come up A few times with respect to abstract Wikipedia and that's basically a set of questions around belonging diversity equity and inclusion um this these questions generally take the form of Uh, how do we make sure that when abstract Wikipedia is is launched And we're generating articles and a whole bunch of languages Um, the perspective that's being represented, uh, is as diverse And inclusive and equitable as the set of languages we support Um, and there are real challenges to that. Uh, these are known challenges and we've been discussed before Right now we don't have answers Uh If I said we did have answers you should be worried because that would be a reflection of the same problematic dynamic Trying to resolve here. Uh, but hopefully we're asking good questions Um, I recently had the opportunity to do an international arts residency and while I was there I got to speak with people from a number of different linguistic backgrounds Uh Many of them spoke all of them spoke at least one hegemonic language But many spoke languages that we would consider under resourced Within the Wikipedia with a Wikipedia project. So This is just a lot of ideas. Uh, how can we use abstract Wikipedia in the future? It also let me with questions, uh, to ask And pointed me toward Where different aspects of inclusion intersect So ultimately I'm hopeful that, uh, we can cultivate partnerships and conversations Around these questions within even broader set of communities So one thing that came up was about the neutrality of facts When I asked people what they would like to share on something on a platform like abstract Wikipedia People would say things that generally related to where they came from places where they lived and This is a little bit challenging because if A place where you where you've lived Um has a particular tree or bird that is not threatened For example because of a war or this the place itself, uh has been Inalterably Terror formed because of war. Uh, you're automatically getting into a political territory. Um, and so It's it's difficult to get the knowledge democratization right Because we want to think that Knowledge can be neutral and it can be objective, but often that's not the case But really these conversations excited me. It means there's a lot to learn about the world and I wouldn't have learned any of this interesting stuff Often sad stuff but interesting stuff without talking to people whose experiences are vastly different from my own. Um, so this Makes me really excited about abstract Wikipedia. It means um You know, what can abstract Wikipedia do it can Expand access to historically under resource languages. That's what we often say it's going to do But it can also amplify the voices of those languages speakers. Um Selfishly I want to read and hear more of these kinds of stories. Um, I want to have my own perspective complicated and I can maybe do that if I have a system like abstract Wikipedia that lets me read In one of my languages stories and perspectives on knowledge that come from people who do not speak my languages um So I felt that was interesting. Uh, another thing that came up was Uh, a set of questions I had around how people interact with technology So a lot of the ways people responded to this were pretty expected, you know Um, it's easier or harder to use software in one language or another If you speak a hegemonic language, obviously the internet opens up greatly. Um But then I also had some conversations that ended up being about literacy And this is really fascinating because I talked to people who were clearly fluent in literate in multiple languages uh multiple hegemonic languages usually but, uh, they also had a language they spoke at home or among friends and Didn't read that language um in particular and that people who spoke, uh, grumanji a Kurdish dialect and Uh, you know, they'd say things like yeah when we're talking with our friends, you know That's all fine and when we exchange text messages, we send like voice messages. We don't send text Uh, we even looked at some online corpora of poetry and It was really hard to read it But like if somebody sat there and sounded it out everybody else could hear it So that was a really interesting thing to witness. Um So when I think about how abstract Wikipedia has generally been conceived as a text-based project, um I'm I'm reminded always that there's and there's It's time to restart a video for some music last Made me hopeful even though it kind of raises the bar on Maybe what we should hope to be able to do with abstract Wikipedia Um, that's my piece. I hope you found it interesting and that we can talk about this Uh, I'm sure there'll be some contact information shared somehow after this. So please reach out I'm a real good man a Google org fellow for abstract Wikipedia working in the natural language generation In the previous video my colleague Cory raised an important question How do we foster an equitable and diverse abstract Wikipedia? A partial answer lies in the following We should foster an inclusive and diverse contributor base For abstract Wikipedia Especially among contributors from low resource language communities This means in particular that we should cater for occasional contributors Or contributors who lack programming skills of formal linguistic knowledge Of course the same time you also want to encourage contributors who want to get more involved for those who have more technical or linguistic skills What does this mean in the context of energy that is natural language generation? This means that the creation of basic natural language renderers a key component of abstract Wikipedia Should be as easy as possible not requiring a lengthy training Well at the same time handling complex linguistic phenomena should be made possible for advance This can be achieved by modeling renderers using templates Templates have been used for a long while within the field of energy In the simplest form templates can be sort of as text interspersed with placeholders or slots Let's look at an example In Wikipedia we can find some information about Mary Cooley such as her date and placeholders An occasional contributor could write a simple template to render this data such as the following Person who was born on date of birth in place of birth When we fill the placeholders with the content from Wikipedia we get a sentence such as Mary Cooley was born on the 7th of november 1867 in Warsaw At this point you may want to tell me come on this doesn't even work correctly for english As even the proposition may change in front of certain place names as the example here shows In other languages which regularly exhibit phenomena such as verbal agreement or case assignment Such an approach might not even work for the simplest cases And indeed you are right For this reason we enhance the template language syntax with a dependency grammar annotations Dependency grammar is a linguistic formalism used for describing sentence action Which can model phenomena such as agreement or case assignment Our idea is that advanced contributors will be able to correct or enhance simplistic templates With the right annotations in order to get grammatical output Just as seasoned micropedians may correct or enhance contribution of occasional contributors In the example here you can see that the french template is augmented with Subject and mode labels Which will instruct a renderer to enforce subject verbal agreement This permits rendering the sentence Marie Curie and Ne The 7th of november 1867 in Warsaw With the correct feminine form of the participle Ne If you are interested in more details, you are welcome to read and discuss our proposal on meta-wiki Stay tuned as well for the upcoming apps of Wikipedia newsletters Which will discuss this proposal. Thank you Look at our software engineer fellow working with the abstract Wikipedia team on the wiki functions project I will be presenting the work we are doing on the semantics of wiki functions First of all, what is semantics? In programming language theory Semantics describes how programs should be executed in a particular language One way to do that is to describe the relationship between the input and output of a program For example, we can convert temperatures in celsius to temperatures in Fahrenheit using this function Take the temperature in celsius Multiply it by 9 divide by 5 and add 32 Here is how the evaluation could look like for the temperature of the human body Here I highlight in each step the part of the program that the evaluation is focusing on First, we replace the function by its definition and substitute the input value of 37 which gives the second line Then we multiply by 37, sorry we multiply 37 by 9 over 5 which gives 66.6 And finally we add 32 which reduces the result 98.6 We have to be careful in the order in which we do the operations Everyone knows it will be wrong to add 32 before multiplying by 9 over 5 That would give an incorrect result This is exactly what semantics is about By defining precisely evaluation rules for the language We can describe how programs should be evaluated in a correct and unambiguous way Semantics is not limited to evaluation order It also describes how variables should be handled and substituted Which terms are considered valid and which ones should produce errors and more As you may imagine In wiki functions, this can get quite complex Not only can we define functions in a native language like javascript or python But we can also define them by composition Using other functions as building blocks And each of those other functions can themselves be defined then anyway In javascript In python Or by composition that even built-ins We need to carefully orchestrate the evaluation of all these functions in order to get the correct result By being rigorous in our definition of the function model and its semantics We can give a precise meaning to all of this And provide a clear definition to which we can refer to Here is a preview of the evaluation rules we are working on written in mathematical notation This work is still in progress But it has already allowed us to find and correct Ambiguities in our model and bugs in our implementation Thanks for your attention and we look forward to your questions Hello all again The video is also available in comments and can be viewed there before the translation yet And sorry for the technical issues that happened at the beginning of the video and in the middle of it So we have no time for q and a where I still have 67 minutes available And I see a few questions already there One was where can I get the wiki functions flag? and The answer is that once we launch we will we will make sure that we set up the usual wiki media merchandising machine And we will have all the usual merge freaky functions if our logo so The team got an early Version of this and I'm very happy to to have it here. Thanks to the community for the logo It's it's really growing on us Second question, maybe have a large language model similar to lambda and others This one Thing where we're very different in large language models So in wiki functions, we use functions to generate natural language text, which means we're using Um Basically rule based systems that say okay if you see this do this and so on created all by Human developers based on the data in wiki data for like so graphical entries we are not Using a large language model in the background Which is machine learned and which is doing this automatically The main reason for that is that those language models are not as editable One main Feature of a wiki media project is that the contributors when the cms take Should be able to click on the right edit button fix it mistake and see it being fixed This is something we don't know how to do with language with large automatic language models yet. This is why we're using this currently quite Out of fashion approach of a rule based natural language generation system And we are building on decades of knowledge in this area in in order to to make this happen, but this means We will not We will not be providing our language model like this It also It also means it's not really something that the wiki media foundation would be best poised to do Because the text that we have available Well, it's either wikipedia's and those are Not really large enough for the kind of large language models that are currently Out there where they usually take the whole web and other sources in order to build them Be created by google or deep mind or facebook Or apple they're always building on a much larger corpus the just wikipedia's so All of this doesn't really put us in the place Where we could provide our language model of our own There are Open organizations working on these kinds of things and we support the work and we there are ideas of how to use language models even within the abstract wikipedia for example to Truth between two equally Well-looking alternatives and similar things are to check whether text makes sense and so on but it's not on our Main route for the beginning those things will probably be added later as we go on next question I've just found out that labels of object properties like set 10024 mushroom can be translated however I try to do the same if set for type or set to a persistent object Their product their properties are not translatable Using the normal edit interface at a cortex probably is going to be translatable too So yes, they will be translatable. Um, they will not be entirely editable But they it should be possible to add translations to them. They are still working on the right rights model and on the right Use the same for so this is not implemented yet. Um, and this is why you can't do it yet on the better Properly, but eventually Yes, it will be possible to add an edit labels on the core models But you won't be able to change the core model. Um, unless you have the The right rights, which will be quite locked down because a change to the core model can cause a lot of issues So How is your time plan holding up still launching this year? Well, the original time plan was to launch last year So it's not so long. It's not holding up great. Um, no, um But things are different than uh, the river planning originally. So um I really hope that I don't want to make any promises. I really hope that the launch will Uh go forth in the not too far future Um, that we will see in the next few months the launch the Better that we started last week gives you an View where we are and you will be able to see how we progress to a place that we consider launchable It will also have a continuous discussion with with the community About what really needs to be in the launch, but what doesn't have to be and within the team as well So, um, I don't want and I and we never made really a promise about when we will launch um We will get out there when we're ready, but particularly with the Google.org fellows and um With the new people that are joining our team. I'm confident that the state, uh, won't be Slipped to be slipping too much into the future So I'd rather have you something to use and play with sooner than later and I think this is a shared, um sentiment on the team. So I'm I'm pretty confident that You'll have something to play with. Um Sooner than later, but again, I'm not making a promise. It's about the date I'm checking in If a pet that they have more questions coming in, um Just because I already have too many screens. So We're good to find. Oh, we're also at the end of the session time. Actually, I just seen so thanks everyone for being here We have monthly volunteer corner. So you can can come with more questions You can also read just on our mailing list and our telegram and irc chats So please feel ready to ask me more with more questions Um, and And to take a look at the video again Without a technical difficulty, I hope. So thanks everyone for being here. Thanks for the organizers for giving us the opportunity to talk And um, I'm very excited about, uh, how the next few months will look like Thanks everyone