 Okay, so let's start. This talk will be about the underreport of LibreOffice. And even if the focus of this talk will be about the core application, like the viewer or this basic editor, there is also this LibreOffice in-person remote. I'm controlling the presentation from this phone. And let's experiment with this. If it will be prior by the end of the presentation, I will have to hold here and press some keys so it will actually work, even if it's kind of a lack. So let's see what has been done up to today. The very first step in the history of the underreport was to get this huge code-based cross-compile that was mostly done back in the Susan times where Thor did spend quite some time on getting this to work. And one of the major problems was that underage is not Linux. Like it's the same kind, but everything else is something custom with stupid limitations. And if this reminds you of Windows, then you are metal. So the linker has this stupid limitation of 128 shared libraries, and we have much more. So we have to link all this code to a single shared library. And because we are cross-compiling, we can't run any unit tasks, and we had to, or we had to solve a number problem still. There was a document of the prototype that was using the Uno Java binding, because initially the idea was that if Uno already has the Java binding and under it we found Java, then it might be a good idea to re-target that for this purpose. It didn't really work out. It looked like this that you could switch between the pages and have the document rendered at a fixed size. I guess you couldn't really zoom into it, or maybe there was some support for it, but it was quite limited in many ways. The purpose was just to be able to exercise the actual rendering. The rendering, the result was quite how the printing result looked like. So it wasn't something that could be later extended to be an editor as well. But it was a start. Another prototype next to this was our other approach, would be that the desktop rendering would be completely reused. Basically we rendered into background bitmap, and when the order bitmap was changed then we copied that to the Android screen. So it looked like this. You can notice the desktop menus and basically really the whole desktop was copied. The reason why we could do this was because at least on Uno's, there is no desktop information integration available. We fall back to this Roax11 background that really had no external dependency. It really wanted to paint on the pixels itself, so it could be something that would be re-proposed for the Android rendering as well. So we had this document loader prototype. It could load a document that was far coded by real time basically. So it was far from something that could be useful for a non-user. So that was a G Suite project which tried to build on top of this document loader and it tried to add this document browser. Actually this is the oldest part that's still part of the current Android viewer. You could say that it's some duplication because of how I tried to be a file manager when we were building a document viewer, a document editor. But again, one more under the stupidity is that it doesn't come with a file manager on most of the phones, so you have to provide some way for the user to choose a file. And even if you have a file manager, Android has some interesting idea about how to determine what application to use with a given file. Like it doesn't do any kind of deep file detection what we do on the library side, but it just does it based on the file extension. And it has a huge hash map about what file extension belongs to our mind type. For example, the ImpressTemplate type is missing from the hash map. So if you would like to open ImpressTemplate, then you still need this document browser because we know that it's an ImpressTemplate that Android doesn't own. So it's like this. There are some minor updates to this in the current version, but it's really similar, this part was physically unchanged. What was extended here later is that now you can also open a remote file. Yeah, I did some work on this so that you can load documents from on-cloud, you can save to-down, and there is an interface between the on-cloud and the document viewer, so you can add multiple backends and so on. So the next thing we say towards this current state where we have a viewer and we do have an additional basic editor is a type rendering. That's the core of the viewer that we have today. And by type, we mean these small bitmaps and we render into these small bitmaps so that when some changes, you don't have to rerun the whole screen, just the change bitmaps. The idea is from Mozilla, they invented this for Firefox, and I guess by today, mostly they use it everywhere. Maybe not on Linux, but I think on Windows and Mac, they use it even in the desktop version. So in one of the earlier talks, it was mentioned that in the long term it would be nice if LibreOffice would do this on the desktop, it might sound like a pipe dream, but actually Mozilla did this and it seems that it was, so it's not completely unfounded idea. So the big thing with this type rendering is that you have to implement it in the given applications. Initially, it was for writer. I think Candy prototyped it and then later, Candy and Thor did work around it. So actually we had a working type renderer for writer, but not for caulk and deep press. There was a full-on g-stock project so that the deep press type rendering got implemented and also there was a lot of time to do it for caulk, but it was done in a future branch and as far as I know, it didn't do good things for the caulk desktop rendering so that now we're running the master. A bit later, Kobai and Henry did most of the work around that so that we can have something that's type rendering for caulk and still not breaking the desktop rendering. So this is how it looked like in this GTK type view. It's a test-by-application. It's built by default. It's not part of the installation set so you have to run this with this VIN slash run shell script. But you can test all the C++ parts of this type rendering and later type editing features. So at the end, you will see how the editing part looks like and I did most of the work on the library side and Thomas Schweinger did most of the Java part. So actually, most of the time I didn't have the Android. I just made some work from this type view and once it was working, then Thomas took over and actually did the rework. That's a great way to always touching Android when you are supposed to do Android work. So Thomas integrated the existing type renderer with the Phanak front-end from Mozilla that's used for Firefox core Android. So we have the C++ core. We are using the whole interface. On the other hand, the output is just ties and some API between the C++ core and the Java application and what you see as a user is some native user interface that's written completely in Java and it's completely separated from the desktop user interface. That's quite a big contrast to the desktop prototype where we already used the machine. It sounds like this is more the way to go. So this is the first screen shot from this Phanak base viewer that already used the type rendering. You will see later that there is an additional toolbar and a few changes but it's basically like this. A major step towards the end-users was when this app was pushed to the Google Play Store that means that we had to fulfill all the requirements even if it's well known that the Play Store users are much more liberal compared to the Apple App Store requirements. We still have this problem with the size limit. The rule from Google is that the AP case, the comprised application can be more than 50 megabytes. That's still a problem today. If you search from LibreOffice in the Play Store, you will find two versions. One is the collaborative version that's still a viewer. It doesn't have the editing capabilities. On the other hand, it can open a few more files, most of them are the Microsoft formats. There is another flavor, just playing games without removing different services so that you can choose what features will be included and you are still not above this size limit. The version from TDF has these basic editing capabilities. On the other hand, it can only open ODF files. There is something to be solved so that if you build yourself and you just push the APK to your Android device, then you can have both at the same time. Once we had something in the Play Store for the viewer, the next big step was the editing. This is what was sponsored by the TDF. This is something that's new since, for example, FOSDOM. There was also a session about the Android port. By the time of this year, FOSDOM was physically the viewer. In case the document had blinking tags, then the rendering result on Android also was blinking. We had some kind of working invalidation, but that was all. The first problem we had to solve was that we saw that we had a working viewer, but the car was still lagging behind. The root of this problem was that by the writer works in logical units in Twitch and by the impress works also in logical units in 100 mm, CAC is switching between these units, even in this pixel sometimes. For the time rendering purpose, we are always working in Twitch. We are always working in logical units, and in case some part of the rendering result would like to work in pixel, then we have a problem. Related commons are in the right turn. The right turn commons are in pixel. That's why currently they are missing from the time rendering result because some additional work would be needed there. Again, if it would be all in logical units, then that would be much easier. But with time, we got something that's usable enough and we could claim that the time rendering in CAC is good enough so that we can go to the editing part. There was a session from our talk by Candy as part of the GSOC panel demonstrating that Christian Peter did this somewhere to improve the CAC situation further. So there is hope that one day CAC will be as clean as right turn and impressed from this logical unit rendering perspective. So, time editing, that's the next thing. So the major thing is that when we have only a viewer, then we render something once and we could assume that it never changes. If we were out of memory or something, then we could drop ties and render them later. But apart from that, once we got a tie, we could keep it as long as we wanted. This is no longer the case with editing. Before, it was possible that the user interface just pulled into the C++ and requested the ties as it wanted. We had to implement the communication in the other direction as well. So there is no callback at a liberal circuit level and that it can signal if something happens like you are supposed to drop ties intersecting with the legal logic or rectangle and stuff like this. The good thing is that the FANAC code base that we imported from Firefox is prepared for this because even if you use the bubbles most of the time as a viewer, as an HTML viewer, there is this strange JavaScript saying that allows you to modify the document content so they have to render things left and right already. So it's something that we can already use for our editing needs as well. I already mentioned the GTK type viewer. It allows a bit more productive development from two perspectives. One is that even if you have debug symbols, then linking this separate shared object isn't that much time. GTK type viewer can use these updated shared objects. So the hack, compile or test alteration can be relatively fast. On the other hand, if you would like to do the same on Android, either you don't have the debug symbols, but still it takes around one minute to do one of these cycles, or you have debug symbols and you have some limitations in the linker, which means that a fully debug GTK linking 400 takes around... If you have enough memory, so it actually finishes, it takes more than 10 minutes or something like this. So that's not far from ideal. What's very strange is that the same situation for iOS isn't that bad, so they do have a linker that can cope with a fully debug GTL large object files and linking them together to a single shared object or single executable in a reasonable time. So let's hope with time the two-chain way will actually catch up with our requirements. So what we need for an editor, we need some input handling from the soft keyboard and touch neurons, we need some cursor, we need selections, we need the invalidation, and then we will have something that allows basic editing from an end user perspective. So as mentioned, this is sponsored by the TDF and the good thing about this is that basically none of the companies or clients have enough budget to get this going, so it was realized that it's necessary to put out a standard that allows somebody to do all the grounding work to get this going and that actually happened. So we bought this part and we could dedicate a couple of months to implement this. So the first thing was this linking tax. It's a nice use case where the rendering result has to be updated but you don't have to have working input handling, so you don't have to solve two problems at the same time, which is exactly what you want and you have to work on this. And this is basically where the presentation at FOSDAW was finished. So what are the use case things done? LibreOffice has the keyboard and mouse handling. We mapped the Android soft keyboard to the keyboard handling of LibreOffice and we mapped the touch events from Android to mouse events. Interestingly, since then there was also some work in parallel with this from the GTK3 part or the GTK3 documents where the touch events are propagated to VCR as real touch events. We wanted to reuse as much as possible, so all the Android touch events are mapped to the mouse events. That's how we got a number of things for free, like if we map, for example, long press to double click, then long pressing on a guard will automatically do this handling so that double click on a guard creates a selection on the guard and features like this. If later the VCR touch support will be good enough, it could be a possibility to bring up the Android touch events to VCR touch events. So the next thing was some native overlay. We wanted to have the cursor and the selections not as part of the rendering result in the files so that if you just navigate with the cursor or you change the selection then we don't have to re-run them. If you ever looked into this, even on the desktop we have overlays for this so all we had to do was to disable these native overlays like from a LibreOffice perspective native or for the title rendering so that we can add these overlays from Java on the native Android UI. So by native I mean the Java native. So we have the cursor, we have many changes and the LibreOffice Kit callback informs the Java layer where the new cursor rectangle leaves and also the same for the selection rectangles. The cursor is easy, it's really just a rectangle but the selection can be more complicated. It's basically a list of rectangles and also in the mobile board it's quite common that you have selection handles so that you can adjust both the start and the end of the selection. This is something we implemented. This is something that LibreOffice coordinated and supported so if on the desktop you create a selection and you are still holding down the mouse button then you can adjust the end of the selection but not the start of it. So this is something that we had to implement. On the other hand for the various selection and various the cursor, that's something that the desktop version already knew and we had to just instrument this code with the necessary LibreOffice Kit callback so that this information is a variable for the Java layer. So this is the first demo. This is showing what's possible in a writer. So before you get this view viewer and then if you tap on it then you get an editor. You can tap as far and then the cursor is moved. You can type something and then the letters even the extended ones, the random unicorn values are supported. Of course if you are in the middle of a bullet list and in the new press center then the cursor jumps you can adjust the selection, the start and the end. You can also drag the cursor. I hope it will be part of one of these demo videos. If you had something selected and you typed something then of course it will be overwritten. If you have some selection inside the table forward then you can adjust the selection handles and between the first selection we turn into a table selection that's quite handy with this limited mobile input capabilities. If you don't press on a table side and it will also create a table selection you can tap on an image and it's possible to move around and resize them. In a writer the shape text is provided by the edit engine that's something completely different. You can also edit that. You can again resize the shapes as well. If you notice the images are resized keeping their aspect ratio as it happens on the desktop. If you resize shapes then this won't happen because it's what users are used to when they use the desktop version. You can create a selection and then use one of these buttons in the toolbar to dispatch all the commands on it so you can mark it as void or italic or strash through or what you prefer. It's really just up to adding more buttons to the toolbar or it's kind of an area of more for somebody who is more into the user experience part to decide exactly what buttons to put there. It's similar to the two bars on the desktop version where we just have an API for this unit commands and then it's up to some of us to define exactly how these two bars are organized and what buttons in what order appear and so on. It's similar here, it's quite configurable. This is for writer. But once we had something working for writer we wanted to make sure that it won't happen that impress and caulk can't catch up. We had this problem with the viewer that the caulk part was lagging behind. So once I'm not exactly sure what we were with the writer room but we weren't ready with this. We also switched to impress and caulk and brought it up to be on par with writer and switched back to writer and did something new again there and it was like this. So this is for writer. The next thing was impress. For the core part again that was mostly my fault to do the necessary instrumentation so that the Java layer is of what's happening on the core side. Again this is known as the tangent tax. You have the blinking cursor and you can track the cursor to some other position and the cursor will be adjusted. The cursor is not moved to some random place but we just send the information while it was dragged and then the core will decide where the cursor position that's the nearest point where it could be placed. And basically we added the same features to impress as well. So if you have a bullet list then you can add the new one. You can create a selection. If you notice this small bubble on the video that's presenting where I was with my finger so you can see how something is started and dragged away and so on. Other than that it's similar. So again we support this and that you select the word and type something that gets over it and just as you would expect from the desktop forum. Something that required a bit more work was this impress there was because again that's completely different code from the writer there was. This is mostly in S-reacts and we wanted to have the same behavior. As you could see you can switch between the slides and as you call them document parts so that can be a slider or a spreadsheet and you can click inside the shape text and there's something you can resize the shapes. Hopefully. What's important is that it should be very close to the desktop version except one. There was already some common behavior that was quite standard in the mobile world and we wanted to be consistent with that. So the last video will be about CAC and this is impressing the wrong button So the last demo video is about CAC we had a single larger sheet so in the video you will see how you can pan around with the document you can also zoom in in case the text is too small so that you could position the cursor precisely. If you notice here the selection is a bit off from the text. Even the cursor has this annoying one pixel difference maybe it's even a bit more that's because of these annoying inconsistencies in the car rendering and switching between the logical units and these rounding arrows can add up. The hope is that this will improve as the car rendering moves more and more to use logical units in the rendering. But other than that it's really the same. On the challenge we had to solve that when you load the document and just render it then the image is not yet dark because if we know where to paint the parts then we already showed that and the image is loaded later it's a bit more expansive operation and then we have some problems with the invalidation so that we wanted to show the document without the image and load the image and when the image was loaded invalidate the necessary files so that you can actually see them but other than that you can see even for car the same user features. So that's where we are and do I still have some time? Okay so I promise I won't use all the time but I don't want to talk to you even if he's not here. So basically that's really still not done but would be really easy to do on the Android user interface layer is the features part we did something we did some API at the Librofi speak level or Librofi sound line but the Android user interface didn't fill it out so what we are saying would be to find out what services are exactly necessary to support flat ODIF and then those documents could be loaded as well because of the size limitations by default all Uno services are excluded and you have to explicitly buy a place or the components or the constructors that are necessary to be included in the product or the APK So the next idea would be support for searching the Librofi speak API is there just the GNI and the Java part would be necessary to implement One more nice feature would be that once you have a selection you could copy it to the Android Kit Flipworn again the lock API is done One more thing to fix and it would not be too hard to implement to have some proper feedback about saving currently we just dispatched the Uno Q1 so that the save will be initiated and sooner or later it will end but it's unsynchronous we don't know when exactly it finished it would be better to handle it explicitly like we do it for the opening of the document we know exactly when it finished because we don't have some kind of progress bar for it and we know if it succeeded or failed it's quite misleading that currently there is no feedback about the saving result so that even if it failed you don't see anything you just close the document open it again and you see the old version so I guess that's it thanks for everyone from my colleagues who have most of the work done by Tom and me but as you can see a number of other people have also the sponsors and then Smooth was the company who founded all the work around the viewer and TDF sponsored the basic editing part and really without these sponsors we as a consulting company couldn't do this because we have to work on something that's paid by something so that's how this became possible so thanks for listening the impression doesn't crash so if you have any questions I may try to answer it yes the question was if there are any minimum system requirements yes there are different API levels that the application can declare and I guess we require under it for the Tomosh Iro here is that correct you require for the Tom right 401 401 I'm afraid it's quite possible this is not done but if you go from the Play Store then the Play Store will prevent you from installing it on one device most of these UIL amounts are something that's not available as is but hopefully for that there is not a huge requirement nowadays anything else any other questions one two three ok thanks for your attention