 You can make a pretty good mobile web browser using WebKit. You have Android, WebI, we have the browser in Blackberry Torch, and we just see more and more. There's also less bootstraining because a lot of the pieces needed to make a good mobile browser is already in place. So, we're talking a totally different form factor than a desktop browser. On mobile, the size of the screen is usually quite small, which makes for a lot of different problems. And we're not using a keyboard, we're not using a mouse to navigate. So, we have a totally different input. You might say to yourself, well, Nokia hasn't been really good at doing web browsers. The track record is really that you're wrong. You can say that today, that might be true, but actually, Nokia was the first company that put WebKit to a mobile platform. And they did that many years ago. And they did pretty well. But there's a lot of things that's changed since then. For instance, there's Wi-Fi everywhere. There's access to 3G. The cost plans are a lot cheaper than they used to be. Before, people weren't really using an Instagram browser on their mobile phone because it was really slow, it was really expensive. And it couldn't really render the real web pages, so we had to get these web pages with limited content. And it wasn't really what people wanted. So, one of these things has changed. We have nice data plans. We have larger mobile screens. We have a different way of interacting with the content using touch up between your fingers. And we have more powerful devices. So, this means that today it's possible to actually bring the real internet to mobile devices. You have more screen building states and you have more powerful processors. So, you can actually load these real pages and process them and show them how to produce them. But, there's a lot of challenges in making a mobile web browser. First of all, as an event user, you have a small phone factor. Things just doesn't fit on the screen. You can't really see a whole page. Then you might have to make it smaller and then it takes it so small that you can't really read it. So, there's a lot of different issues. Another problem is loading the page from the net. When you're having a slower net connection, you also have to understand past this on a slower processor. Then we have to go through the layout page to figure out where should the text be, should it be reflowed, should it be a box here. We have to do all this before we can paint. I think we have to paint. So, that's very slow in comparison with a desktop browser. But we have maybe a phone called computer. Then the subject also is very, very important. Touch interaction requires a very, very responsive user experience. I'll get into that a bit later. Also, fingers are pretty huge. So, you're trying to click a small link with a big finger and you can't even see where you're clicking. So, because content is small, you normally zoom out. So, you can't see it. We'll talk about that as well. There's also the web apps. People are trying to make web apps today like email, clients, etc. So, that brings a whole lot of different challenges. But we'll talk about that at the end of this presentation. So, let's look at a small format. There are always some really, really great ideas to deal with. The first is zooming. It seems to be possible to zoom out or zoom in to look at what's interesting for you. Some good ideas, for instance, you can do that double tap to some specific area of the page. For instance, if you have different columns on the news page at New York Times, double tap on one area and it will zoom in. You can have an example of that here. In New York Times, double tap on another one and it will zoom in. This is not perfect. There are areas where you can't find these nice areas to zoom in to. So, you also want to be able to do that manually. So, people can do a pinch zoom. With two fingers, you want to see this area. Maybe you can move the page around. With a pinch zoom, you can see the two dots. With two fingers, you want to try to pinch out. I can see the whole page. It will probably just go back. If you pinch in, it will be a bit more fun. But there are still a lot of issues. Even though you saw that I zoomed in here, it might be that the text or that column is pretty wide. So, the text becomes so small that it's impossible to actually zoom in. There are different solutions for fixing this. Androids come to socket with text reflowing. The rest of this is the same article. I don't know, it's pretty dark. It means that if you zoom in to something, you don't want it to become the whole text here. And in order to be able to see it, you have to zoom in to this area. So, you have to read here, scroll to the side, read the next, scroll back to start the next one. But that's pretty annoying. So, androids fix that by reflowing the text. So, it means that if you make a specific zoom, you just read the outer text. This is something that's really, really hard to get right. I don't know if people have tried androids, sometimes it just doesn't work. And you end up with text that's always outside the view. You try to move it a bit, it's still out of the view, because it doesn't reflow again. Something else is that I said before, doing the layout is really expensive. So, you can interrupt with user experience. And there's a third issue, that when you zoom in, you get this nice animation where everything zooms in, and then everything just changes position. So, maybe even what you wanted to see is now outside the screen. So, gyphony has a different solution for the same problem. It's mostly desktop browsers that you can have a setting of minimal font size. You say, enforce this minimal font size. Never show font smaller than this. Gyphony brought that a bit further, except the minimal font size is curve width. So, when you double tap some area, they enforce the minimal font size when you zoom in to the area. So, if I zoom in to this area again, they'll say, well, the font size of this area has to be 10 or 20 or 12. This means that in the original page, you'd like to see that maybe the font size in the first column is like this one, and then in the middle column, because they have a better widen width when zoomed in, the font size might even be bigger. None of you have noticed that on the iPhone. You don't really see it. If you know it's there, you can go to New York Times on this page and you'll see that sometimes in some columns, the widened columns are, they often have increased font size a little bit. It does the right thing. It just makes sure it works with a nice animation and it's always possible to read the text and zoom in. But there's a lot of different ideas and there's probably more ideas that can be explored as well to solve these issues. So, let's get back to the fingers. As I said before, our fingers are pretty big. So it's really, really difficult. On a computer, you're having a mouse that has one dot. You click on one specific point. On a finger, it has like a surface you're living on. People even shake, it's really, really common. Everyone shakes a little bit when we try to click on something. There's a little friction there. We remove the page a bit. So, we need solutions when we deal with these things. Like I said before, like on this page, one of the problems we have in this process is that you look at some little area and once you go to the next page, you need to be able to pad around the page as fast as possible. On a digital browser, you only have big screens so you see everything and first and little later, you start scrolling a bit. On a mobile page, you often zoom into some specific area and you want to be able to get an overview and get back soon. So the dot tap is pretty good for zooming into a specific area and you can do it the same way that if you dot tap the same area that it will zoom out in some of the zoomable areas maybe just text. So you want to be able to pad around fast. So, it may be possible to like compare it around with your finger that you don't add to it on your iPhone but still also if you pad in a bigger area with a little force it will start doing some kinetic scrolling making it scroll faster. So you can easily get to the point that you want to see. You want to scroll fast. It's not nice to be scroll able to pad and also to take some space on your screen. So most mobile browsers they don't show scroll fast. Instead they use what they call scroll indicators. Things that show themselves when you're panning or pinching and it will disappear when you don't need them. Because when you're panning for instance anyway you're panning pretty quickly so it's pretty good. You can just look at the sides and discover and you can see if you're at the top or at the bottom. You can also make other ways to show that you reach the boundary like the bottom of the video. For instance by doing your mouse screen or making yourself shake a little bit there are different ways to do that. As I said before fingers often shake a little bit also when you click. Like when you click before starting panning and say okay you have to move like 10 pixels before you start pinching you have to move 10 pixels just to ignore all these small imprecise movements that people make. So this is very common and you'll notice that if you take an iPhone this is insulated this way. To start pinching you can pinch like maybe 5-10 pixels before it starts pinching which means that if you click on the bottom it doesn't start moving the page or you can actually explore what you really want to do. Another way is that if I'm trying to pan I'll scoot into a page like this one I'll start to pan it's very very difficult to do like a movement. Straight up you always move with a little bit to the right of the two of us. So you can try to guess or look at the various you find okay the guy is probably going to pan off upwards and then we'll just ignore it unlock it in that direction. So you don't, if you're winning an artsy you will start to scroll out of it and have to take control of it so that all these different small tricks we have to apply to discuss it as well. Something else that I said before is that on fingers when you click on the device you have a touch area. So if you want to find out like a point where you clicked normally this can be the touch area you get from the touch point and normally you can do another user test you find out that people they actually want to click a bit above that area like where I put the dot here. So you can take that as a starting point but instead of just looking at what's below it because most of the time you make the missing because you have big fingers they're not precise and you can't see where you're clicking so you need some algorithm that will try to figure out whether anything is clickable in this area where I clicked. So it was I think just below it so the user probably wants to click on that link and then we will click on that for the user. Another big issue we have you scrolling the content you're using if your fingers are scrolling is that we want it to be like one area that you're scrolling. It really confuses people if you're scrolling a song you hit some area, soft area and then scroll they will really confuse people because suddenly it starts scrolling differently that could be like a page that has frame sets or a page that has like iframes that are scrollable. So what the iPhone does is it says there are no there are no scrollable iframes so they simply expand the iframes to their size to the size of their content. Do you think that if something that will break what the price doesn't? They do the same thing for frames as we also do. For instance, this is a test we want the webcam test it shows the frame set, it's a bit strange but you see like if you have this on a small phone that would be really difficult to read the text you have to pan a little for the area, you have to pan this area you just want the area to be like one big area that you can pan. So the idea is that we expand all the soft frames to their content area and then sometimes we get some area that not occupies and we expand these other frames a bit further so we get something like this so this will frame set frame flattening the label so we get one big area that's scrollable so the conclusion is that touch is a really really nice way to interact with the webcam it feels very natural, it feels easy or there is one huge issue is if you really film it if the page is blocked like I said before if you have a desktop browser it's opened up, you have a big desktop browser you know the page there's the contents, you almost see everything you want to see and then you start scrolling so normally you start scrolling when it's already finished loading but even if it blocks a bit you don't really notice it but on a mobile device that's slower when you're using your fingers it's being stored and browsers they store hours they do that during loading the most common idea is to separate the UI and the web I have different threads on different processes one way to do it is that you have the web, let's just talk about the UI process and the web process it will paint something it might store the painting sequence send it to UI process and then repaint it this is for what I know is what they're doing on Android or you can back share the painting area it paints into some shared memory of the web process when you're ready you can show it on the UI process what is really important is that everything is non-blocked so you need to design a home view of the API that makes sure that you can do almost everything non-blocked because if something blocks everything blocks and you view it there's one example I've written down here think about it it paints as like fixed elements so if you have fixed elements it has to always be in the bottom as is the web engine that's painting it every time you're scrolling you have to tell the web engine that scrolls and you have to tell it back for each pixel it has to paint and read on it but if we did it this way it would be blocking the whole time so we have to find solutions for all these small corner keys and I'll tell you there are a lot of them sometimes it's not the perfect solution it breaks contents like JIFO doesn't support fixed elements for instance I believe there are very few mobile browsers that support that I have some good ideas how we can support at least 90% of fixed elements but there are all these different issues which we have to solve and it's really important we do this pinching or while you rotate our solution this is similar to what the blackberry and what they're doing on JIFO is using a synchronous title it means that we look at the whole concept as one big image like we paint the whole concept into an image then you can pan an image you can scale like pinch zoom an image and you can rotate it at 6 to 20 seconds it's like a magical number so here's an example the green part that is our screen is what we're currently looking at and the grey and blue area is like the content so the content is divided into different tiles some of these tiles I call it's tile because I've changed the size a bit because I wanted to be able to still pan out and maybe pan back we wanted all to be shown but this is the basic idea so when the tile becomes visible when you're scrolling if it's already painted by the web process which we have memory met IO then we can just split it just paint that tile on the screen if it's not available we can do some indication that the content is not ready yet for instance by painting checkerboards those are the images that the case like Photoshop or even the iPhone is using the content is not ready yet to be shot in this means that we have decoupled the whole web engine from the UI the UI just paints tiles images that sometimes get updated by the web process of course pages they change and there are calendars so when it paints change like on the work the tile is visible we don't have to re-paint the whole tile or we can just paint like the area we have to make sure that we paint the area for all tiles in this way so that all tiles can sync we can't have like for instance a tile that's outside the view that wasn't updated and what is inside the view that got updated because then you might see that they interact that here it's painted and here it's not painted like if you have blue box then you see the blue box here but the blue box doesn't appear here so all the tiles have always been synced when you get update readings to tiles outside the screen a common idea is to store like a dirty reading you don't need to paint it right now you just mark this area of the tile as dirty if you get another dirty area for the same tile you might just merge those two areas or if it's too big you just might just discard the whole tiles and you paint the whole tile the next time so this is not exactly enough to get to make sure you are advising like at 60% for panning painting or rotation because we're having to re-paint we have a loading we can have a load wire pan and we can be javascript executing so the basic idea is that you just suspend on javascript if you're panning your engine you don't care because of it you suspend javascript you defer loads if there's a load going on later then people have to finish panning and they go re-paint because people don't care about these things you don't notice it but it just makes sure that we can re-paint things in 60% this wouldn't be possible for instance if you did a paint sweep on a paste you don't know how long it will take to re-paint that area because there might be more text there might be more retangles it might take a bit longer so when we do a paint sweep we do something more we don't paint anything so basically we stall the whole tiles we keep the tiles what we had and we just scale it just like an image just to figure this whole tile is one big image you just scale it up for a second we have problem when you're finished we re-paint everything so this nice sharp text the taste is also a bit tricky normally you want to see the same thing if I'm looking like a portrait I'm reading that this comes in front of the flowering flavoring so I'm monitoring it should just scale up so I'm looking at the same area so that's the basic thing look at the difference between the width and the height and either multiply or divide with that factor depending on how you work JVM the iPhone did a bit more they can make it possible for web developers web authors to decide how what they should be laid out this is something that's particularly important to work out the cases for instance, if you look at the booting case they're meant as web apps it says like the maximum scale is one don't do this scale up but I won't take it I don't want it so it will be if we look at normal booting here I won't take it there's some area missing so what we do the others do what I do what everyone are doing is that we have to re-lay out we just said ok it doesn't fit we can re-lay out it has a width that was larger basically like what you see there but we don't want to do these re-lay out while we're rotating because then we can't ensure the 60 frames per second so we have to about as a middle step if you rotate it you keep the comfort in your hands just paint some shaker boards something else in there where there's no contents and when you finish you re-lay out so you get a smooth animation rotation animation and then you get the content that you want so something else you freeze the backing store you just spend the jobs you're paying etc you keep the old contents rotate so let's go on to web apps I'm not going to details why people should make web apps when it makes sense I haven't made a lot of presentation about that there's a link here for those people I think I'll flash before we get on to the faster page later so we'll be able to check that out but let's just say that web apps make sense in a lot of situations so how do we go about 40 years well the idea with web apps is that this should be aimed like new apps what does that mean well it means that once these s-picks should be like a device pixel you don't want it to be scales not known apps don't have a scale you don't want to have pins to at least not the whole application because not all applications don't have that and you want to be able to control the input touch you put yourself so for instance you could implement your web app like a pin pin zoom etc so mobile browsers used to today support touch that which makes it possible for the pages to deal with the events in the way they say see fit but that also means that area event you get touch that you get you have to send to the web process which would then have to happen or ignore it if it ignored it it would have to get back to us and then we can go opinion on that but it means that we get this dependency or a slowdown send to the web process wait for it to return which is what we don't want so there's some tricks that can be applied you can check if the page is actually registered and listen to the touch event if it didn't don't send any event to the web process just deal with it on the UI side for controlling the layouts there has been a lot of different ideas for doing that there are a few different meta tags but the latest one was introduced by the iPhone it's called the new port meta tag and it's been quite popular and used on most of the mobile pages out there basically the iPhone will lay out every contents as you would weaponize it at the width of 980 pixels on Android the device manufacturer can decide when they want it to be 980 to 1100 some values like that they have a few values to choose between but for instance if you open up the Apple site with 980 it won't fit and you have this big image on the top that is larger so there will be that page to be laid out with a larger view so they added this meta tag to give this control to the web order to decide how to lay out contents how to deal with scaling etc so the features is that you can define the layout width you can even define the height it's not very used but it's possible to initial the minimum and maximum scale so you can control how you can zoom in and zoom out and I think it's possible to just disable or enable user scaling so here's a few examples there's one example that shows how it could be done this I have to say that this is defined in portrait mode for instance the a real iPhone at 312 pixels and a dpi at 160 so it would be to make a page look a page perfect on an iPhone so you set the width to 320 and then maximum scale of 1 so you can't scale in doing any figures today as people have different devices with different resolutions etc it's possible to define like device width so if your device has 480 it will lay out as with 480 as the width what's the big issue with web apps is that for instance if you have a bottom a bottom, a scoreboard whatever you have done how it's shown to the user the size you will have like how it is big in comparison with your finger that depends on the dpi of the device if I have a high dpi it might be so small that I can't really interact with it and it's not what you want for a mobile app for a web app the dpi of the iPhone has a dpi of 160 so when this problem starts to arise especially with Android phones coming up with dpi of 240 people start to figure out how should we solve this issue so Google they made a definition of something called density in a different pixel and that's defined as a pixel on a 160 dpi phone that is because most of all web apps today are designed with the iPhone in mind and now that the Android and the other application does exactly the same this is how it's going to work so if you look at French and Firefox in the Nokia in 900 Android device will keep going 240 or beyond they actually scale up the content so if you have a 320 on the page and you actually scale it up by 1.5 2.4 about 160 this is not what you always want because oh about this really nice device and this beautiful screen nice dpi so Android has something called target density dpi where it's possible to override by default I think it's going to be medium dpi with 160 where it's possible to send dpi and then via css figure out if it's scaled or not so you can load a different css file with better resolution with images better resolution so summing up making an over browse with touch devices has a lot of different requirements than making a desktop browser our focus is on the touch interaction on senior content so we need touch event we need this engine application separation and a lot of small tricks to make everything work better I've just shown a few of them today but there's a lot more a lot of ideas you can do to make this really work and for doing web apps the application needs to support the web order to decide whether the browser should support scaling maybe whether it should even show the chrome etc. so I've just shown some good questions yes so I guess QWQS no QWQS QWQS is based on normal QWQ QWQS that's from other links in the end yeah I guess you can do that just like I said it's based on normal QWQS it can run on QWQS so you could use QWQS you can even if you are in another women's room it doesn't support that it's used to use it because QWQS is possible have a different set to frame of second it's something that you can do well he was trying to ask about if it's usually like a QWQS on an event platform with QQ how you can ensure having like 60 frames per second it's not something that we in QQ can ensure that you have 60 frames per second on a free device it's something the device has to do with if the device can support like 60 frames per second for panning pinch zooming image well then with this idea in detail today it's possible because the painting is done separately from showing it for example for most mobile devices you can make this have maybe the OVGL textures so the shared texture you write to them the scale and everything is done in the graphic style you would need to provide a platform that can give you 6 frames per second between pinching panning and rotating an image and there's something you have to be careful about that is the texture output usually with OVGL these are different types it's not just running this it's a tile image it's made out of tile so every time you have to to blend something you have to maybe upload that texture stop it a little bit while painting while panning not really the way this can be done is like I said use a shared memory area so you can implement it the way you want this is just basically the idea of how to do it implementation can vary depending on your hardware yes so if you want to so that you can get basically the whole it's possible to take some variations and then you can work with the tiles often there's a theory for that what do you do if you're panning actually having a tile is always a bit of what you're saying it's just being in a large area so while I already have a tile outside I might make a decision that always paint two tiles outside a physical area and then just paint them when they're ready so you just schedule a layout paint these tiles when they're ready but don't paint them without these because they have memory constraints as well so it's all like a trade-off something that it's performance you have to find out what makes no sense in terms of your experience of course the idea is that you should use all the memory of the device and you should try to make it possible to make it almost impossible to see the check of all the areas so that you don't have to wait the whole time but this is like there's a lot of things that you can do now yes do you use only the dpi scale for the most of the wire or the schedule well the dpi I was talking he was asking whether we were only using the dpi for the font scale or for everything well with the dpi in the middle for everything yes it's basically as I said we're painting the content with the scale so we just take the dpi and look out he was asking how responsive should the application be for the user to notice well that is basically what the 60 frames second is about that is what your eye can register so that you can do better than 60 frames or at least 60 frames per second then that's what you should aim for you don't really need to have something like 50 frames per second like more than 60 frames per second you don't really need that unless the user can perceive ok then thank you very much for coming thank you