Google I/O 2010 – Keynote Day 2 Android Demo – Full Length

ladies and gentlemen please welcome vice president of engineering for Google Vic Gundotra well good morning everybody you made it even at this early hour I hope you enjoyed that party last night how yeah how how about that spider was that cool or what after a few drinks I thought that thing was going to chase me a lot of fun let me also welcome the many thousands who are watching our live stream on YouTube yesterday just for your information we had over 24,000 people watching it concurrently live so in addition to the five thousand plus folks we had here we had almost 30 thousand watched us yesterday and so welcome to everyone watching on YouTube to begin today's keynote I'd like to start with a story it's a story of my very first day on the job at Google now I'm sure you've all been at a new job you understand the apprehensiveness you might feel with a new office new people it was on that very first day that I met a man named mr.

Andy Rubin now I suspect most of you know who Andy Rubin is at the time he was responsible for what was then a secret project codenamed Android and on that first day Andy enthusiastically described to me the team's mission and purpose and as he spoke I'll level with you I was skeptical in fact I interrupted Andy and I said Andy I don't get it does the world really need another mobile operating system Google is about advertising shouldn't we be on every phone to this day I remember Andy's response and he made two points the first point Andy made was that it was critically important to provide a free mobile operating system an open-source operating system that would enable innovation at every level the stack in other words Oh AMS should be free to build all kinds of devices devices with keyboards without keyboards with front-facing cameras two inches three inches four inches that operators should be able to compete on the strength and coverage of their network 2g 3G 4G LTE CDMA and that in the end with innovation coming at every layer it would be the consumer who would be able to benefit by getting the best device out the best network for them I remember Andy's second point he argued that if Google did not act we faced a draconian future a future where one man one company one device one carrier would be our only choice that's a future we don't want so if you believe in openness if you believe in choice if you believe in innovation from everyone then welcome to Android now let's get started let's talk a little bit about the momentum that we've achieved in 18 months even a year and a half since we've been started with Android how are we doing let's do a little bit of a report card first of all let's demonstrate some momentum it's hard to believe that in only 18 months we've achieved over 60 compatible devices where your software where your applications can run and these devices are not just from you know people you haven't heard of these are from the leading consumer electronic companies in the world Sony Ericsson HTC Motorola and many others who are producing devices that meet the needs of consumers we think this is pretty fantastic progress in just 18 months of course it's not just the creation of the devices it's the 2100 EMS in 48 countries and over 59 carriers who've joined the Android revolution of course producing devices making them available across a multitude of countries and carriers does it necessarily mean that we're going to see adoption have users found Android to be something that they desire well last year we reported late last year that we had reached a sales run rate a daily activation rate of over 30,000 units a day in February just a few months later we announced that our daily run rate had achieved sixty thousand units a day I'm very proud to announce today that our run rate daily activations has now passed a hundred thousand a day go Android of course that momentum has led to some pretty significant milestones one of the ones we're most proud of is that this quarter we are now second in the United States in smartphone sales second only to rim and that's pretty amazing progress in 18 months we're second in smartphone sales but according to AdMob data we are now first this quarter in total web and app usage that's fantastic you know we set a crazy internal milestone for ourselves you know the phones are being used by consumers but what are they being used for and we set a crazy internal goal when we shipped turn-by-turn navigation for Android six months ago we thought it might be possible to have a half a billion miles navigated in the first year in hardly six months we've now crossed a billion miles navigated with turn-by-turn navigation on Android so users just love that feature thank you there are some who say that users don't use Google search on smartphones well we're a company driven by data not by opinions and you know what the data shows the data shows that we've seen a 5x growth in the past two years that's not just on Android but that's across all smartphone categories people love Google search you give them a great browser and they do Google search tremendous awesome usage of the web on these devices of course what they love is applications and today I'm happy to announce that we've crossed 50,000 applications in the Android Marketplace and really the credit there goes to you thank you for the 180,000 developers thank you 180,000 developers who joined the Android revolution really it's your hard work that's paired with the innovation that's coming from OEMs and carriers that makes the mobile ecosystem work we certainly couldn't have done this without you thank you thank you thank you for supporting Android now let's talk about the platform we finished the section on momentum let's talk about what we're doing to make the Android platform continue to evolve and get better now in this section I have over 20 demos 22 demos I believe in order to help me with that I'd like to invite up onstage Matt Waddell my partner in crime here many of you may remember Matt from last year and we're going to go through a number of demos that's going to really showcase what we're doing with Android there are five major areas of investments that we're making in the platform now we've been quick to iterate with Android in fact there's been seven releases in those 18 months and today we're announcing the next release Android 2.2 codenamed Froyo what's in Froyo let's talk about five pillars number one let's begin with speed now as you developers know the Android architecture is one that's built upon a virtual machine the dalvik virtual machine and we think it's very important it's in a critical design decision we made that future proofs your application we have big dreams for Android and part of those dreams mean that Android will go to new places with new chip architectures but by having your applications write to the virtual machine why we believe we can go carry the entire ecosystem to exciting new areas of course that only works if the virtual machine is fast and the dalvik VM has done its job being fast efficient and automatic and easy for developers but we can do even better and we're very proud to announce that in Froyo we've added a a just-in-time compiler which gives up to 2 to 5 X speed-up of your apps on the exact same hardware now this is best demonstrated and let's do that let's go to our first demo we're going to show you a game the game has been modified for purposes of this demo you guys may know replica Island here's what we've done to modify the game the game now will show the frame rate that it's rendering in the bottom right hand corner if the framerate drops beneath 30 frames a second the screen will flash red exact same Hardware exact same game top is running Froyo with JIT compilation the bottom is running Eclair we're going to introduce a crazy number of monsters into this game so you see the monsters keep getting added increasing the complexity of the game you see the framerate at the top and as we add complexity it starts to slow down you'll note there at the bottom there are times and we're dropping beneath 30 frames a second and it's flashing red you'll note at the top with Froyo exactly the same game runs much better never dropping beneath 30 frames a second all because of the JIT compiler alright let's go back to slides now as Android adoption is skyrocketed people have been taking these devices to work and we've heard loud and clear from the enterprise that they need very specific features well in Froyo we introduced over 20 new features designed to meet the needs of enterprise let me touch on two of them number one we become Microsoft Exchange friendly that means yes thank you that means things like auto-discovery integration with the global address book the security policies that are available in exchange can be enforced upon the device number two we've added new api's for device management so you can build software that does critically important things like remote light of the device if necessary and there's many many other things that we've added that you'll see in the documentation let's talk about new services available for developers in the SDK now one of the first services I want to talk about is the application data backup API now if you use Android and you've gone to another device you know that Android will automatically backup your applications in other words you get a new device you log in and your applications come along with that however what Android has not done is backed up the data associated with those applications so for example I have a particularly favorite application that helps me monitor my exercise and my weight but when I moved to a new Android device the application moves but all my personal data in history does it starting with Froyo will provide an application data backup API that is BS can take advantage of and move the data along with the application we think that's a great feature in addition to that if we have a brand new API a cloud-to-device messaging API now let me be clear this is not a push notification API designed to compensate for the lack of basic functionality like multitasking in the operating system okay we've done something very clever I think you're going to love as a developer you can send a message to our servers which will do very smart things like collapse similar messages our servers which will optimize for the latency of mobile networks and make sure that that message gets down to the device but that's only the first step we've done deep integration with Android such that when you send a message that message can trigger an Android intent let me show you how powerful this is let's go to a demo now what Matt has on the screen here on his laptop is Google Maps he's in Google Maps and he's using the Chrome browser and he's added a Chrome extension the Chrome extension is in the upper right hand corner and that sends it to his phone so in the map he's got directions he's got directions from the Moscone Center to Google headquarters and he wants to send that to the phone what do you think happens do you think we send a text message that says this is the address do you think we send an email no we send an Android intent in other words when he says send a phone the message gets sent to our server gets pushed down to the device the device kicks into navigation mode automatically keep your eye when he clicks on send a phone keep your eye on the Android device so send it to phone and on the Android device boom right into navigation how hot is that that's how you do a device cloud-to-device API let me show you another example you're reading an article on the desktop you think it's awesome but you're out of time and you need to run you want to keep reading that on the Android phone why don't you send that to the device map to the device opens up the browser takes you right to that article not ever having to press any other Keys not great we can't wait to see what you're going to go do with this API alright let's go back to slides you're going to love this I mean come on if you're like me you have a plethora of devices you carry around with you and that all those devices shouldn't mean added complexity in yet another bill right you should be able to at the platform level enable tethering and so now your Android device can in fact become a portable hotspot and really serve the needs of these other devices that you might have with you in fact let's show you a demo of this working matt has a nexus one running Froyo he'll go right into tethering and portable hotspot he'll enable the hotspot you'll give it a name and in fact I think he's already named this as Android AP and that hotspot will turn on we'll give it a second there there we go tethering hotspot is now active now you'll go to another device that doesn't have connectivity how about the app that iPad and there you go one bill isn't that beautiful all right let's go back to slides now it turns out do you know what the most popular thing people do with these smartphones what do you think it is fairly obvious it's they actually use the phone the number two thing they do is they do text messaging and the third most used application is the browser and so it's critically important for us to make the Android browser Rock and we're going to constantly improve that browser in Froyo is a major step in that direction what have we done in Froyo well I think you're going to love this we have a two to 3x performance in the browser how well we take this we took the same javascript interpreter that we had in chrome that make chrome so fast v8 and we brought that into Froyo now the best way to show you how much of an advance we've made is to do a demo so let's go and show you this fantastic performance boost now we're going to do this demo with a number of devices we're going to do it we're going to do this demo three devices we're going to do this demo with Froyo Eclair and an iPad now here's the demo we're going to do there's an industry standard test in fact there's two is a suite of 26 tests the SunSpider tests that really exercise JavaScript performance of all kinds and when Matt presses start across these three devices we're going to exercise each one of these 26 tests as we complete one test that little Android robot will take one breast stroke forward and as all 26 tests are complete we're going to complete an entire lap okay so how are we going to go do here let's go ahead and start this test let's see what happens oh you started the iPad first trying to give it a little bit of a lead here huh you guys here it's great I really wonder if we'll be able to get that in the app store oh it's a web app how great is that all right let's go back to slides we think with the performance improvements that we just showed you that we can claim that Froyo has the world's fastest mobile browser and that's a pretty great accomplishment one we're extremely proud of okay we're not done I showed you what's in Froyo but make no mistake about our commitment to maintain leadership in the browser on on Android one of the ways that we're doing that is we're working with standards bodies to enable web developers to get capabilities that were formerly only limited to native access to the platform what kind of capabilities well think about the things that you can access on a device things like the magnetometer the accelerometer think about things like the camera or being able to access speech wouldn't it be great if you could access this from from the from the browser we're going to show you a sneak peek at something beyond Froyo an early development build that will give you a flavor of where we're going next let's go back to demos now remember we already got some of this work underway last year we worked with the standards committees to introduce geolocation into the browser and today almost an all major modern platforms why you can make an API call from the browser get location with the users permission and then do a feature like my location right within the browser but we want to do more how about the accelerometer what about the Magda the the the access to the tilt and the direction of the device watch this isn't that great right within the browser all right now another key feature that we've heard from developers that they want access to in the browser is the camera here for example is the buzz web application we'd love to not only post a buzz but we'd love to be able to include a picture well how do you access the camera from the web browser well in this development build we've built those api's right into the browser such that Matt now can simply select if he wants to it looks like in this case this demo is not going to work we'll give it one shot and we'll move on if this doesn't work but you can access that camera capability right from within the browser and we'll show you that in the sandbox later it's great let's talk about another capability that's available but before we talk about that capability let me remind you of what's possible in Android Google starting about several years ago really made a deep investment in voice recognition you know we recognized that the mobile device because of its limited input capability would be the platform in which people used voice input more than any other platform in fact we see a stunning number of queries being done on mobile devices where the input is a human voice now just as a reminder of how fantastic our capability has become over the past two years let me do a few demos or more accurately Matt why don't you highlight a few cases pictures of Barack Obama with the French president at the g8 summit that's a tough one boom look at that because this is so fun let's do a few more pictures of the Golden Gate Bridge at sunset these are queries you're likely not to type in let alone get a response back you know a few seconds is that great and one more del dotto vineyards Napa my favorite vineyard Napa boom now what that when not just showed you was phenomenal voice recognition that we're now shipping in Mandarin Japanese English a number of languages and that's constantly increasing but what's coming next is also the ability to understand human intentions we'll give you a very simple sneak peek at the kinds of things that we're going to enable call fifth floor restaurant so in this case because he said call it got it trigger and there's many many more intentions we're going to go build in and make it very very simple for you to use voice input as a first-class way to interact with your Android device hey Vic let's give this one a try oh you got okay let's let's go back to the camera let's see if this thing works eh this is a web app it's a buzz web app there's the camera from the browser boom right into the camera isn't that great and now you can post a picture and give web developers the same type of capabilities that you'd expect in a native application glad that worked no need for you guys to visit the sandbox alright now fetch getting back to the speech demonstration what we showed you was capability that we make available to developers but what about accessing that capability from the browser here's a web app it is a Translate app from Google with one new feature you'll note the microphone right there at the top Matt if you could just point to the microphone and watch what Matt does can you help me find the nearest hospital Judy you media Judah you people you people awesome awesome alright let's go back to slides now we're not only committed to making sure that Android you can clap I'll let you clap you know I love it when I don't even have to speak to a slide and we just get applause I'm so tensed oh go on but let me let me just make a point here you know we're not only committed to having the world's fastest browser we're committed to having the world's most comprehensive browser it turns out that on the internet people use flash and part of being open means you're inclusive rather than exclusive and you're open to innovation you know this was driven home to me very powerfully when my daughter picked up my iPad and went to her favorite website a website Nickelodeon and this is what she saw on the iPad can we switch to the iPad a sea of orange she said daddy can i play with your android device and this is what she saw the full Nickelodeon site isn't that great that's what openness means and by the way a special thanks to Adobe for their incredible willingness to work with us engage with us on Android and in chrome in many other areas it's really fun to work with other folks in the ecosystem to meet the needs of users much nicer than just saying no alright let's go back to slides now we've also made significant improvements in the Android Marketplace we listened carefully to the feedback that we've gotten from developers and from users and I think you're going to like the enhancements let's talk about what we've done on average our data shows that users are installing more than 40 apps on their Android device and so finding those apps is a challenge also they want to search within the data of those apps they want to move those apps to the SD card and they want to update all without having to update each individual app by themselves let me show you through a series of demos what we've done here let's go to demos the first demonstration we're going to show you is just searching the apps and now with the quick search box we've made it trivially easy to scope it to apps so as soon as you start typing a letter Y the app you're looking for just comes up it's a simple and easy way to go find apps not only can you find apps but we've improved the quick search box so that developers can plug into the search framework in this case that icon is for mint mint is a financial application and now you're searching within the data of that application bringing up your financial records we think developers are going to extend this in all sorts of exciting ways making not only Android more usable but their own applications be surfaced discoverable and more fun and engaging for the user another issue we've heard is people want to take advantage of the openness of Android the ability for you to plug in an SD card with arbitrary amounts of memory and they want to move apps not just in the internal memory but off to the SD card two points we've enabled that capability in a secure way with Froyo but we've also made it so that the user never has to worry about it when they install an application will intelligently look at the space and if appropriate move it to the SD card if the user wants to get involved and manually move things around that's what this demonstration shows in this case Matt is going to take a brand new game Need for Speed and he's going to shift that game from the internal memory onto the SD card and there you go there the whole app 50 megabytes is being moved over why don't we just start the game just so we can see you can see it's on the SD card once you launch that game isn't that awesome great new game need for speed that's coming and it can live on your SD card okay let's move on not Matt we got to go I know the games fun all right let's go into the marketplace and talk about all right let's let's talk about update all functionality Matt that's you he's easily distracted all right today you have to update each application individually in Froyo we've made it simple at the bottom of the Android Marketplace you see the update all button I'm kind of embarrassed you have to clap on that we're glad it's there we've gone one step further with Froyo why should the user have to take any action starting with Froyo with the users permission you can allow automatic updating and all your apps are updated all the time is that great and the user doesn't have to worry about this at all let's go back to slides another key feature we're adding in Froyo is designed to meet the needs of developers we want the best apps on Android the highest quality apps and that means we need to close the loop when there's a problem let's go back to a demonstration here and I'm going to show you an app called crash e it does exactly that really well it crashes and so Matt why don't you start the app and it crashed now note that there's a button there now called report and the user can press report can provide a description of what happened they can look at all the data that were that they're going to go send make sure they're comfortable with that data and then they can hit Send and that is sent back to the developer note on this screen the Android Marketplace note a brand new tab bugs and when you click on bugs we will show you those reports for your application and we will show you the entire stack trace of what happened we thought you might like that and closing the loop allowing apps to get better and better all right let's go back to slides I want to go a little bit beyond Froyo and show you a sneak peek at other exciting things that we're doing in the android marketplace I think you're going to like some of these things let's go back to demo mat is now showing you the Android Marketplace except this is unlike anything you've ever seen before it's the Android Marketplace accessible from a browser on a PC and that's great now mat can browse apps he can look at reviews but you'll note in the upper right hand corner Matt is signed in and because he signed in we know every device Android device that he has yeah he's a little weird in this case the device that he has on the stage he has selected as Nutter Butter nutty-buddy something whatever he like I said he's weird he's got his phone selected and so now he can go browse a particular app he's going to find an app that he likes like open table and a hold on Matt hold on hold on he's gonna hit free and he's going to go and download the app what do you think happens well let's see what happens today on on some other systems you find the app you download it to your PC or Mac you then have to tether your device you have to then once it's down on your PC you have to then convert it over that tethering to your device and then make that sync happen well guess what we discovered something really cool it's called the Internet and when Matt presses download keep your eye on the Android device over-the-air isn't that totally great over-the-air installed on your device now it turns out it's called the Android or the marketplace because it's more than apps fact what we're about to show you is a new category how about some music Matt why don't you find an album or song you like and why don't you just send it to the device using the internet over the air and boom keep your eye on the Android device there you go there's the music coming right on down now we're not done yet some of you may be thinking this is really cool this is so hot but I have all my music already on my device at home my windows media or iTunes library I've got thousands of songs there what what do I do with those well we recently acquired a company called simplified media and it is our intention to incorporate that technology into the marketplace as well and that allows you at home to run a simple piece of software that makes all of your non DRM music available to your Android device would you like to see this lets go to your Android device Android device has two songs on it that's nice but I want more than two songs I want all of my music I have so Matt will go down and select I guess he has to plug in the device you can hear it adoptions touch 9 ok thank you for calling the 5th floor restaurant book my god let me offer a public apology now what's really odd is that they didn't pick up all that time Wow okay getting back to music to songs I want all my music he's going to select that button that says all and magically all the music is now available and you might be wondering what just happened we are making your home music library available to your Android device as as a stream so for example if Matt was to select a particular song I guess Matt's an Earth Wind and Fire fan hit play this work yeah because usually we're going to love this feature alright let's go back to slides let's talk a little bit about advertising it turns out that we know a little bit about advertising today happens or this year happens to be Google's 10th anniversary of providing advertising solutions and we've learned a few things in that decade one of the foremost things we've learned is that if you want to have a healthy ecosystem of advertising you need advertisers and we have hundreds of thousands of them that means if you have an ad spot why we have a relevant ad that can fill it it's called inventory we're not new at this game we're not working with a handful of partners and charging them a million dollars each to be part of our program okay so we can be your advertising partner we've also learned something else in that decade that advertisers and publishers have different kinds of needs they're local advertisers there's direct response advertisers there are brand advertisers and if you want to play you need formats that meet the needs of those kinds of advertisers and we have those formats a third thing that we've learned advertising needs to be measured and that means it needs to have great tools why does it need to be measured well you need to know if the money you're spending is giving you a return on the investment you need to know which campaigns are working so you can invest more in them or tweak them as necessary and it turns out that we have some tools you might have heard of called double-click analytics Adsense AdWords the tools that in the industry uses knows and loves are being extended seamlessly to work in the mobile environment and finally we've learned that it's important to be open to innovation now I don't even know what the ponies have to do with this slide but but part of being open is that when you get a designer who gives you a slide you go with it let's show you each one of these four points in and go to demos now these are real advertisers and real applications let me make that point clear you're looking at backgrounds one of the most popular applications available in the android marketplace at the top you're looking at a Google Ad our ad program which has been available only to a limited number of a beta participants is called Adsense for mobile apps athma and you're looking at one of those atma ads you're looking at our large inventory of text ads being optimized and served on that device we're taking advantage of the small space that we have and you're seeing the nice rotation but that's just one ad format what happens if Matt selects something like beaches well then we can serve a contextually relevant ad in this case kayak is an a popular website and application for travel but in this case when you click on the ad we take you to deep within the Android Marketplace so that the user can download the app okay that's another important ad format let's show you another format this is a third format for brand advertising in other words a banner ad where an advertiser has displayed as decided to convey their message through an image we support that how about another category and this is something that we're announcing available today and that's an expandable ad format one of the things that we've seen through our research is that users if at all possible would like to stay within the context of the application they're using so in this case when Matt selects that Volkswagen ad it expands right out you can look at that at and when he's done with it he can simply make it slide right back in and we think that's great for advertisers and we think that's great for users that expandable format is available today let me show you another brand new one we're making available today this is an expandable ad format this is a Flixter application and why don't you expand that ad out in this case it's not just display it's rich media he's linking directly to a trailer he clicked on it and the trailer can be okay and the movie will go ahead and start Matt don't watch the movie let's come back okay let me show you another ad format this is also an expandable ad format but in this case this expandable ad format has I'm sorry in this case I jumped ahead of you let's go back to the other one in this case I'm going to show you a very popular ad format that has been testing incredibly well this is a click-to-call ad in this case we're leveraging the fact that the user has given us permission to use their location we know that they're in San Francisco and DirecTV would like to make a special offer to customers in San Francisco and they have a new click-to-call option so simply clicking on that button on that ad allows the customer to make that phone call to DIRECTV and that's incredibly valuable let me talk about another ad format this is an expandable format with a very clever twist Matt why don't you expand this one out it includes a map and directions and click-to-call isn't that great that's a new format that will become shortly available as well okay finally let me show you what I meant when I said open to innovation this is the weather comm application and they're using the industry standard double-click software to serve this ad from Google the ad is at the bottom now when Matt clicks on that ad you get a full-screen immersive ad and it's a pretty amazing ad it's got trailers and TV spots and galleries not only does it have a bunch of media that you can interact with look at the bottom there the bottom has the ability for you to tap directly into Fandango to purchase movie tickets but you know what the kicker is of what you're seeing it's not a Google ad you're seeing an immersive ad format delivered by a great advertising company called media let's but because the publish because the advertiser excuse me the ISB the developer use double-click to serve the ads double-click is open to have any ads show up and that's what it means to be open that Weser the most relevant ad to the user sometimes from Google sometimes not but you the developer and the publisher have the choice that's openness all right let's go back to slides if you'd like to learn more about our advertising solutions we encourage you to go to mobile ads and to you know entice your interest a little bit we're doing two things today number one we're opening up Astra Adsense for mobile apps to everybody at the conference all 5,000 of you and we're going to sweeten the deal a little bit if you go sign up we will give you $100 free advertising credit to start learning the system and advertise your apps in the Android Marketplace so please go try that one of the most powerful ways to demonstrate what I talked about at the beginning that innovation comes from all levels of the stack is to highlight one of my favorite devices this is the device by HTC it's the evo device now there's several things I love about this device one thing I love about the device is it's absolutely gorgeous 4.3 inch screen I also love how fast it is I love the great work HTC has done to add value I love the Sprint network it's a 4G network you know what a 4G network can do it can give you a peak of up to 10 megabits per second imagine which you can watch on this device at 10 megabits per second I love the fact that this device has a great little stand so you can just set it on the desk or if you're on an airplane just set it right there and watch your content it's got a great battery it's got HDMI output it's got a camera that's 8 megapixels and that will do 720p HD video recording and you know what I love most about this device is that in partnership with Sprint and HTC Google is going to make this device available to every one of you today Oh but those of you who are watching on YouTube I'm sorry remember to register next year quickly I think you're going to love that device we ask you to do two things number one don't run out right now after the keynotes are over we're going to make it available all day until 6 p.m.

so you'll have plenty of time to get this device and really get to see the latest in Android innovation number 2 thing we're going to ask thank you for supporting Android thank you for voting on the side of openness and choice please keep building those great apps now we're at the halfway point this morning we're not done yet if you'd like to see the next step in the evolution of Android and where we're about to go next hang tight.

As found on YouTube

Build Fast and Smooth Web Apps from Feature Phone to Desktop (Google I/O ’19)

oh right good morning thank you for showing up at 8:30 I am Mariko I'm from web developer ecosystem team within Chrome let's talk about building fast and smooth smooth web apps so it's safe to assume I think that all of us use Internet using computers touch device or your phone every day but what if I tell you that millions more people are accessing the Internet or coming to access the internet using device like this a feature phone so for those of you feel remember a days of feature phone you might remember typing the text on those keys I actually have it here I grew up in Japan where the first popular mobile web network was built and using feature phone to access the website and doing all of the emails and text was like my part of high school and college years so it feels City nostalgic to me but also I went into the industry after that so I remember the days building a website specifically for feature phone in Japanese we call agaric a site oh it's really hard it you needed you to subset of HTML or sometimes completely different markup language yeah so bad memories a little bit but don't worry these phones are new phones that's coming out to the market it's developed right now sold right now all over the world and it's not just me telling you that these phones or popular according to a counterpoint research 303 hundred and seventy million smart feature phones are expected to be sold between this year and 2021 so what is that smart feature phone right this is a new feature phone so smart feature phone means it has a new OS sometimes such as kayo s there is a few other OS to Cairo S is awesome because it's all web-based so you can build an app using web technologies it has a modern web-browser it's a little bit about virgins behind but it supports HTML CSS in JavaScript for those who develop the website for feature phones in Japan it's a great news and then it also comes with app ecosystem which means you can download Google Maps YouTube games on your phone just like you would do with your smartphone so software wise it feels like a smart phone but it comes in the sides of hardware so the screens are usually small I think most of the KY OS devices or QVGA screen size the navigation wise it does have Mouse shows up on the screen but you have to navigate it with your d-pad and if you want to input anything any text on the field you have to use the number key a and the t9 key input to do that so that part is a feature phone so what does it look like to browse the web today using these new feature phone here's a website you might have visited once or twice before Google com no it's great it looks great but we all agree that Google com the top phase it's just a bunch of links in one input field so huh that's not impressive let's see how it's like to build load a web apps right so here's a Skoosh it's a web apps using web assembly you can do image completions it's a PWA we build it last year for c.d.s how does it perform not great the CSS layout is completely off so the thing is when we build a Skoosh our team built it we wanted Skoosh to be the best in class web app so the first load is only 16 kilobyte it lost nearly fast even on this feature phone but we didn't need it quite tested on the screen sides this small we tested it on the desktop tablet and iPhones but we never thought of having somebody accessing the site using these phones news for the one of the depressing layout of the website is at i/o website not great not great it's so frustrating to navigate i/o so using this or oh it's almost unusable but not everything on the web is bad my favorite is Twitter so Twitter mobile I'm evaldo when you access that on the smart feature phone you almost get the same exact value you get the same exact same experience you can tweet you can lead to it you can load video you can search for gifs you can like a touch image everything that's provided in the Twitter website available on the smart feature phone that brings me to the project that I'm going to discuss today so I briefly mentioned I am part of web developer ecosystem team in chrome and within that I'm in the small subset of the team that tried to build a real-world web applications that we can share so that we can share our learnings by building something new right so our first project what Skoosh it's a image compression application that completely built within a browser we use what was them to provide a new were file formats like web P for browsers that doesn't support web beat and you can select all of the knobs and like bunch of settings to see how much better the compression can be and you can download it and I'm upload it to your blog so that's the idea and then after we build this last November we got to discussing what should be the next project and we decided let's just build a game we wanted to build a game because whenever we asked web developers what is the way of really good ad everybody says it's a document and we wanted to say completely opposite of document to see if we can handle that so games seem like a good topic and developing a game came with a bunch of problems that we faced as a web developer everyday such as like how do we handle a lot of inputs coming from all of these UI or can we really provide a graphics heavy application on the web and on top of that because we were hearing that the feature phones are getting popular decided that our apt this year it's going to support everything from feature from the desktop and everything in between so I would like to explain what we built first and then we're gonna get into how we build it introducing procs proxy the game of proximity inspired by the legendary game a minesweeper game is situated in the space and your job is to find black hole you can play it works on any kind of devices from desktop to tablet to d-pad even with screen readers one dimmed button column 9 of 16 hidden button column 10 of 16 it is a PWA so you can download it and save it on your desktop or on your phone and play the game wherever you want even when you're offline so that's the game we built and you can access the app at the procs dot app that's the URL so let's discuss how we build procs so even before we go into the project we decided on the baseline like three of us got together Jake Sora and me and talked about like what is this app going to be three points we decided that every device will get a same core experience meaning we are not gonna build a three different apps for desktop tablet and feature phone we decided that it has to be accessible both on the input device so like all of the mouse keyboard touch and d-pad to support it and then we said why not make sure that the screen readers accessible to performance wise our team Nene likes to build a performant web app so we said it has to be really really good performance so we said our performance budget to be initial payload of less than twenty five kilobyte time to interactive less than five seconds on throw 3G network and animation should learn 60 frames per second whenever you can so let's with that understanding three of us got to working so let's let me explain how this app is laid out so the game started with a game logic file the jaicro on the way on the long-haul flight because he wanted to play a minesweeper but there was no internet on the flight and he wrote it he's that kind of engineer game logic just contains the logic there is no UI element to it it just of like how big of the field is where the mine is and with certain points are clicked how should it the deal that kind of thing on top of that we built a new iron state services so you I we use pre-act and State Services is a thin leper on top of game logic to communicate game logic and the UI service we also rotate our own landing pipeline and which we will get into later later so for y and we also have a little bit of utilities to goo it all together so simple issue you can understand all of this could learn in main thread everything in one file well it could be separated but it could learn in main thread however from the get-go we knew we wanted to do graphics heavy design we wanted to animate it and we were like not sure about this so we decided to move a game logic and state service into web worker so web worker those of you who are not familiar is a way to learn your JavaScript off the main thread and separate thread to communicate between worker and main thread you use API called post message and it's like not an enjoyable experience keeping track of the message passing it's it's a lot of work likely team members are my foods sitting there wrote a library called a comlink it's a sin leper or abstraction on top of post message to make using worker a lot more enjoyable in fact we improve the comlink by doing this project calling vfo so we were working on this project using comlink and testing it on the feature phone and realize that it's not really working great and the eyes data comm link has a processing in intensive work happening so we fixed that and comlink v4 was the East so if you're interested in doing our offloading all of the tasks to worker you should definitely check out connect UI wise we use pre-act we chose pre-act because hmm three of us use pre-act on the previous project and we liked it and and also it's still the smallest UI library out there that fits in a performance budget the app wise it is a standard single page application all of the custom elements are blended on one div and then appended into the body but we knew we have an aggressive goal of initial payload less than 25 kilobyte so we decided to do a kind of strategy where we will build time pre-render the first load the first interaction using a little hack using puppeteer so basically we have app and whenever we build our up we learn a puppeteer and puppeteer will download oh by the way puppeteer is a way to control a headrest Chrome from your script so puppeteer will open up the headrest Chrome do the thing store download app react react will just build a HTML or put it into the Dom and then we just grab whatever the output that was in there and put it into index.html and that's what gets uploaded to our static site hosting notify and then that gets served to the user as a first payload this is just showing you how easy it is to start with puppeteer you just started instance browser puppeteer at launch and then create a page go to that link evaluate that link and just put it into the HTML and that's all there to it so the basic architecture whites two key points we use Walker to flee app the main thread as much as possible because we knew going in we want to do gothic heavy thing main thread can only up the graphic stuff main thread only can only do like Walker cannot do the graphic stuff so free that app for graph and then we also plea lender at the build time for speedy loading of the initial bundle so that leads me to talk about graphics perhaps the biggest performance choice we made was to have our own graphic rendering pipeline so uh when you show up Ram was to completely use the Dom we were thinking oh yeah we can just have a table and then put a bunch of buttons in it and you know we can use the CSS animation like transform opacity that think that lands on the GPU to do the animations and that would be great right well it turns out we think that we might hit a Chrome browser bug where when we what when all of this was in the one layer and I want to update a just a single button chrome was the painting entire table which is not a great for performance well way to solve these problem if you see it is to put a buttons or some elements that you want to update in the separate layers using things like world change transform but we have a lot of buttons on the game because each of them ice the game cell it's going to be a layers and that might solve a painting problem but then it creates excessive amount of layers and that hooks up the memory so we are creating another problem so we're like hmm this louder is not good we should go for another loud so we decided to do our all of our graphics in canvas in fact we have two canvas or no screen one is for background animation and one is for each animation that is doing the game cell so these are generated and Leonard every frame 60 frames per second using requestanimationframe so if you're not familiar with the crystal means your frame browser requestanimationframe is a way to schedule your script and every tick the browser the flash is the black graphics so you put some kind of JavaScript in our case the drawing drawing call for the canvas and you put that in callback of the request animation frame and next stick that one at the next stick and if you're doing animation stuff you probably want to recursively call that so that you put that task into every frame and then this is how we update our animation if you were curious about all the stuff that I was talking about painting layers compositors CSS requestanimationframe I wrote a full part blog series about inside look at modern web browser that explains what happens when you down when your code gets the browser and how its executed so you should check that out and so we have to canvas now and there's a few other things that we did for the graphics for performance for example the background animation which we call it nebula animation in fact it's only a 1/5 of screen size so whatever device you have we only create a canvas size 1/5 of your device and just stretch it out to full screen we were lucky because the design the designer came up was already blurry image so clearing a small one and stretching it didn't make much and saved us a lot of memoria do that so the for the grid animation weekly we basically do a sprite animation and we generate these sprites on the client side so we do not send any image data down the wire to users we send the JavaScript to generate the sprite and then JavaScript creates the sprites and once that's done it's saved in index DB this way we can create the most optimal sprites for each of devices so different devices have a different device pixel ratio some of them have won some bottom two sometimes three so this way any device is accessing our site we don't need to create an image they can create an image or two sprites on the client side so that's graphics let's talk about accessibility which is exciting point that I I work so nice I'm really excited so as I explained the game is now into canvas and we can totally build the game just with campus a lot of games do you know whenever use abuse of mouse or clicks on you just get the coordinate of the mouse and then you write your own JavaScript to say hey did it hit the square underneath and then you just lead roll the animation but we decided we are going to keep a Dom that table and buttons remember that was throwing a painting Yeller so we just fixed that by putting in opacity zero there's a reason why we kept the dumb version on top of canvas because if we have an element we can focus on it or we can attach our event listeners so when you are playing the procs what you are seeing on screen is a canvas but but your javascript is interacting with is an invisible buttons and tables and this way we can tap into browsers native accessibility features so here's a screenshot of me playing a game with voiceover it is written down to what the voice whoever said it says hidden button column fifteen or sixteen hidden is a state of the button and that's the only part that we manipulated from our JavaScript by adding a Ilia label everything from button to column sixteen fifteen or sixteen kind of like suggesting users where the locations are that just came out of the box by using the table so when we clear the start a new game we generate a table right clear the table and in theory you should just add a low grid and the screen reader things should be all taken care but somehow maybe because we are displaying the table opacity:0 the browser wasn't quite legislate as a grid so inside when we create all of the lows and columns we also needed to specify that this TR element is a low and this TD element is grid so and that's all this problem but at the beginning we were like why is it not working the documentation said that if we put the grid it should work so that was a fun challenge to do code or foe and inside of each cell we generate a button that user click on it speaking of buttons we use a accessibility technique called tab loving tab index when we clear a button for each cell first top left corner of the cell gets tab index of 0 which means it's tab accessible when keyboard navigation user hits the tab the focus on that button but then everything else we set having the X of minus 1 which means it cannot access it cannot focus by having it this way the keyboard user doesn't have to tab hundred times to get to the end of the game so that they can get to the other menu button down at the bottom of the screen and when the key screen keyboard user access the game they focus on one cell and they switch to using an auto key and a low key will emit like which direction how many times that the focus should move so the cell focus method basically takes like okay current focus is here just gonna make it and on top of all or focus on focus of all and then the new button that will be focused it's going to be focus of all by putting the zero and then just set the focus let's set the focus and this is how we implemented loving tabindex I did not any I did not know any of this until I did this project embarrassingly we got a lot of help from Rob Dodson who is on our team who does a lot of accessibility work and he has a guide on web dev for accessibility to all which explains all of the techniques that I explained right now so you should definitely check that out and thank you very much Rob so this is a half way of the project and we were feeling great our animations were learning future phones game was working fine but then we notice you know I explained that they feature phone to desktop everything in between so we were testing iPad Android go phone which is a low and side of the Android phone and we were noticing that the game is clashing on Android golf on quite a lot we were like how like feature phones or the weakest powered phone enjoy Goa's should be upgrade why is it why is it crashing what turns out how many pixels are on the screen matters a lot so on the feature phones case they may have a weaker chips but they only need to care about a certain number of pixels to drive right on under the girl phone which is a smart phone with touchscreen with much bigger screen they might have a slightly powerful hardware but they have four times or more pixels to drive and that makes them sad and just shuts down some time so we decided that okay at this point we should just check if the hardware can support animation and if not just lender a static version so basically when the game allows we do a little bit of a check saying can this harder we're a support animation and if they can we load a WebGL animations and if not we support a canvas city static graphics studied graphic is something we need to make anyway for accessibility for those who prefers a reduced motion action so we just expose that to a little lower grade hardware phones – so this is just a our our applets – checking this might be a little naive but this is just one check shader box this class is just a class abstract on top of WebGL and we check a high precision vertex shader and if that's possible then we just learned animation and if not we go with a studio animations oh sorry 2d static Len during we we know it is a little naive but we found that the devices that supports high P usually can handle animations of course if user have a preference set for prefers reduce motion we check for the media query and then that becomes a default – let's talk about supporting different with devices because I've been kissing keyboard touchscreen deeper right so game has a two main functions click to open the cell and click to flag the cell when you are playing on the mouse it is just regular click and right-click if you are doing that on the keyboard then we assign a toggle button to the F key for the flag so you click on it and then you can go back and forth with the mode and then navigation wise so you use number key and enter I sorry the arrow key and enter for the phones and tap we went with just tapped and toggle method so you can see at the bottom we have a toggle button and every time a user wants to switch to that mode you just click on that toggle and you might be feeling looking at this play video of a phone saying like I want to pinch zoom in to see if how many cells are left right I want to do means out to see like how our game is doing and this is something we discussed while we were developing this and we actually needed to discuss whether to do it or not because we had this goal of the app has to be performant planning smoothly we had this debate about like yeah but if we support pink zoom then we lose native scroll and that means scroll gets slow and it's that ok so eventually we just decided that we for this app we are going to go with native scroll over pinch zoom so we're not supporting the pinch zoom yet maybe hopefully the thing about pinch zoom is that once you have the pinch zoom action then you lose the native scroll and you need to implement your own scroll physics right and that it's not gonna be comparable to comparable fast to the native scroll and on a web platform we don't have the way to tap into browsers scroll physics we would love to have them and we would love to explore the possibility but for now native scroll only which also gave I also wanted this interaction with a double tap to flag I was like why do I have to toko the mode can I just double tap and if like it and it was immediately shut down because of performance if you implement the tablet app that means we all have to wait for single tap to check it's a double tap is it double tap which means we create a few milliseconds delay for the user the interaction so based on the baseline that we agree that it has to be smooth we said no that what up possibilities or in the future hasn't implemented it long tap holding on the thing we can do that performant Lee this is just a question of a UX and design we don't know how to flatter notify the user that you hold it enough to flag it now you can take it off and you know the black hole is not gonna get revealed so once we figure out the design we might implement it which brings me to this phone that does not have any of it yeah I mean it has a click you can click the button but like there's no right click there's no touch screen there a toggle key for F F doesn't exist it's only number key so what are we – we yeah we added a custom key navigation to the number key so when you click on five the number go the sales focus goes up when you click on hit on zero goes down and then when you hit on eight it is a click action and then when you hit a hash find the the sharp side then that's the mode toggle so another thing we found was that we need to show users where the focus are it's very hard on these small screen to see which button are they about to click they also have a mouse indicator or two but it's sometimes you can't really see like it's this mouse pointing to the one button or the other button so we made sure that we highlight the focus and tell the user this highlighted element will be open once you click on it another thing we added which my favorite is a key shortcut guide so if you access our game on the feature phone then you will see these tiny icon indicating you can click on the hash button to start the game or you can click on asterisk button to open the informations this is piece of UX that I took from a 2000 mobile development in Japan so the Japan had a feature phone web network and when you go to a long document site or something they usually have a table of contents on top and then the in page link inside and those are usually map to number keys so you will have a number emoji right next to the table of contents indicating oh if you want to go to chapter 3 just click on 3 and then you can just move down so I took that UX and then put it into this game another thing that's really important if you're designing a website or with game for feature phone it's to have a way for user to get away from that view so I have a close button there whenever user opens the settings model which is quite a long because it also contains how to play the game and Scrolls down whenever they think oh ok I got it they can just tap up though press on the asterisk key and in just close that model if we didn't do this UX and have the standard design that we have for the smartphone and desktop and all which is a floating X button this happens in the middle of the page user have to scroll up up up up up hit the top and finally the mouse can move to that close button to close it and this is really frustrating so the element itself the floating X button and the close button at the bottom is the same element but depending on the device if its feature form or not I just changed the CSS to put the location differently so that's the feature from designs let's talk about a fine strategy as I talked about this game is a PWA it used serviceworker to cache all of the resources so even if you're fine you can play the game and whenever you do offline there's always question of how do we update the game if there's a new update is there we might have seen these Mauro saying like hey update is available little this app or dismisses and then use the old version this is directly took from the previous project that we did but in this case we did not want to block the users of wanting to play the game or playing the game right now so we hit this logic inside of this page this button to be exact so whenever user comes to the app and hit start and whenever there is a network they make a call to hey is there updated and if there is update then it starts fetching down the new version and once that's done it Lowe's a new version of app skip the opening screen because we already know the how the game settings for it and launch it into game directly so when user sees this page the game is already updated to new versions and this is how we do the offline and versioning lastly I want to talk about resource loading so after all of this WebGL and feature phones and all of that our total packet became a hundred kill by gzipped we feel quite good about this site and out of 100 kilobyte gzipped packet twenty kilobyte is a first payload so we hit the goal of under 25 kilobyte D zipped first payload basically this is just an index.html that gets sense when the first request goes in which means this index.html contains this page I like actually this page so all of the animations and the opening title role that we hunter crafted with CSS animations or lazy loaded so this is the minimal set of features and the buttons that user need to start interacting the first action the first action could be starting the game first action could be opening the information icon clicking the information icon to open the settings or clicking the fullscreen button to go into the full screen mode so we even subset the font so we look at all of the glyphs that's used on this page subset the font and inline it into the index.html so nearly our index.html is the twenty kilobyte of data that kind of look like this yeah all of a lot of inlining but once that gets to the users and users start interacting then little by little chunk so download at least lazy loaded and then game fully interactive for doing this we used all up we really loved and enjoyed using dollop we even wrote our own plugins for things that load up didn't really provide out of the box but we even felt comfortable doing that and kind of mixing and matching it which was not the case on the previous project that we used a different building process and Lola worked really great for our setup using Walker's so as I mentioned our codes are separated into the worker and main thread and commlink is a shared dependency to communicate each other if you do this in web pack then webpack creates a two different chunks of the dependency and you know separated it and that's just duplicating it but load up out of the box just keep it as a one chunk and then share that as a dependency for worker and the main thread so this was out of the box great feeling for a project for module loading because JavaScript modules are not supported in web workers we used AMD and Soma wrote a a tiny pregnant called Lola plugin Lee loader which is a AMD like loader that is Leedy tiny specifically made for all app output so you might want to check that out but that's part of our build process and even doing that tools cannot be the help the fine tuning of shipping down the data we needed to go in and look at our index.html and what gets loaded and then see why is our indexing getting bigger and bigger and bigger so if you wanna check what kind of refactoring we did there's an epic PR up on the github code I need stuff smaller by Jake and things that he did or things he discovered was things like this so our game screen has a element called top bar that has a you know the number of cells that's open the timer counter all of that but those are only for game time right whenever the game is not learning like opening screen or win screen it's only just a title banner but when we are loading the index dot HTML which only need title banner they also load the logic of timers and logic of the open count and everything so we separate that element into top bar the full on version and top bar simple and just load it two differently and those shot some data so this is great if somebody have a time to go in and dig through and every now and then check if we're doing great but we try to keep reminding our self that every P for the quest that we make be conscious about the sides so every poll request we make on our Lippo we run a little script called Travis size report on a Travis CI to just check what changed it is says this file name changed or this file size changed and you know this screenshot isn't you know particularly interesting nothing really changed but sometimes you find an unexpected change like oh why did this file name changed or why did this file were suddenly this big so this was a good reminder so that's the process we took I would like to end with the three learnings that we had I definitely think that having a set baseline for the project was great we started with the story the project with understanding of what's important to us for this project and how we make decisions and that get got us you know showing up to the stand up saying hey I want to implement double top and immediately Jake says no you can do it's not performant and I'm like I wouldn't be offended or not feel like defending because I was just going oh that's right performance was an important thing for this project another thing we think the worker is a crucial for learning a smooth application we need to learn JavaScript off of the main set as much as possible I don't think we could make this game possible on the future phone if we didn't do the worker lastly if you're feeling like I'm not game developer I'm not gonna do the game on the web one thing you can take away from this talk is just study what's on the first instruction for your website or your web applications and just remove everything you don't need that makes your first load data small and an user can get to your service quicker so if you want to check out the app here's the link for the game all of us source code are open sourced on the github so you should check that out future Lee Quest or bug fix yourself very much welcome and if you have any questions or want to play them a game on the big touchscreen all of us will be at the sandbox tent a after this thank you very much [Music]

As found on YouTube