hi there my name is Johnson uh today a client has called me and they want to actually rent a property uh the property is somewhere around there uh correct this side maybe okay somewhere around there that area is on the first floor so uh the uh the owner or the agent uh want to actually rent this place out and it's on the first floor so we're gonna do shooting and I'm gonna show you how the process of doing this 360 shooting by using a mobile phone and this process that I'm going to show you step by step how you should to go about to do all the shooting okay Whenever you go to this kind of house you must always wear your mask abide with Government Regulations.
The Real Estate agents having trouble and that's why we are doing this virtual tour so that the client will be able to see Virtually before they come and do visit for the final time, so please wear your mask properly and I'm gonna go and get my gear. Camera Stand is a very important tool that you must have We reached the house and the owner has given me has given the key so I'm going to use the key to open it and because of the Covid many people don't want to visit the home so uh 360 virtual is one of the ways. 360 virtual tour will be the future so here we in virtualtourcreator.my we have created the best way for the real estate agent uh to make even for a budget property this is a budget property with two rooms one bath and hall and it's a there's a very interesting balcony here very hugely that is a good selling point so I have the key.
Okay this property they have two entries one of the entrance is here which is the main entrance and interestingly this particular house, the owner has actually sublet one part of it to another tenant. now this door here belongs to the sublet tenant. The balcony is one of the selling points for this owner so I'm gonna do few shots and before I do the shots let me bring you in. Our shoe is one of the things 360 photography people may forget so if possible please hide the shoes. As you see when we go into a house the amount of light in the house is lesser than outside. In photography, this is one of the very important things to look at in 360 because when you capture the 360 light intensity. We call it EV (Exposure Value) now the EV outside is high inside is low so possibly when you enter into any house please on the lights.
We need to have enough lights in the house so just on the lights but don't on the fan okay make the fan stationary. See in part 2.
[MUSIC PLAYING] [VIDEO PLAYBACK] [PAPER CRUMPLING] [MUSIC PLAYING] [SQUEAK] [SQUEAK] [SEAGULL CRYING] [CARS HONKING] [ZAP] [CHALK ON CHALKBOARD] [CAR HONKING] [SCRAPING] [TEARING] [CHEERING AND APPLAUSE] [CHIMES] [FOOTSTEPS] [BIRDS CHIRPING] [TAPPING] – Hm. [BIRDS CHIRPING] [POP] – [GASP] [CHUCKLES] [MUSIC PLAYING] – Hm? [BIRDS CHIRPING] [HEAVY FOOTSTEPS] – Mm. [TING] [THUNDERING IN DISTANCE] [RAINFALL] [THUNDER] [THUNDER] – [STRAINING] [THUNDER] – [GASP] – [MANY STRAINING] – [SIGH] – Hmm. [GLEAMING] – Huh? – Oh? – [GASP] Hmm. [CLACK] – Woohoo! Whoa. [ROCKETING] [TAP] [THUMP] [END PLAYBACK] [APPLAUSE] SUNDAR PICHAI: Good morning. Welcome to Google I/O. [CHEERING] AUDIENCE: I love you, Sundar! [LAUGHTER] SUNDAR PICHAI: I love you guys, too. [LAUGHTER] Can't believe it's one year already. It's a beautiful day.
We're being joined by over 7,000 people, and we are live streaming this, as always, to over 400 events in 85 countries. Last year was the 10th year since Google I/O started, and so we moved it closer to home at Shoreline, back where it all began. It seems to have gone well. I checked the Wikipedia entry from last year. There were some mentions of sunburn, so we have plenty of sunscreen all around. It's on us. Use it liberally. It's been a very busy year since last year, no different from my 13 years at Google. That's because we've been focused ever more on our core mission of organizing the world's information. And we're doing it for everyone. And we approach it by applying [? deep ?] computer science and technical insights to solve problems at scale.
That approach has served us very, very well. This is what allowed us to scale up seven of our most important products and platforms to over a billion monthly active users each. And it's not just the scale at which these products are working, users engage with them very heavily. YouTube, not just has over a billion users, but every single day, users watch over 1 billion hours of videos on YouTube. Google Maps. Every single day, users navigate over 1 billion kilometers with Google Maps. So the scale is inspiring to see, and there are other products approaching this scale. We launched Google Drive five years ago, and today, it is over 800 million monthly active users. And every single week, there are over 3 billion objects uploaded to Google Drive.
Two years ago at Google I/O, we launched Photos as a way to organize user's photos using machine learning. And today, we are over 500 million active users, and every single day, users upload 1.2 billion photos to Google. So the scale of these products are amazing, but they are all still working up their way to what's Android, which I'm excited as of this week, we crossed over 2 billion active devices of Android.
[APPLAUSE] As you can see, the robot is pretty happy, too, behind me, so it's a privilege to serve users of this scale. And this is all because of the growth of mobile and smartphones, but computing is evolving again. We spoke last year about this important shift in computing from a mobile first to a AI first approach. Mobile made us reimagine every product we were working on. We had to take into account that the user interaction model had fundamentally changed, with multi-touch, location, identity, payments, and so on. Similarly, in a AI first world, we are rethinking all our products and applying machine learning and AI to solve user problems. And we are doing this across every one of our products. So today, if you use Google Search, we rank differently using machine learning.
Or if you're using Google Maps, Street View automatically recognizes restaurant signs, street signs, using machine learning. Duo with video calling uses machine learning for low bandwidth situations. And Smart Reply and Allo last year had great reception. And so today, we are excited that we are rolling out Smart Reply to over 1 billion users of Gmail. It works really well. Here's a sample email. If you get an email like this, the machine learning systems learn to be conversational, and it can reply, I'm fine with Saturday, or whatever. So it's really nice to see.
Just like with every platform shift, how users interact with computing changes. Mobile brought multi-touch. We evolved beyond keyboard and mouse. Similarly, we now voice and vision as two new important modalities for computing. Humans are interacting with computing in more natural and immersive ways. Let's start with voice. We've been using voice as an input across many of our products. That's because computers are getting much better at understanding speech. We have had significant breakthroughs, but the pace, even since last year, has been pretty amazing to see. Our word error rate continues to improve, even in very noisy environments.
This is why if you speak to Google on your phone or Google Home, we can pick up your voice accurately, even in noisy environments. When we were shipping Google Home, we had originally planned to include eight microphones so that we could accurately locate the source of where the user was speaking from. But thanks to deep learning, we use a technique called neural beamforming. We were able to ship it with just two microphones and achieve the same quality. Deep learning is what allowed us about two weeks ago to announce support for multiple users in Google Home, so that we can recognize up to six people in your house and personalize the experience for each and every one. So voice is becoming an important modality in our products. The same thing is happening with vision. Similar to speech, we are seeing great improvements in computer vision. So when we look at a picture like this, we are able to understand the attributes behind the picture.
We realize it's your boy in a birthday party. There was cake and family involved, and your boy was happy. So we can understand all that better now. And our computer vision systems now, for the task of the image recognition, are even better than humans. So it's astounding progress and we're using it across our products. So if you used the Google Pixel, it has the best-in-class camera, and we do a lot of work with computer vision. You can take a low light picture like this, which is noisy, and we automatically make it much clearer for you.
Or coming very soon, if you take a picture of your daughter at a baseball game, and there is something obstructing it, we can do the hard work remove the obstruction– [APPLAUSE] –and– [APPLAUSE] –have the picture of what matters to you in front of you. We are clearly at an inflection point with vision, and so today, we are announcing a new initiative called Google Lens. [APPLAUSE] Google Lens is a set of vision-based computing capabilities that can understand what you're looking at and help you take action based on that information.
We'll ship it first in Google Assistant and Photos, and it'll come to other products. So how does it work? So for example, if you run into something and you want to know what it is, say, a flower, you can invoke Google Lens from your Assistant, point your phone at it, and we can tell you what flower it is. It's great for someone like me with allergies. [LAUGHTER] Or if you've ever been at a friend's place and you have crawled under a desk just to get the username and password from a Wi-Fi router, you can point your phone at it. [APPLAUSE] And we can automatically do the hard work for you. Or if you're walking in a street downtown and you see a set of restaurants across you, you can point your phone.
Because we know where you are and we have our Knowledge Graph and we know what you're looking at, we can give you the right information in a meaningful way. As you can see, we're beginning to understand images and videos. All of Google was built because we started understanding text and web pages. So the fact that computers can understand images and videos has profound implications for our core mission. When we started working on Search, we wanted to do it at scale. This is why we rethought our computational architecture. We designed our data centers from the ground up. And we put a lot of effort in them. Now that we are evolving for this machine learning and AI world, we are rethinking our computational architecture again. We are building what we think of as AI first data centers. This is why last year, we launched the tensor processing units. They are custom hardware for machine learning. They were about 15 to 30 times faster and 30 to 80 times more power efficient than CPUs and GPUs at that time. We use TPUs across all our products, every time you do a search, every time you speak to Google.
In fact, TPUs are what powered AlphaGo in its historic match against Lee Sedol. I see now machine learning as two components. Training, that is, how we build the neural net. Training is very computationally intensive, and inference is what we do at real time, so that when you show it a picture, we'd recognize whether it's a dog or a cat, and so on.
Last year's TPU were optimized for inference. Training is computationally very intensive. To give you a sense, each one of our machine translation models takes a training of over three billion words for a week on about 100 GPUs. So we've been working hard and I'm really excited to announce our next generation of TPUs, Cloud TPUs, which are optimized for both training and inference. What you see behind me is one Cloud TPU board. It has four chips in it, and each board is capable of 180 trillion floating point operations per second. [WHOOPING] And we've designed it for our data centers, so you can easily stack them. You can put 64 of these into one big supercomputer. We call these TPU pods, and each pod is capable of 11.5 petaflops. It is an important advance in technical infrastructure for the AI era. The reason we named it cloud TPU is because we're bringing it through the Google Cloud Platform. So cloud TPUs are coming to Google Compute Engine as of today. [APPLAUSE] We want Google Cloud to be the best cloud for machine learning, and so we want to provide our customers with a wide range of hardware, be it CPUs, GPUs, including the great GPUs Nvidia announced last week, and now Cloud TPUs.
So this lays the foundation for significant progress. So we are focused on driving the shift and applying AI to solving problems. At Google, we are bringing our AI efforts together under Google.ai. It's a collection of efforts and teams across the company focused on bringing the benefits of AI to everyone. Google.ai will focus on three areas, state-of-the-art research, tools, and infrastructure– like TensorFlow and Cloud TPUs– and applied AI.
So let me talk a little bit about these areas. Talking about research, we're excited about designing better machine learning models, but today it is really time consuming. It's a painstaking effort of a few engineers and scientists, mainly machine learning PhDs. We want it to be possible for hundreds of thousands of developers to use machine learning. So what better way to do this than getting neural nets to design better neural nets? We call this approach AutoML. It's learning to learn. So the way it works is we take a set of candidate neural nets. Think of these as little baby neural nets. And we actually use a neural net to iterate through them till we arrive at the best neural net.
We use a reinforcement learning approach. And it's– the results are promising. To do this is computationally hard, but Cloud TPUs put it in the realm of possibility. We are already approaching state of the art in standard tasks like, say, for our image recognition. So whenever I spend time with the team and think about neural nets building their own neural nets, it reminds me of one of my favorite movies, "Inception." And I tell them we must go deeper.
[LAUGHTER] So we are taking all these AI advances and applying them to newer, harder problems across a wide range of disciplines. One such area is health care. Last year, I spoke about our work on diabetic retinopathy. It's a preventable cause of blindness. This year, we published our paper in the "Journal of the American Medical Association," and [? verily ?] is working on bringing products to the medical community. Another such area is pathology. Pathology is a very complex area. If you take an area like breast cancer diagnosis, even amongst highly trained pathologists, agreement on some forms of breast cancer can be as low as 48%.
That's because each pathologist is reviewing the equivalent of 1,000 10-megapixel images for every case. This is a large data problem, but one which machine learning is uniquely equipped to solve. So we built neural nets to detect cancer spreading to adjacent lymph nodes. It's early days, but our neural nets show a much higher degree of accuracy, 89% compared to previous methods of 73%. There are important caveats we do have higher false positives, but already giving this in the hands of pathologists, they can improve diagnosis. In general, I think this is a great approach for machine learning, providing tools for people to do what they do better. And we're applying it across even basic sciences. Take biology. We are training neural nets to improve the accuracy of DNA sequencing.
[? Deep ?] [? Piriant ?] is a new tool from Google.ai that identifies genetic variants more accurately than state-of-the-art methods. Reducing errors is in important in applications. We can more accurately identify whether or not a patient has genetic disease and can help with better diagnosis and treatment. We're applying it to chemistry. We're using machine learning to predict the properties of molecules. Today, it takes an incredible amount of computing resources to hunt for new molecules, and we think we can [? accelerate ?] timelines by orders of magnitude.
This opens up possibilities in drug discovery or material sciences. I'm entirely confident one day, AI will invent new molecules that behave in predefined ways. Not everything we are doing is so profound. We are doing even simple and fun things, like a simple tool which can help people draw. We call this AutoDraw. Just like today when you type in Google, we give you suggestions, we can do the same when you're trying to draw, even I can draw with this thing.
So it may look like fun and games, but pushing computers to do things like this is what helps them be creative and actually gain knowledge. So we are very excited about progress even in these areas as well. So we are making impressive progress in applying machine learning, and we are applying it across all our products, but the most important product we are using this is for Google Search and Google Assistant. We are evolving Google Search to be more assistive for our users. This is why last year at Google I/O, we spoke about the Assistant, and since then, we've launched it on Google Pixel and Google Home. Scott and team are going to talk more about it, but before that, let's take a look at the many amazing ways people have been using the Google Assistant. [VIDEO PLAYBACK] – OK, Google. [MUSIC PLAYING] – Hey, Google? – Hey, Google. – OK, Google. – Hey, Google. [BLING] – Play some dance music. – Sure. [BLING] – This is "Fresh Air." My guest will be– – Kimmy Schmidt on Netflix. [BLING] – OK, Google. Count to 100.
– Sure. 1, 2, 3– – Play vacuum harmonica on my TV. [VACUUMING] [HARMONICA PLAYS] – –71, 72– – No! – –73– – Play the "Wonder Woman" trailer. – Hey, Google. Talk to Domino's. – Talk to Lonely Planet. – Talk to Quora. – Show me my photos from last weekend. [BLING] [SCREAMING] – Your car is parked at 22B. [BEEP BEEP] – Today in the news– [BLING] – Turn the living room lights on. – OK, turning on the lights. – I'm back, baby. – Hey, Google. Drop a beat. – Flip a coin. – Call Jill. – Set a timer. – Talk to Headspace. [TING] – And then just for a moment, I'd like you to let go of any focus at all. Just let your mind do whatever it wants to do. – Done. – Hey, Google. Good night. – Turning off all the things. See you tomorrow. [END PLAYBACK] [MUSIC PLAYING] [APPLAUSE] SCOTT HUFFMAN: Hey, everyone. Last year at I/O, we introduced the Google Assistant, a way for you to have a conversation with Google to get things done in your world.
Today, as Sundar mentioned, we're well on our way, with the Assistant available on over 100 million devices. But just as Google Search simplified the web and made it more useful for everyone, your Google Assistant simplifies all the technology in your life. You should be able to just express what you want throughout your day and the right things should happen. That's what the Google Assistant is all about. It's your own individual Google. So that video we saw really captures the momentum of this project. We've made such big strides and there's so much more to talk about today. The Assistant is becoming even more conversational, always available wherever you need it, and ready to help get even more things done.
First, we fundamentally believe that the Google Assistant should be, hands down, the easiest way to accomplish tasks, and that's through conversation. It comes so naturally to humans, and now Google is getting really good at conversation, too. Almost 70% of requests to the Assistant are expressed in natural language, not the typical keywords that people type in a search box. And many requests or follow-ups that continue the conversation. We're really starting to crack the hard computer science challenge of conversationality by combining our strengths in speech recognition, natural language understanding, and contextual meaning. Now recently, we made the Assistant even more conversational, so each member of the family gets relevant responses just for them by asking with their own voice. And we're continuing to make interacting with your Assistant more natural. For example, it doesn't always feel comfortable to speak out loud to your Assistant, so today, we're adding the ability to type to your Assistant on the phone.
Now, this is great when you're in a public place and you don't want to be overheard. The Assistant's also learning conversation beyond just words. With another person, it's really natural to talk about what you're looking at. Sundar spoke earlier about how AI and deep learning have led to tremendous strides in computer vision. Soon, with the smarts of Google Lens, your Assistant will be able to have a conversation about what you see. And this is really cool, and Ibrahim is here to help me show you a couple of examples of what we'll launch in the coming months. So, last time I traveled to Osaka, I came across a line of people waiting to try something that smelled amazing.
Now, I don't speak Japanese, so I couldn't read the sign out front, but Google Translate knows over 100 languages, and my Assistant will help with visual translation. I just tap the Google Lens icon, point the camera, and my Assistant can instantly translate the menu to English. And now, I can continue the conversation. IBRAHIM ULUKAYA: What does it look like? GOOGLE ASSISTANT: These pictures should match. SCOTT HUFFMAN: All right. It looks pretty yummy. Now notice, I never had to type the name of the dish.
My Assistant used visual context and answered my question conversationally. Let's look at another example. Some of the most tedious things I do on my phone stem from what I see– a business card I want to save, details from a receipt I need to track, and so on. With Google Lens, my Assistant will be able to help with those kinds of tasks, too. I love live music, and sometimes I see info for shows around town that look like fun.
Now, I can just tap the Google Lens icon and point the camera at the venue's marquee. My Assistant instantly recognizes what I'm looking at. Now, if I wanted to, I could tap to hear some of this band's songs, and my Assistant offers other helpful suggestions right in the viewfinder. There's one to buy tickets from Ticketmaster, and another to add the show to my calendar. With just a tap, my Assistant adds the concert details to my schedule. GOOGLE ASSISTANT: Saving event.
Saved Stone Foxes for May 17th at 9:00 PM. SCOTT HUFFMAN: Awesome. [APPLAUSE] My Assistant will help me keep track of the event, so I won't miss the show, and I didn't have to open a bunch of apps or type anything. Thanks Ibrahim. So that's how the Assistant is getting better at conversation– by understanding language and voices, with new input choices, and with the power of Google Lens. Second, the Assistant is becoming a more connected experience that's available everywhere you need help, from your living room to your morning jog, from your commute to errands around town, your Assistant should know how to use all of your connected devices for your benefit. Now, we're making good progress in bringing the Assistant to those 2 billion phones, and other devices powered by Android, like TVs, wearables, and car systems. And today, I'm excited to announce that the Google Assistant is now available on the iPhone. [APPLAUSE] Woo. So no matter what smartphone you use, you can now get help from the same smart assistant throughout the day at home, and on the go. The Assistant brings together all your favorite Google features on the iPhone.
Just ask to get package delivery details from Gmail, watch videos from your favorite YouTube creators, get answers from Google Search, and much more. You can even turn on the lights and heat up the house before you get home. Now, Android devices and iPhones are just part of the story. We think the Assistant should be available on all kinds of devices where people might want to ask for help. The new Google Assistant SDK allows any device manufacturer to easily build the Google Assistant into whatever they're building. Speakers, toys, drink-mixing robots, whatever crazy device all of you think up, now can incorporate the Google Assistant. And we're working with many of the world's best consumer brands and their suppliers, so keep an eye out for the badge that says, "Google Assistant built-in" when you do your holiday shopping this year. Now obviously, another aspect of being useful to people everywhere is support for many languages. I'm excited to announce that starting this summer, the Google Assistant will begin rolling out in French, German, Brazilian Portuguese, and Japanese on both Android phones and iPhones.
By the end of the year, we'll also support Italian, Spanish and Korean. So that's how the Assistant is becoming more conversational, and how it will be available in even more contexts. Finally, the Assistant needs to be able to get all kinds of useful things done for people. People sometimes ask if the Assistant is just a new way to search. Now of course, you can ask your Assistant to get all sorts of answers from Google Search, but beyond finding information, users are also asking the Assistant to do all sorts of things for them.
Now as you've already seen, the Assistant can tap into capabilities across many Google Apps and services, but Google's features are just part of the story. We also open the Assistant to third-party developers who are building some really useful integrations. I'll turn it over to Valerie to share more about how the developer platform is getting stronger. [MUSIC PLAYING] [APPLAUSE] VALERIE NYGAARD: Hi. OK, so with the actions on Google Platform, it's been awesome to see how developers like you have been engaging with the Google Assistant. Like honestly, you've built some really cool integrations. Like, I can ask Food Network about the recipe that's on TV right now. I can work out with Fitstar, ask CNBC about the news, or my husband and I can play name that tune with SongPop, which he is surprisingly good at.
Until now, these experiences have been available through the Assistant on Google Home. But today, we're also bringing them to Android phones and iPhones. It's over 100 million devices on Android alone. So now people can get to Google features and third-party services from anywhere, and they can even pick up where they left off across devices. So, not only are third-party integrations available in more places. They'll be able to do more. Starting today, actions on Google will be supporting transactions. It's a complete end-to-end solution for developers, including payments, identity, notifications, receipts, even account creation. The platform handles all the complexity. Let me show you how one will work. GOOGLE ASSISTANT: Hi, how can I help? VALERIE NYGAARD: I'd like delivery from Panera. PANERA: Hi, this is Panera. I'll need your delivery address.
Which one can I get from Google? GOOGLE ASSISTANT: We'll go with 1600 Amphitheater. PANERA: What can I get you started with? VALERIE NYGAARD: I'll have the strawberry poppy seed salad with steak instead of chicken. PANERA: Got it. How about one of these cool drinks? VALERIE NYGAARD: And here, I can just swipe through my options. See what looks good. Agave lemonade. PANERA: Great. Are you ready to check out? VALERIE NYGAARD: Yep. PANERA: OK, the total is $18.40. Are you ready to place the order? VALERIE NYGAARD: Yes. I'll just scan my fingerprint to pay with Google, and that's it. [APPLAUSE] PANERA: Thanks. You're all set. VALERIE NYGAARD: Yeah, super easy, like I was talking to someone at the store. So here I was a new Panera customer. I didn't have to install anything or create an account. You've also probably noticed I didn't have to enter my address or my credit card.
I just saved those earlier with Google, and Panera used built-in platform calls to request the information. Now, I was in control over what I shared every step of the way. So– AUDIENCE: Woo! VALERIE NYGAARD: [CHUCKLES] The developer platform's also getting much stronger for home automation integrations. Actions on Google can now support any smart home developer that wants to add conversational control. Today, over 70 smart home companies work with the Google Assistant, so now in my Google Home or from my phone, I can lock my front door with August locks, control a range of LG appliances, or check in on my son's room by putting the Nest cam on TV. All right, now that we're talking about making your home smarter, we also have a lot of news to share today about Google Home, our own smart speaker with the Google Assistant built in. Here to tell you more is Rishi Chandra.
[MUSIC PLAYING] [APPLAUSE] RISHI CHANDRA: Thanks, Valerie. You know, it's really hard to believe we launched Google Home a little over six months ago, and we've been really busy ever since. Since launch, we've added 50 new features, including some my favorites like support for Google Shopping, where I can use my voice to order items from Costco right to my front door. Or I can get step-by-step cooking instructions from over 5 million recipes. Or I can even play my favorite song just by using the lyrics. Now in April, we launched in the UK to some great reviews. And starting this summer, we're going to be launching in Canada, Australia, France, Germany, and Japan. [APPLAUSE] And with support for multiple users, we can unlock the full potential of Google Home to offer a truly personal experience. So now, you can schedule a meeting, set a reminder, or get your own daily briefing with My Day by using your own voice.
And get your commute, your calendar appointments, and your news sources. Now today, I'd like you share four new features we'll be rolling out over the coming months. So first, we're announcing support for proactive assistance coming to Google Home. Home is great at providing personally relevant information for you when you ask for it, but we think it'd be even more helpful if it can automatically notify you of those timely and important messages. And we do this by understanding the context of your daily life, and proactively looking for that really helpful information, and providing for you and a hands-free way. So for example, let's say I'm relaxing and [? playing game ?] with the kids. Well, I can see that the Google Home lights just turned on. Hey, Google, what's up? GOOGLE ASSISTANT: Hi, Rishi. Traffic's heavy right now, so you'll need to leave in 14 minutes to get to Shoreline Athletic Fields by 3:30 PM. RISHI CHANDRA: That's pretty nice. The Assistant saw the game coming up on my calendar, and got my attention because I had to leave earlier than normal.
So now, my daughter can make it to that soccer game right on time. Now, we're going to start simple, with really important messages like reminders, traffic delays, and flight status changes. And with multiple-user support, you have the ability to control the type of proactive notifications you want over time. All right, and second, another really common activity we do in the home today is communicate with others. And a phone call is still the easiest way to reach someone. So today, I'm excited to announce hands-free calling coming to Google Home. [CHEERING AND APPLAUSE] It's really simple to use. Just ask the Google Assistant to make a call, and we'll connect you. You can call any landline or mobile number in the US or Canada completely free. And it's all done in a hands-free way. For example, let's say I forgot to call my mom on Mother's Day. Well now, I can call her while I'm scrambling to get the kids ready for school in the morning. I just see and say, hey Google. Call mom. GOOGLE ASSISTANT: Sure, calling mom.
[RINGING] [RINGING] SPEAKER 1: So, you're finally calling. Mother's Day was three days ago. RISHI CHANDRA: Yeah, sorry about that. They made me rehearse for I/O on Mother's Day. Speaking of which, you're on stage right now. Say hi to everyone. SPEAKER 1: Oh, hi, everyone. AUDIENCE: Hi. RISHI CHANDRA: So, hopefully, this makes up for not calling, right? SPEAKER 1: No, it doesn't. You still need to visit and bring flowers.
RISHI CHANDRA: OK, I'm on it. Bye. SPEAKER 1: Bye. RISHI CHANDRA: It's that simple. We're just making a standard phone call through Google Home. So mom didn't need to learn anything new. She just needs to answer her phone. There's no additional setup, apps, or even phone required. And since the Assistant recognized my voice, we called my mom.
If my wife had asked, we would have called her mom. We can personalize calling just like everything else. And now, anyone home can call friends, family, even businesses. Maybe even a local florist to get some flowers for your mom. Now, by default, we're going to call out with a private number, but you also have the option to link your mobile number to the Google Assistant. And we'll use that number whenever we recognize your voice. So whoever you call [? must ?] know it's coming from you. Now, we're rolling out hands-free calling in the US to all existing Google Home devices over the next few months. It's the ultimate hands-free speakerphone. No setup required, call anyone, including personal contacts or businesses, and even dial out with your personal number when we detect your voice. We can't wait for you to try it out.
OK, third, let's talk a little about entertainment. We designed Google Home to be a great speaker, one that can put in any room in the house or wirelessly connect to other Chromecast built-in speaker systems. Well today, we're announcing that Spotify, in addition to their subscription service, will be adding their free music service to Google Home, so it's even easier to play your Spotify playlists. [APPLAUSE] We'll also be adding support for SoundCloud and Deezer to the largest global music services today. [APPLAUSE] And these music services will join many of the others already available through the Assistant. And finally, we'll be adding Bluetooth support to all existing Google Home devices. So you can play any audio from your iOS or Android device. AUDIENCE: Yes! [APPLAUSE] But Google Home can do much more than just audio. Last year, we launched the ability to use your voice to play YouTube, Netflix, and Google Photos right on your TV.
And today, we're announcing additional partners, including HBO NOW. [APPLAUSE] So just say you want to watch, and we'll play it for you all in a hands-free way. With Google Home, we want to make it really easy to play your favorite entertainment. OK, finally, I want to talk a little bit how we see the Assistant evolving to help you in a more visual way. Voice responses are great, but sometimes a picture is worth a thousand words. So today, we're announcing support for visual responses with Google Home. Now to do that, we need a screen. Well, fortunately, many of us already have a ton of screens in our home today, our phones, our tablets, even our TVs.
The Google Assistant should smartly take advantage of all these different devices to provide you the best response on the right device. For example, with Google Home, I can easily get location information. OK, Google. Where is my next event? GOOGLE ASSISTANT: Your Pokemon GO hike is at Rancho San Antonio Reserve. RISHI CHANDRA: It's for my kids. GOOGLE ASSISTANT: It's at 11:00 AM today. RISHI CHANDRA: It's for my kids. Relax. [LAUGHTER] But if I want to view the directions, the best place to do it is on my phone. Well soon, you could just say, OK, Google. Let's go. GOOGLE ASSISTANT: All right, I'm sending the best route to your phone. RISHI CHANDRA: And will automatically your phone– and notify your phone, whether it's Android or iOS, and take you straight to Google Maps.
So you can glance at directions, interact with the map, or just start navigation. It's really simple. Now TVs are another natural place to get help from the Google Assistant, and we've a great place to start with over 50 million Chromecast and Chromecast built-in devices. So today, we're announcing that we'll be updating Chromecast to show visual responses on your TV when you ask for help from Google Home. For example, I can now say, OK, Google. Show my calendar for Saturday. GOOGLE ASSISTANT: Showing it on your TV. RISHI CHANDRA: It'll show up right on TV screen. I'll immediately get results from the Assistant. [APPLAUSE] And since the Assistant detected my voice, we're showing my calendar. Others would see their calendar by using their voice. We can personalize the experience, even on the TV. They can continue to follow-up the conversation. Looks like I have a biking trip to Santa Cruz.
What's the weather in Santa Cruz this weekend? GOOGLE ASSISTANT: This weekend in Santa Cruz, it will be clear and sunny most of the time. RISHI CHANDRA: So it's really easy. It's all hands-free. Your Assistant can provide a visual response to a TV to a lot of different types of questions. We talked about how easy it is to play what you want to watch on the TV screen, but what about those times you don't know what to watch? Well, soon, you could just ask, hey, Google. What's on YouTube? GOOGLE ASSISTANT: Here you go.
RISHI CHANDRA: And it'll show me my personalized results right on the TV screen. If I don't like any of the options, I can continue the conversation with my voice. Show my Watch Later list. GOOGLE ASSISTANT: All right. RISHI CHANDRA: Play "Send My Love." GOOGLE ASSISTANT: Playing "Send My Love" from YouTube. [MUSIC – "SEND MY LOVE"] RISHI CHANDRA: It's really simple. Again, no remotes or phone required. In a short conversation, I found something really interesting to watch using Google Home. I can even do it with other things. OK, Google. What's on my DVR? GOOGLE ASSISTANT: Here you go. RISHI CHANDRA: Here we're showing how it works with YouTube TV, a new live TV streaming service that gives you live sports and shows from popular TV networks. And YouTube TV includes a cloud DVR, so I can easily play my saved episodes. Everything can be done in a hands-free way all from the comfort of my couch. And over time, we're going to bring all those developer actions that Valerie had already talked about right to the TV screen.
So we'll do even more over time with Google Home. And that's our update for Google Home. Proactive assistance will bring important information to you at the right time, simple and easy hands-free calling, more entertainment options, and evolving the Assistant to provide visual responses in the home. Next up is Anil, who's going to talk about Google Photos. [APPLAUSE] [MUSIC PLAYING] ANIL SABHARWAL: Two years ago, we launched Google Photos with an audacious goal– to be the home for all of your photos, automatically organized and brought to life so that you could easily share and save what matters.
In doing so, we took a fundamentally different approach. We built a product from the ground up with AI at its core. And that's enabled us to do things in ways that only Google can. Like when you're looking for that one photo you can't find, Google Photos organizes your library by people, places, and things. Simply type, "Anil pineapple Hawaii," and instantly find this gem. [LAUGHTER] Or when you come home from vacation, overwhelmed by the hundreds of photos you took, Google Photos will give you an album curated with only the best shots, removing duplicates and blurry images. This is the secret ingredient behind Google Photos, and the momentum we've seen in these two short years is remarkable. As Sundar mentioned, we now have more than half a billion monthly active users, uploading more than 1.2 billion photos and videos per day. And today, I'm excited to show you three new features we're launching to make it even easier to send and receive the meaningful moments in your life.
Now, at first glance, it might seem like photo sharing is a solved problem. After all, there's no shortage of apps out there that are great at keeping you and your friends and family connected, but we think there's still a big and different problem that needs to be addressed. Let me show you what I mean. [VIDEO PLAYBACK] – If there's one thing you know, it's that you're a great photographer. If there's a second thing you know, it's that you're kind of a terrible person. – What? – Yeah, you heard me. The only photo of the birthday girl in focus? Never sent it. The best picture of the entire wedding? Kept it to yourself. This masterpiece of your best friend? We were going to send it, but then you were like, oh, remember that sandwich? I love that sandwich.
If only something could say, hey, Eric looks great in these. You want to send them to him? And you can be like, great idea. Well, it can. Wait, it can? Yup. With Google Photos. [END PLAYBACK] [APPLAUSE] ANIL SABHARWAL: So today, to make us all a little less terrible people, we're announcing Suggested Sharing, because we've all been there, right? Like when you're taking that group photo and you insist that it be taken with your camera, because you know if it's not on your camera, you are never seeing that photo ever again. [LAUGHTER] Now thanks to the machine learning in Google Photos, we'll not only remind you so you don't forget to share, we'll even suggest the photos and people you should share with. In one tap, you're done. Let's have a look at Suggested Sharing in action. I'm once again joined onstage by my friend, and Google Photos product lead, David Leib. [APPLAUSE] All right, so here are a bunch of photos Dave took while bowling with the team last weekend.
He was too busy enjoying the moment, so he never got around to sharing them. But this time, Google Photos sent him a reminder via notification, and also by badging the new Sharing tab. The Sharing tab is where you're going be able to find all of your Google Photos sharing activity, and at the top, your personal suggestions based on your sharing habits and what's most important to you. Here is the Sharing Suggestion that Dave got from his day bowling. Google Photos recognized this was a meaningful moment, it selected the right shots, and it figured out who he should send it to based on who was in the photos. In this case, it's Janvi, Jason, and a few others who were also at the event. Dave can now review the photos selected, as well as update the recipients.
Or if he's happy with it, he can just tap Send. And that's it. Google Photos will even send an SMS or an email to anyone who doesn't have the app. And that way, everyone can view and save the full resolution photos, even if they don't have Google Photos accounts. And because Google photo sharing works on any device, including iOS, let's have a look at what Janvi sees on her iPhone. She receives a notification, and tapping on it lets her quickly jump right into the album. And look at all the photos that Dave has shared with her. But notice here at the bottom, she's asked to contribute the photos she took from the event, with Google Photos automatically identifying and suggesting the right ones.
Janvi can review the suggestions and then simply tap Add. Now all of these photos are finally pulled together in one place, and Dave gets some photos he's actually in. [LAUGHTER] Which is great, because a home for all your photos really should include photos of you. Now, though Suggested Sharing takes the work out of sharing, sometimes there's a special person in your life whom you share just about everything with. Your partner, your best friend, your sibling. Wouldn't it be great if Google Photos automatically shared photos with that person? For example, I would love it if every photo I ever took of my kids was automatically shared with my wife. And that's why today, we're also announcing Shared Libraries. [APPLAUSE] Let me show you how it works. So here, we're now looking at my Google Photos account. >From the menu, I now have the option to go ahead and share my library, which I'm going to go ahead and do with my wife, Jess.
Importantly, I have complete control over which photos I automatically share. I can share them all, or I can share a subset, like only photos of the kids, or only photos from a certain date forward, like when we first met. In this case, I'm going to go ahead and share all. [LAUGHTER] [LAUGHS] We did not meet today. [LAUGHTER] And that's all there is to it. I've now gone ahead and shared my library with my wife, Jess. So, let's switch to her phone to see what the experience looks like from her end. She receives a notification, and after accepting, she can now go to see all the photos that I've shared with her, which she can access really easily from the menu. If she see something she likes, she can go ahead and select those photos and simply save them to her library.
We'll even notify her periodically as I take new photos. Now, this is great, but what if Jess doesn't want to have to keep coming back to this view and checking if I shared new photos with her? She just wants every photo I take of her or the kids to automatically be saved to her library, just as if she took the photos herself. With Shared Libraries, she can do just that, choosing to autosave photos of specific people. Now, any time I take photos of her or the kids, without either of us having to do anything, they'll automatically appear in the main view of her app. Let me show you. Now, I couldn't justify pulling the kids out of school today just to have their photo taken, but I do have the next best thing.
[APPLAUSE] Let me introduce you to [? Eva ?] and [? Lilly. ?] All righty here. So I'm going to go ahead, take a photo with the girls. Smile, kids! [LAUGHTER] Wow, fantastic. And since this is too good of an opportunity, I'm going to have to take one with all of you here, too, all right? [CHEERING] Here we go. Woo! Brilliant. All right. OK, so thank you, girls. Much appreciated. Back to school we go. [LAUGHTER] All right. So, using nothing more than the standard camera app on my phone, I've gone ahead and taken one photo with my kids and one photo with all of you here in the audience. Google Photos is going to back these two photos up. It's going to share them with Jess, and then it's going to recognize the photo that has my kids in them and automatically save just that one to her library, like you can see right here. [APPLAUSE] Now finally, Jess and I can stop worrying about whose phone we're using to take the photos. All the photos of our family are in my Google Photos app, and they automatically appear in hers too.
And best of all, these family photos are part of both of our search results, and they're included in the great collages, movies, and other fun creations that Google Photos makes for us. But notice how only the photos with the kids showed up in Jess's main view. But because I shared my entire library with her, I can simply go to the menu, and Jess can now see all of the photos, including the one with all of you. [APPLAUSE] And that's how easy sharing can be in Google Photos. Spend less time worrying about sharing your memories, and more time actually enjoying them. Suggested Sharing and Shared Libraries will be rolling out on Android, iOS, and web in the coming weeks.
Finally, we know sharing doesn't always happen through apps and screens. There's still something pretty special about looking at and even gathering around an actual printed photo. But printing photos and albums today is hard. You have to hunt across devices and accounts to find the right photos, select the best among the duplicates and blurry images, upload them to a printing service, and then arrange them across dozens of pages. It can take hours of sitting in front of a computer just to do one thing. Thankfully, our machine learning and Google Photos already does most of this work for you, and today, we're bringing it all together with the launch of Photo Books. [APPLAUSE] They're beautiful, high quality with a clean and modern design, but the best part is that they're incredibly easy to make, even on your phone. What used to take hours now only takes minutes. I recently made a book for Jess on Mother's Day. And let me show you just how easy and fast that was. First, thanks to unlimited storage, all my life's moments are already here in Google Photos. No need to upload them to another website or app.
So I'll select a bunch of photos here. And the good news is I don't have to figure out which are the right photos and which are the good ones because this is where Google Photos really shines. I'm just going to go ahead and hit plus. Select Photo book. I'm going to pick a hardcover book. We offer both a softcover and a hardcover. And notice what happens. Google Photos is going to pick the best photos for me automatically, automatically suggesting photo– 40, in this case. [APPLAUSE] How awesome is that? And it's even going to go ahead and lay them all out for me. All that's left for me to do is make a couple of tweaks, check out, and in a few days, I'll end up with one of these beautiful printed photo books. [APPLAUSE] And soon, we'll make it even easier to get started, applying machine learning to create personalized photo books you'll love.
So when you go to Photo Books from the menu, you'll see pre-made books tailored just for you. Your trip to the Grand Canyon, time with your family during the holidays, or your pet, or even your kids artwork, all easily customizable. We'll even notify you when there are new Photo Books suggestions. AUDIENCE: [INAUDIBLE] ANIL SABHARWAL: Photo Books are available today in the US on photos.google.com, and they'll be rolling out on Android and iOS next week, and will be expanding to more countries soon.
[APPLAUSE] I am really excited about this launch, and I want all of you to be the first to try it out. And that's why everyone here at I/O will be receiving a free hardcover photo book. [APPLAUSE] It's a great example of machine learning at work. AUDIENCE: [? $10? ?] Take that photo [INAUDIBLE] ANIL SABHARWAL: So those are the three big updates related to sharing in Google Photos. Suggested Sharing, Shared Libraries, and Photo Books. Three new features built from the ground up with AI at their core. I can't wait for all of you to try them out real soon. Now before I go, I want to touch on what Sundar mentioned earlier, which is the way we're taking photos is changing.
Instead of the occasional photo with friends and family, we now take 30 identical photos of a sunset. We're also taking different types of photos, not just photos to capture personal memory, but as a way to get things done– whiteboards we want to remember, receipts we need to file, books we'd like to read. And that's where Google Lens and its vision-based computing capabilities comes in. It can understand what's in an image and help you get things done. Scott showed how Google Lens and the Assistant can identify what you're looking at and help you on the fly. But what about after you've taken the photo? There are lots of photos you want to keep, and then look back on later to learn more and take action.
And for that, we're bringing Google Lens right into Google Photos. Let me show you. So let's say you took a trip to Chicago. There's some beautiful architecture there. And during your boat tour down the Chicago River, you took lots of photos, but it's hard to remember which building is which later on. Now, by activating Lens, you can identify some of the cool buildings in your photos, like the second tallest skyscraper in the US, Willis Tower. You can even pull up directions and get the hours for the viewing deck. And later, while visiting the Art Institute, you might take photos of a few paintings you really love. In one tap, you can learn more about the painting and the artist. And the screenshot that your friend sent you of that bike rental place? Just activate Lens, and you can tap the phone number and make the call right from the photo.
[APPLAUSE] Lens will be rolling out in Google Photos later this year, and we'll be continually improving the experience so it recognizes even more objects and lets you do even more with them. And those are the updates for Google Photos. [CHEERING AND APPLAUSE] Now, let's see what's next from YouTube. [MUSIC PLAYING] SUSAN WOJCICKI: All right. Good morning, everyone. I am thrilled to be here at my first ever I/O on behalf of YouTube. [APPLAUSE] Thank you. So that opening video that we all just saw, that's a perfect glimpse into what makes YouTube so special– the incredible diversity of content. A billion people around the globe come to YouTube every month to watch videos from new and unique voices. And we're hard at work to make sure that we can reach the next billion viewers, which you'll hear about in a later I/O session today.
We want to give everyone the opportunity to watch the content on YouTube. So, YouTube is different from traditional media in a number of ways. First of all, YouTube is open. Anyone in the world can upload a video that everyone can watch. You can be a vlogger broadcasting from your bedroom, a gamer live streaming from your console, or a citizen journalist documenting events live from your phone on the front lines. And what we've seen is that openness leads to important conversations that help shape society, from advancing LGBTQ rights to highlighting the plight of refugees, to encouraging body positivity. And we've seen in our numbers that users really want to engage with this type of diverse content.
We are proud that last year we passed a billion hours a day being watched on YouTube, and our viewership is not slowing down. The second way that YouTube is different from traditional media is that it's not a one-way broadcast. It's a two way conversation. Viewers interact directly with their favorite creators via comments, mobile live streaming, fan polls, animated GIFs, and VR. And these features enable viewers to come together, and to build communities around their favorite content. So one of my favorite stories of a YouTube community is the e-NABLE network. A few years ago, an engineering professor named Jon Schull saw a YouTube video about a carpenter who had lost two of his fingers. The carpenter worked with a colleague for over a year to build an affordable 3D-printed prosthesis that would enable him to go back to work.
They then applied this technology for a young boy who was born without any fingers. So inspired by this video, the professor posted a single comment on the video asking for volunteers with 3D printers to help print affordable prostheses. The network has since grown into a community of over 6,000 people who have designed, printed, and distributed these prosthetics to children in over 50 countries. [APPLAUSE] So today, thousands of children have regained the ability to walk, touch, play, and all because of the one video– one comment– and that incredible YouTube community that formed to help. And that's just one example of the many passionate communities that are coming together on YouTube around video. So, the third feature of this new medium is that video works on-demand on any screen. Over 60% of our watchtime now comes from mobile devices. But actually our fastest growing screen isn't the one in your pocket. It's the one in your living room. Our watchtime in our living room is growing at over 90% a year. So, let's now welcome Sarah Ali, Head of Living Room Products, to the stage to talk about the latest features in the living room.
[MUSIC PLAYING] [APPLAUSE] SARAH ALI: Thank you, Susan. So earlier today, you heard from Rishi about how people are watching YouTube on the TV via the Assistant. But another way people are enjoying video is through the YouTube app, which is available over half a billion smart TVs, game consoles, and streaming devices. And that number continues to grow around the world. So, when I think about why YouTube is so compelling in the living room, it isn't just about the size of the screen. It's about giving you an experience that TV just can't match. First, YouTube offers you the largest library of on-demand content. Second, our recommendations build channels and lineups based on your personal interests, and what you enjoy watching. And third, it's a two-way interactive experience with features like voice control.
And today, I'm super excited to announce that we're taking the interactive experience a step further by introducing 360 video in the YouTube app on the big screen. And you know that you can already watch 360 videos on your phone or in your Daydream headset. But soon, you'll be able to feel like you're in the middle of the action, right from your couch, and on the biggest screen you own. Now, one of my personal interests outside of work is to travel. And one place I'd love to visit is Alaska to check out the Northern Lights. So, let's do a voice search. Aurora Borealis 360. Great. Let's choose that first video. And now, using my TV remote, I'm able to pan around this video, checking out this awesome view from every single angle. Traveling is great, especially when I don't have to get on a flight, but 360 is now a brand-new way to attend concerts. I didn't make it to Coachella, but here I can experience it like I was on stage.
And to enhance the experience even further, we are also introducing live 360 in the living room. Soon, you'll be able to witness moments and events as they unfold in a new, truly immersive way. So whether you have a Sony Android TV, or an Xbox One console, soon, you'll be able to explore 360 videos right from the comfort of your couch and along with your friends and family. And now, to help show you another way we're enabling interactivity, please join me in welcoming Barbara McDonald, who's the lead of something we call Super Chat. [MUSIC PLAYING] [APPLAUSE] BARBARA MACDONALD: Good morning I/O, and to everybody on the live stream.
As Susan mentioned, what makes YouTube special is the relationships that creators are able to foster with their fans. And one of the best ways to connect with your fans is to bring them live, behind the scenes of your videos, offering up can't-miss content. In the past year, the number of creators live streaming on YouTube has grown by 4x. This growth is awesome, and we want to do even more to deepen the connection between creators and their fans during live streams. That's why earlier this year, we rolled out a new feature called Super Chat. When a creator is live streaming, fans can purchase Super Chats which are highlighted, fun, chat messages. Not only do fans love the recognition, but creators earn extra money from it. In the past three months since launch, we've been amazed by the different ways creators are using Super Chat. Even April, our favorite pregnant giraffe, who unfortunately could not be here with us today, has raised tens of thousands of dollars for her home, the Animal Adventure Park.
But, OK. [CLAPPING] OK, we can clap for that. [APPLAUSE] [LAUGHS] But enough talking from me. We are going to do a live stream right here, right now, to show all of you how Super Chat works. And to help me, I am very excited to introduce top YouTube creators with 9 million subscribers and over 1 billion lifetime channel views. On the grass back there, The Slow Mo Guys! [CHEERING AND APPLAUSE] GAVIN FREE: Hello, everyone. DANIEL GRUCHY: Wow, hey. Happy to be here. How's it going? BARBARA MACDONALD: It's great to have you. So let's pull up their live stream. And just look. Chat is flying. Now, I love The Slow Mo Guys, and I want to make sure that they see my message, so I'm going to Super Chat them. Pulled up the stream. And right from within live chat, I am able to enter my message, select my amount, make the purchase, and send.
Boom. See how much that message stands out? And it gets to the top. It's cool, right? DANIEL GRUCHY: Yeah, thanks, Barbara. It's actually lovely at the minute. Although, I feel like there's a high chance of showers. GAVIN FREE: Very local showers, like, specifically to this stage. DANIEL GRUCHY: Very sudden. Yeah. BARBARA MACDONALD: Ooh, I wonder. I wonder. Well, because we know developers are incredibly creative, we wanted to see what you can do to make Super Chat even more interactive. So we've launched an API for it.
And today, we're taking it to the next level with a new developer integration that triggers actions in the real world. This means that when a fan sends a Super Chat to a creator, things can happen in real life, such as turning the lights on or off in the creator's studio, flying a drone around, or pushing buttons on their toys and gadgets. The Slow Mo Guys are going to create their next slow motion video using Super Chat's API. We have now rigged things up so that when I send my next Super Chat, it will automatically trigger the lights and a big horn in this amphitheater, OK? And that is going to signal our friends back there on the lawn to unleash a truckload of water balloons at The Slow Mo Guys. GAVIN FREE: I'm scared. [CHEERING] DANIEL GRUCHY: Yeah. BARBARA MACDONALD: Yeah. [LAUGHS] DANIEL GRUCHY: That's right. For every dollar, we're going to take another balloon. So, more money means more balloons. Although, I did hear a guy over here go, oh, we're going to really nail these guys.
All right, that's going to be at least $4 right there. So, yeah. Each dollar donated goes to the causes Susan mentioned earlier, the e-NABLE network. BARBARA MACDONALD: OK, so, how much do you think we can send? I can start at $1 and go anywhere upwards from there. So, it's for charity. How do we think– $100. How's that sound? AUDIENCE: More. BARBARA MACDONALD: OK, higher, higher. $200? $200? GAVIN FREE: How about $500 for 500 balloons? BARBARA MACDONALD: $500? I can do that. I can do that. OK. So I'm going to send my Super Chat and hit Send. $500. Boom. [HORN BLOWS] DANIEL GRUCHY: Oh! Balloons, oh [INAUDIBLE] god! Agh! BARBARA MACDONALD: [LAUGHS] DANIEL GRUCHY: Ugh. Yep. All right. All right. BARBARA MACDONALD: Keep going. Keep going. DANIEL GRUCHY: Oh! BARBARA MACDONALD: It's 500. DANIEL GRUCHY: It's finished. It's finished. GAVIN FREE: It never ends, ah! DANIEL GRUCHY: Ah! [INAUDIBLE] BARBARA MACDONALD: That was amazing.
Thank you, everybody, for your help. So this obviously just scratches the surface of what is possible using Super Chat's open APIs. And we are super excited to see what all of you will do with it next. So Susan, how about you come back out here, and let's check out the video we've all made. [VIDEO PLAYBACK] [MUSIC PLAYING] [APPLAUSE] BARBARA MACDONALD: [LAUGHS] AUDIENCE: [? Yeah, guys! ?] BARBARA MACDONALD: Wow.
[APPLAUSE] Thank you, Slow Mo Guys. Thank you, Barbara. I'm really happy to announce that YouTube is going to match The Slow Mo Guys' Super Chat earnings from today 100x to make sure that we're supplying prosthetics to children in need around the world. [APPLAUSE] So that 360 living room demo and the Super Chat demo– those are just two examples of how we are working to connect people around the globe together with video.
Now, I hope that what you've seen today is that the future of media is a future of openness and diversity. A future filled with conversations, and community. And a future that works across all screens. Together with creators, viewers, and partners, we are building the platform of that future. Thank you, I/O, and please– [APPLAUSE] Please welcome Dave Burke, joining us to talk about Android. [CHEERING AND APPLAUSE] [VIDEO PLAYBACK] [MUSIC – JACKIE WILSON, "HIGHER AND HIGHER"] [BUZZING] [CHEERING] [SATELLITE BEEPS] – Yay! Woohoo! [FIREWORKS LAUNCHING] – Yay! Woohoo! [FIREWORKS LAUNCHING] [END PLAYBACK] [CHEERING AND APPLAUSE] DAVE BURKE: Hi, everybody. It's great to be here at Google I/O 2017. As you can see, we found some new ways to hardware accelerate Android.
This time, with jet packs. But seriously, 2 billion active devices is incredible. And that's just smartphones and tablets. We're also seeing new momentum in areas such as TVs, and cars, and watches, and laptops, and beyond. So let me take a moment and give you a quick update on how Android is doing in those areas. Android Wear 2.0 launched earlier this year with a new update for Android and iPhone users. And with you partners like Emporio Armani, Movado, and New Balance, we now enable 24 of the world's top watch brands. Android Auto. We've seen a 10x user growth since last year It's supported by more than 300 car models and the Android Auto mobile app. And just this week, Audi and Volvo announced that their next generation nav systems will be powered by Android for a more seamless, connected car experience. Android TV. We partnered with over 100 cable operators and hardware manufacturers around the world. And we're now seeing 1 million device activations every two months.
And there are more than 3,000 Android TV apps in the Play Store. This year, we're releasing a brand-new launcher interface, and bringing the Google Assistant to Android TV. Android Things previewed late last year, and already there are thousands of developers in over 60 countries using it to build connected devices with easy access to the Google Assistant, TensorFlow, and more. The full launch is coming later this year. Chromebooks comprise almost 60% of K-12 laptops sold in the US, and the momentum is growing globally. And now, with the added ability to run Android apps, you get to target laptops, too. Now, of course, platforms are only as good as the apps they run. The Google Play ecosystem is more vibrant than ever. Android users installed a staggering 82 billion apps and games in the past year. That's 11 apps for every person on the planet. All right, so let's come back to smartphones. And the real reason I'm here is to talk about Android O. Two months ago, we launched our very first developer preview. So you could kick the tires on some of the new APIs.
And of course, it's very much a work in progress, but you can expect the release later this summer. Today, we want to walk you through two themes in O that we're excited about. The first is something called Fluid Experiences. It's pretty incredible what you can do on a mobile phone today, and how much we rely on them as computers in our pockets. But there are still certain things are tough to do on a small screen, so we're doing a couple of features in O that we think will help with this, which I'll cover in just a moment. The second theme is something we call Vitals. And the concept here is to keep vital system behavior in a healthy state so we can maximize the user's battery, performance, and reliability. So let's jump straight in and walk through four new fluid experiences, with live demos, done wirelessly.
What could possibly go wrong? [LAUGHTER] All right. These days, we do a lot of [? wants ?] on our phones, whether it's paying for groceries while reading a text message you just received, or looking up guitar chords while listening to a new song. But conventional multi-window techniques don't translate well to mobile. They're just too fiddly to set up when you're on the go. We think Picture-in-Picture is the answer for many cases. So let's take a look.
My kids recently asked me to build a lemonade stand. So I opened up YouTube, and I started researching DIY videos. And I found this one. Now, at the same time, I want to be able to jot down the materials I need to build for this lemonade stand. So to multitask, all I do is press the Home button, and boom, I get Picture-in-Picture.
You can think of it as a kind of automatic multi-window. I can move it out of the way, I can launch Keep, I can add some more materials. So I know I need to get some wood glue, like so. Then when I'm done, I just simply swipe it away like that. It's brilliant. Picture-in-Picture lets you do more with your phone. It works great when video calling with Duo.
For example, maybe I need to check my calendar while planning a barbecue with friends. And there are lots of other great use cases. For example, Picture-in-Picture for Maps navigation, or watching Netflix in the background, and a lot more. And we're also excited to see what you come up with for this feature. We're also making notification interactions more fluid for users. >From the beginning, Android has really blazed a trail when it comes to its advanced notification system. In O, we're extending the reach of notifications with something we call Notification Dots. It's a new way for app developers to indicate that there's activity in their app, and to drive engagement. So take a look. You'll notice that the Instagram app icon has a dot in it. And this is it indicating that there's a notification associated with the app. So if I pull down the shade, sure enough, you can see there's a notification. In this case, someone's commented on a photo I'm tagged in. What's really cool is I can long press the app icon, and we now show the notification in place.
One of the things I really like about the Notification Dot mechanism is that it works with zero effort from the app developer. We even extract the color of the dot from your icon. Oh, and you get your erase the icon by simply swiping the notification like that. So you're always in control. Another great feature in O that helps make your experience more fluid is Autofill. Now, if you use Chrome, you're probably already familiar with Autofill for quickly filling out a username and password, or credit card information with a single tap. With O, we've extended Autofill to apps. Let's say I'm setting up a new phone for the first time, and I open Twitter.
And I want to log in. Now, because I use twitter.com all the time on Chrome, the system will automatically suggest my username. I can simply tap it. I get my password. And then, boom. I'm logged in. It's pretty awesome. [APPLAUSE] Autofill takes the pain out of setting up a new phone or tablet. Once the user opts in, Autofill will work for most applications. We also provide APIs for developers to customize Autofill for their experience. I want to show you one more demo of how we're making Android more fluid by improving copy and paste. The feature is called Smart Text selection.
So let's take a look. In Android, you typically long press or double tap a word to select it. For example, I can open Gmail. I can start composing. If I double tap the word "bite," it gets selected like so. Now, we know from user studies that phone numbers are the most copy-and-pasted items. The second most common are named entities like businesses, and people, and places. In O, we're applying on-device machine learning– in this case, a [? feed ?] [? for a ?] neural network– to recognize these more complicated entities. So watch this. I can double tap anywhere on the phrase, "Old Coffee House," and all of it is selected for me. No more fiddling around with text selection handles. [APPLAUSE] It even works for addresses. So if I double tap on the address, all of it is selected. And what's more– [APPLAUSE] There is more. What's more is the machine learning model classifies this as an address and automatically suggests Maps. So I can get directions to it with a single click. And of course, it works as you'd expect for phone numbers. You get the phone dialer suggested.
And for email addresses, you get Gmail suggested. All of this neural networking processing happens on-device in real time, and without any data leaving the device. It's pretty awesome. Now, on-device machine learning helps to make your phone smarter. And we want to help you build experiences like what you just saw. So we're doing two things to help. First, I'm excited to announce that we're creating a specialized version of TensorFlow, Google's open source machine learning library, which we call TensorFlow Lite. It's a library for apps designed to be fast and small, yet still enabling state-of-the-art techniques like [? compnets ?] and LSTMs. Second, we're introducing a new framework at Android to hardware accelerate neural computation. TensorFlow Lite will leverage a new neural network API to tap into silicon-specific accelerators. And over time, we expect to see DSPs specifically designed for neural network inference and training.
We think these new capabilities will help power our next generation of on-device speech processing, visual search, augmented reality, and more. TensorFlow Lite will soon be part of that open source TensorFlow project, and the neural network API will be made available later in an update to O this year. OK, so that's a quick tour of some of the fluid experiences in O.
Let's switch gears and talk about Vitals. So to tell you more, I want to hand it over to Steph, who's been instrumental in driving this project. Thank you. [MUSIC PLAYING] STEPHANIE SAAD CUTHBERTSON: Hi, everyone. OK, so all the features Dave talked about are cool. But we think your phones' foundations are even more important– battery life, security, startup time, and stability. After all, if your battery dies at 4:00 PM, none of the other features that Dave talked about really matter. So in O, we're investing in what we call Vitals, keeping your phone secure and in a healthy state to maximize power and performance. We've invested in three foundational building blocks– security enhancements, OS optimizations, and tools to help developers build great apps. First, security. Android was built with security in mind from day one with application sandboxing. As Android has matured, we've developed vast mobile security services. Now, we use machine learning to continuously comb apps uploaded to Play, flagging potentially harmful apps.
Then, we scan over 50 billion apps every day, scanning every installed app on every connected device. And when we find a potentially harmful app, we disable it or remove it. And we found most Android users don't know these services come built-in with Android devices with Play. So for greater peace of mind, we're making them more visible and accessible, and doubling down on our commitment to security, with the introduction of Google Play Protect. [APPLAUSE] So here, you can see Play Protect has recently scanned all your apps. No problems found. That's Google Play Protect.
It's available out of the box on every Android device with Google Play. Second, OS optimizations. The single biggest visible change in O is boot time. On Pixel, for example, you'll find, in most cases, your boot time is now twice as fast. And we've made all apps faster by default. We did this through extensive changes to our runtime. Now, this is really cool stuff, like concurrent compacting garbage collection and code locality. But all you really need to know is that your apps will run faster and smoother. Take Google Sheets– aggregate performance over a bunch of common actions is now over two times as fast. And that's all from the OS. There are no changes to the app. But we found apps could still have a huge impact on performance. Some apps were running in the background, and they were consuming tons of system resources, especially draining battery.
So in O, we're adding Wise Limits to background location and background execution. These boundaries put sensible limits on usage. They're protecting battery life and freeing up memory. Now, our third theme is helping developers build great apps. And here, I want to speak directly to all the developers in the audience. Wouldn't it be cool if Android's engineering team could show you what causes performance issues? Today, we've launched Play Console Dashboards that analyze every app and pinpoint six top issues that cause battery drain, crashes, and slow UI. For each issue the app has, we show how many users are affected and provide guidance on the best way to fix. Now, imagine if developers could also have a powerful profiler to visualize what's happening inside the app. In Android Studio, we've also launched new unified profiling tools for network, memory, and CPU. So, developers can now see everything on a unified timeline, and then dive into each profiler. For example, on CPU, you can see every thread.
You can look at the call stack, and the time every call is taking. You can visualize where the CPU is going. And you can jump to the exact line of code. OK, so that's Android Vitals. [APPLAUSE] How we're investing in your phone's foundational security and performance. Later today, you'll see Android's developer story from end to end. Our hard work to help developers build great apps at every stage– writing code, tuning, launching, and growing.
But there is one more thing. One thing we think would be an incredible complement to the story. And it is one thing our team has never done for developers. We have never added a new programming language to Android. And today, we're making Kotlin an officially supported language in Android. [APPLAUSE] So, Kotlin– Kotlin is one our developer community has already asked for. It makes developers so much more productive. It is fully Android runtime compatible. It is totally interoperable with your existing code. It has fabulous IDE support. And it's mature and production ready from day one. We are also announcing our plans to partner with JetBrains, creating a foundation for Kotlin. I am so happy JetBrains CEO, Max Shafirov, is here today. [APPLAUSE] This new language is wonderful, but we also thought we should increase our investment in our existing languages. So we're doing that, too. Please join us at the developer keynote later today to hear our story from end to end.
OK, so let's wrap up. There are tons more features in Android O, which we don't have time to go into today. Everything from redesign settings, to Project Treble, which is one of the biggest changes to the foundations of Android to date, to downloadable fonts with new emoji, and much more. If you want to try some of these features for yourself– and you do– I'm happy to announce we're making the first beta release of O available today. Head over to android.com/beta. [APPLAUSE] But there's more. [LAUGHS] You probably thought we were done talking about Android O, but, I'd like you to hear some more about Android. And from that, please welcome Sameer. Thank you. [MUSIC PLAYING] [APPLAUSE] SAMEER SAMAT: Thanks, Steph. Hi, everyone. >From the beginning, Android's mission has been to bring the power of computing to everyone. And we've seen tremendous growth over the last few years, from the high end to entry-level devices, in countries like Indonesia, Brazil and India. In fact, there are now more users of Android in India than there are in the US.
And every minute, seven Brazilians come online for the first time. Now, all this progress is amazing. For those of us who have a smartphone, we intuitively understand the profound impact that computing is having on our daily lives. And that's why our team gets so excited about how we can help bring this technology to everyone. So we took a step back to think about what it would take to get smartphones to more people. There are a few things that are clear. Devices would need to be more affordable, with entry-level prices dropping significantly. This means hardware that uses less power-packed processors and far less memory than on premium devices. But the hardware is only half the equation. The software also has to be tuned for users' needs around limited data connectivity and multilingual use. We learned a lot from our past efforts here with Project Svelte and KitKat, and the original Android One program.
But we felt like the time was right to take our investment to the next level. So today, I'm excited to give you a sneak peek into a new experience we're building for entry-level Android devices. Internally, we call it Android Go. Android Go focuses on three things. First, optimizing the latest release of Android to run smoothly on entry-level devices, starting with Android O. Second, a rebuilt set of Google Apps that use less memory, storage space, and mobile data. And third, a version of the Play Store that contains the whole app catalog, but highlights the apps designed by all of you for the next billion users. And all three of these things will ship together as a single experience starting on Android O devices with 1 gigabyte or less of memory.
Let's take a look at some of the things we're working on for Android Go. First, let's talk about the operating system. For manufacturers to make more affordable entry-level devices, the prices of their components have to come down. Let's take one example. Memory is an expensive component. So we're making a number of optimizations to the system UI and the kernel to allow an Android O device built with the Go configuration to run smoothly with as little as 512 megabytes to 1 gigabyte of memory.
Now on-device performance is critical, but data costs and intermittent connectivity are also big challenges for users. One person put it best to me when she said, mobile data feels like currency. And she wanted more control over the way she spent it. So on these devices, we're putting data management front and center in Quick Settings. And we've created an API that carriers can integrate with, so you can see exactly how much prepaid data you have left, and even top up right there on the device. But beyond the OS, the Google Apps are also getting smarter about data. For example, on these devices, the Chrome Data Saver feature will be turned on by default. Data Saver transcodes content on the server and simplifies pages when you're on a slow connection. And, well, now we're making the savings more visible here in the UI.
In aggregate, this feature is saving users over 750 terabytes of data every day. I'm really excited that the YouTube team has designed a new app called YouTube Go for their users with limited data connectivity. Feedback on the new YouTube app has been phenomenal, and we're taking many of the lessons we've learned here and applying them to several of our Google Apps. Let me show you some of the things I love about YouTube Go. First, there's a new preview experience, so you can get a sneak peek inside a video before you decide to spend your data to watch it. And when you're sure this is the video for you, you can select the streaming quality you want, and see exactly how much mobile data that's going to cost you. But my favorite feature of YouTube Go is the ability to save videos while you're connected. So you can watch them later when you might not have access to data. And if you want to share any of those videos with a friend, you can use the built-in peer-to-peer sharing feature to connect two of your devices together directly, and share the files across without using any of your mobile data at all.
[APPLAUSE] But beyond data management, the Google Apps will also make it easier to seamlessly go between multiple languages, which is a really common use case for people coming online today. For example, Gboard now supports over 191 languages, including the recent addition of 22 Indian languages. And there's even a transliteration feature, which allows you to spell words phonetically on a QWERTY keyboard to type in your native language script. Now, Gboard is super cool, so I want to show it to you. I grew up in the US, so for any of my family that's watching, don't get too excited by the demo. I haven't learned Hindi yet. And I'm sorry, mom, OK? [LAUGHTER] So let's say, I want to send a quick note to my aunt in India. I can open up Allo, and using Gboard, I can type how it sounds phonetically. [HINDI], which means "how are you" in Hindi. And transliteration automatically gives me Hindi script. That's pretty cool. Now, let's say I want to ask her how my I/O speech is going, but I don't know how to say that in Hindi at all.
I can use the built-in Google Translate feature to say, "how is this going?" And seamlessly, I get Hindi script, all built right into the keyboard. [APPLAUSE] My family is apparently a tough audience. All right. Well, the Google Apps are getting Go-ified, what had always propelled Android forward is the apps from all of you. And no surprise, many of our developer partners have optimized their apps already.
So to better connect users with these experiences, we'll be highlighting them in the Play Store. One example is right here on Play's home page. To be eligible for these new sections, we published a set of best practices called "Building for Billions," which includes recommendations we've seen make a big difference in the consumer experience. Things such as designing a useful offline state, reducing your APK size to less than 10 megabytes, and using GCM or JobScheduler for better battery and memory performance. And also in "Building for Billions," you'll find best practices for optimizing your web experience. We've seen developers build amazing things with new technologies, such as progressive web apps. And we hope you can come to our developer keynote later today to learn a whole lot more.
OK, that was a quick walkthrough of some of the things coming in Android Go. Starting with Android O, all devices with 1 gigabyte of RAM or less will get the Go configuration. And going forward, every Android release will have a Go configuration. We'll be unveiling much more later this year, with the first devices shipping in 2018. We look forward to seeing what you'll build, and how we can bring computing to the next several billion users. Next up– next up, you'll be hearing from Clay on one of Google's newest platforms that we're really excited about– VR and AR. Thank you. [APPLAUSE] [MUSIC PLAYING] CLAY BAVOR: Thank you, Sameer. So, Sundar talked about how technologies like machine learning and conversational interfaces make computing more intuitive by enabling our computers to work more like we do. And we see VR and AR in the same light. They enable us to experience computing just as we experience the real world. Virtual reality can be transporting. And you can experience not just what it's like to see someplace, but what it's like to really be there.
And augmented reality uses your surroundings as context, and puts computing into the real world. A lot has happened since Google I/O last year, and I'm excited to share a bit of what we've been up to. So let's start with VR. Last year, we announced Daydream, our platform for mobile virtual reality. And then in October, to kick-start the Daydream ecosystem, we released Daydream View, a VR headset made by Google. And it's super comfortable. It's really easy to use. And there's tons to do with it. You can play inside alternate worlds, and games like "Virtual Virtual Reality." And you can see any part of our world with apps like Street View.
And you can visit other worlds with apps like Hello Mars. There's already a great selection of Daydream phones out there, and we're working with partners to get Daydream on even more. First, I'm pleased that LG's next flagship phone, which launches later this year, will support Daydream. And there's another. I'm excited to announce that the Samsung Galaxy S8 and S8 Plus will add Daydream support this summer with a software update. [APPLAUSE] So, Samsung, of course, they make many of the most popular phones in the world.
And we're delighted to have them supporting Daydream. So great momentum in Daydream's first six months. Let's talk about what's next. So with Daydream, we showed that you can create high quality mobile VR experiences with just a smartphone and a simple headset. And there are a lot of nice things about smartphone VR. It's easy. There aren't a bunch of cables and things to fuss with. You can choose from a bunch of great compatible phones. And of course, it's portable. You can throw your headset in a bag. We asked, how we take the best parts of smartphone VR and create a kind of device with an even better experience? Well, I'm excited to announce that an entirely new kind of VR device is coming to Daydream– what we call standalone VR headsets. And we're working with partners to make them. So what's a standalone headset? Well, the idea is that you have everything you need for VR built right into the headset itself. There's no cables, no phone, and certainly, no big PC. And the whole device is designed just for VR. And that's cool for a couple of reasons.
First, it's easy to use. Getting into VR is as easy as picking the thing up. And it's one step and two seconds. And second, presence. And by that, I mean really feeling like you're there. By building every part of the device specifically for VR, we've been able to optimize everything– the displays, the optics, the sensors– all to deliver a stronger sense of being transported. And nothing heightens the feeling of presence like precise tracking– how the headset tracks your movement. And we've dramatically improved tracking with the technology that we call WorldSense. So WorldSense enables what's known as positional tracking. And with it, your view in the virtual world exactly matches your movement in the real world. And it works by using a handful of sensors on the device that look out into your surroundings. And that means it works anywhere. There's no setup.
There's no cameras to install. And with it, you really feel like you're there. Now, just as we did with Daydream-ready smartphones, we're taking a platform approach with standalone headsets, working with partners to build some great devices. To start, we worked with Qualcomm to create a Daydream standalone headset reference design, a sort of device blueprint that partners can build from. And we're working closely with two amazing consumer electronics companies to build the first headsets. First, HTC, the company that created the VIVE. [APPLAUSE] We're excited about it, too. [CHEERING AND APPLAUSE] They're a leader in VR, and we're delighted to be working with them on a standalone VR headset for Daydream. And second, Lenovo. We've been partners for years, working together on Tango. And now, we're excited to work with them on VR.
These devices will start to come to market later this year. So that's the update on VR. Great momentum with apps, more Daydream-ready phones on the way, and a new category of devices that we think people are going to love. So let's turn to augmented reality. And a lot of us were introduced to the idea of AR last year with Pokemon GO. And the app gave us a glimpse of AR, and it showed us just how cool it can be to have digital objects show up in our world. Well, we've been working in this space since 2013 with Tango, a sensing technology that enables devices to understand space more like we do. Two years ago in 2015, we released a developer kit. And last year, we shipped the first consumer-ready Tango phone. And I'm excited to announce that the second generation Tango phone, the ASUS ZenFone AR will go on sale this summer.
Now, looking at the slides, you may notice a trend. The devices are getting smaller. And you can imagine far more devices having this capability in the future. It's been awesome to see what developers have done with the technology. And one thing we've seen clearly is that AR is most powerful when it's tightly coupled to the real world, and the more precisely, the better. That's why we've been working with the Google Maps team on a service that can give devices access to very precise location information indoors. It's kind of like GPS, but instead of talking to satellites to figure out where it is, your phone looks for distinct visual features in the environment, and it triangulates with those.
So you have GPS. We call this VPS, Google's Visual Positioning Service. And we think it's going to be incredibly useful in a whole bunch of places. For example, imagine you're at Lowe's, the home improvement store that has basically everything. And if you've been there, you know it's really big. And we've all had that moment when you're struggling to find that one, weird, random screwdriver thing. Well, imagine in the future, your phone could just take you to that exact screwdriver and point it out to you on the shelf. Turns out we can do this with VPS.
And let me show you how. And this is working today. So here we are walking down an aisle at Lowe's. And the phone will find these key visual feature points, which you can see there in yellow. By comparing the feature points against previously observed ones, those colorful dots in the back, the phone can figure out exactly where it is in space down to within a few centimeters. So GPS can get you to the door, and then VPS can get you to the exact item that you're looking for.
Further out– [APPLAUSE] Further out, imagine what this technology could mean to people with impaired vision, for example. VPS and an audio-based interface could transform how they make their way through the world. And it combines so many things that Google is good at– mapping, computer vision, distributed computing. And we think precise location will be critical for camera-based interfaces. So VPS will be one of the core capabilities of Google Lens. We're really excited about the possibilities here. So the last thing I wanted to share is something that we've been working on that brings many of these capabilities together in a really important area. And that's education. Two years ago, we launched Expeditions, which is a tool for teachers to take their classes on virtual reality field trips.
And 2 million students have used it. Today, we're excited to announce that we're adding a new capability to Expeditions– AR mode, which enables kind of the ultimate show-and-tell right in the classroom. If we could roll the video, please. [VIDEO PLAYBACK] – All right, who wants to see a volcano? 3, 2, 1. – Whoa! – Look at that lava. Look at that smoke coming out of that. Pretend you're an airplane and fly over the tornado. – That's the top of it. – What do you see? – It's either a asteroid, meteorite– – We're learning about DNA and genes– things that we can't see. And so, the most exciting thing for me with the AR technology was that I could see kids get an "aha" moment that I couldn't get by just telling them about it. – The minute I saw it pop up on the screen, it made me want to get up and walk to it. – You actually you get to turn around and look at things from all angles, so it gave us a nice perspective.
– See if you can figure out what that might be based on what you know about the respiratory system. – I got to see where the alveoli branched off, and I could look inside them and see how everything worked, which I never saw before. And it was really, really cool. [END PLAYBACK] CLAY BAVOR: We're just delighted with the response we're seeing so far. And we'll be rolling this out later in the year. So, VR and AR, two different flavors of what you might call immersive computing– computing that works more like we do. We think that's a big idea. And in time, we see VR and AR changing how we work and play, live and learn.
And all that I talked about here, these are just the first steps. But we can see where all of this goes, and we're incredibly excited about what's ahead. Thanks so much. Back to Sundar. [APPLAUSE] [VIDEO PLAYBACK] – We wanted to make machine learning have an open source project so that everyone outside of Google could use the same system we're using inside Google. [MUSIC PLAYING] [END PLAYBACK] [APPLAUSE] SUNDAR PICHAI: It's incredible [? with ?] any open source platform, when you see what people can do on top of it. We're really excited about the momentum behind TensorFlow. It's already the most popular ML repository on GitHub. And we're going to push it further. We are also announcing the TensorFlow Research Cloud.
We are giving away 1,000 cloud TPUs, which is 180 petaflops of computing to academics and researchers for free so that they can do more stuff with it. I'm always amazed by the stories I hear from developers when I meet them. I want to highlight one young developer today, Abu Qader from Chicago. He has used TensorFlow to help improve health for everyone. Let's take a look. [VIDEO PLAYBACK] [MUSIC PLAYING] [CHATTER] – My name is Abu. I am a high school student. 17 years old. My freshman year, I remember Googling machine learning. I had no clue what it meant. That's a really cool thing about the internet, is that someone's already doing it, so you can just YouTube it, and [CLICK] it's right there. Within a minute, I really saw what machine learning can do. It kind of like hit something within me. This need to build things to help people.
My parents are immigrants from Afghanistan. It's not easy coming in. The only reason we made it through some of the times that we did was because people showed acts of kindness. Seeing that at an early age was enough for me to understand that helping people always comes back to you. [INAUDIBLE] – How are you? – And then it kind of hit me– a way where I could actually generally help people. Mammograms are the cheapest imaging format there is. It's the most accessible to people all around the world. But one of the biggest problems that we see in breast cancer is misdiagnosis. So I decided I was going to build a system for early detection of breast cancer tumors, that's successful to everyone, and that's more accurate. How was I going to do it? Machine learning. The biggest, most extensive resource that I've used, is this platform called TensorFlow. And I've spent so many hours going really deep into these open source libraries and just figuring out how it works. Eventually, I wrote a whole system that can help radiologists make their decisions. All right. – Ready? – Yeah. I'm by no means a wizard at machine learning.
I'm completely self-taught. I'm in high school. I YouTubed and just found my way through it. You don't know about that kid in Brazil that might have a groundbreaking idea, or that kid in Somalia. You don't know that they have these ideas. But if you can open source your tools, you can give them a little bit of hope that they can actually conquer what they're thinking of.
[END PLAYBACK] [CHEERING AND APPLAUSE] Abu started this as a school project, and he's continued to build it on his own. We are very, very fortunate to have Abu and his family here with us today. [CHEERING AND APPLAUSE] Thank you for joining us. Enjoy I/O. We've been talking about machine learning in terms of how it will power new experiences and research. But it's also important we think about how this technology can have an immediate impact on people's lives by creating opportunities for economic empowerment. 46% of US employers say they faced talent shortages and have issues filling open job positions while job seekers may be looking for openings right next door.
There is a big disconnect here. Just like we focused our contributions to teachers and students through Google for Education, we want to better connect employers and job seekers through a new initiative, Google for Jobs. Google for Jobs is our commitment to use our products to help people find work. It's a complex, multifaceted problem, but we've been investing a lot over the past year, and we have made significant progress. Last November, we announced the Cloud Jobs API. Think of it as the first fully end-to-end, pre-trained, vertical machine learning model through Google Cloud, which we give to employers– FedEx, Johnson & Johnson, HealthSouth, CareerBuilder, and we're expanding to many more employers. So in Johnson & Johnson's career site, they found that applicants were 18% more likely to apply to a job suggesting the matching is working more efficiently. And so far, over 4 and 1/2 million people have interacted with this API. But as we started working on this, we realized the first step for many people when they start looking for a job is searching on Google.
So, it's like other Search challenges we have worked in the past. So we built a new feature in Search with a goal that no matter who you are or what kind of job you are looking for, you can find the job postings that are right for you. And as part of this effort, we worked hard to include jobs across experience and wage levels, including jobs that have traditionally been much harder to search and classify– think retail jobs, hospitality jobs, et cetera. To do this, well, we have worked with many partners– LinkedIn, Monster, Facebook, CareerBuilder, Glassdoor, and many more. So let's take a look at how it works. Let's say you come to Google and you start searching for retail jobs. And you're from Pittsburgh. We understand that. You can scroll down and click into this immersive experience. And we immediately start showing the most relevant jobs for you. And you can filter. You can choose Full-time. And as you can see, you can drill down easily.
I want to look at jobs which are posted in the past three days. So you can do that. Now, you're looking at retail jobs in Pittsburgh, posted within the last three days. You can also filter by job titles. It turns out employees and employers use many different terminologies. For example, retail could mean a store clerk, a sales representative, store manager. We use machine learning to cluster automatically, and so that we can bring all the relevant jobs for you. As you scroll through it, you will notice that we even show commute times. It turns out to be an important criteria for many people. And we'll soon add a filter for that as well.
And if you find something that's of interest to you– so maybe the retail position [? in ?] Ross. And you can click on it, and you end up going to it right away. And you're one click away. You can scroll to find more information if you want. And you're one click away from clicking and applying there. It's a powerful tool. We are addressing jobs of every skill level and experience level. And we are committed to making these tools work for everyone. As part of building it, we literally talked to hundreds of people. So whether you are in a community college looking for a barista job, a teacher who is relocating across the country and you want teaching jobs, or someone who is looking for work in construction, the product should do a great job of bringing that information to you. We are rolling this out in the US in the coming weeks, and then we are going to expand it to more countries in the future.
I'm personally enthusiastic for this initiative because it addresses an important need and taps our core capabilities as a company, from searching and organizing information, to AI and machine learning. It's been a busy morning. We've talked about this important shift from a mobile first to a AI first world. And we're driving it forward across all our products and platforms so that all of you can build powerful experiences for new users everywhere.
It will take all of us working together to bring the benefits of technology to everyone. I believe we are on the verge of solving some of the most important problems we face. That's our hope. Let's do it together. Thanks for your time today, and enjoy Google I/O. [APPLAUSE] [MUSIC PLAYING].
welcome I'm truly excited to be here today to talk to you guys about location and sensors get the clicker my name is Steve malkos I'm the technical program manager for the Android location and context team I was amazed earlier today to hear sundar talk so much on sensors and context being part of the location and context team we touch every part of Google and and I'm truly excited about that so without further ado okay let's get into it today we carry the world in our pocket we carry more computing power and faster data speeds than we did just nearly half a decade ago from our desktop PCs one item that has remained the same throughout a computing history has been building these innovative software applications and you guys are continuously shaping this world what we need to do more we can do more we could build deeper and richer experiences in this talk will focus on what it takes to build an awareness application will go through sensors and location focusing on bringing richer experiences by using them then we'll take a deep dive into the Android sensor hub and go through the cool new things that we're building for it awesome ok so what does it take to build these highly contextual apps so what does it take to build these high these these high quality apps today sorry the notes are not right let me take care of that for you our phones are very personal and interactive they have transformed the way we compose our day we use them to get traffic info health and safety and notifications and much more so it's understandable that your biggest concerns are on how to make your apps more contextually aware so let's focus on what it takes to build these high quality apps when creating these awareness application there are typically three main layers that we need to look at that's sensors algorithms and user experiences think of an awareness application like a human sense for example your inner ear acts as the accelerometer and gyroscope of the device your eyes act as the camera sensor that raw data from your ears to your eyes gets sent up to your brain and processed like the algorithms on the device that's then classified into the context and outputs as the user experience like I'm running or I'm running with a buddy MEMS sensors allows us to take a deeper programming perspective on comprehending the environment around us so when building these contextual applications we're gonna focus on these four main pillars that's coverage accuracy latency and power will refer to these pillars throughout the entire talk these are the building blocks for creating higher-quality applications let's walk through examples of each of these the first pillar is coverage for location it must work all the time indoors outdoors and in every country at Google were continuing to improve our coverage maps with our crowdsource models and Google databases to make location better for activity recognition it has to support a wide array of different types of activities like I'm running or I'm walking or I'm in a vehicle and and nearby notifications needs to work across a wide array of different types of devices finally sensors have to work seamlessly and uniformly a across the entire ecosystem the second pillar accuracy from the macro view GPS must be very accurate users don't want position jumps they want the positions to be under five meters all the time indoor and outdoor transitions also have to be as smooth as possible from the macro view that we saw on the previous column to the micro view on position accuracies it must work perfectly for our virtual world examples of better virtual reality can come from sensors sensors that output less drift at lower noise our user also wants very accurate activity detection for example it's not okay to classify biking activity if you're shaking your leg in a moving car the third column that I'd like to discuss is latency from the macro view users don't want to wait for their positions location should show up instantaneously and activity detection needs to happen near real-time so the users entire activity could be tracked with almost no latency from the macro to the micro on Layton sees our sensors cannot lag when we have sensor and display lags like this your virtual world can go from making you from making you a beautiful scene to making you nauseated very quickly we want the experience to be crystal clear we take great care and ensuring that our sensor latency requirements are met this ensures that when you write your applications you could be certain that they'll perform to your users expectations our final column power that's battery life one of the most important columns when building your applications we all know that if your application sucks up the users battery life they'll quickly uninstall it from you their device we need to make it or you guys don't have to worry about power we can do this by avoiding too many knobs simplicity is key power has even to subtly NC and accuracy it means we could run more often improving on Layton sees it also means we could get access to more signals thus improving on accuracy so in the theme of building better user experiences by bringing higher quality applications focused on coverage accuracy latency and power let's hear from Ashutosh on Android sensors hi thanks T for setting out the lay of the land for the rest of the presentation I'm Ashutosh and over the next few minutes we shall be talking about how we set about achieving these four pillars of context for sensors I hope that gave you some insight into the challenges we face and the power we derived by work in working with the ecosystem I shall show you some of the new and shiny things we have in store for you for Android n and finally we share some of the lessons that we've learned in developing Android experiences which will probably be of value to you as well I challenge that we face is the constant tension between increasing coverage but formally defining new sensors in Android and ensuring that we can keep a consistent user experience across the ecosystem let me give you an example we have received requests to allow accelerometers that only support two axes these are required for low-end markets where the z-axis is not required if the only thing that accelerometer does is detect landscape and portrait detection on the other hand we have been asked to mandate a higher dynamic range for accelerometers because it allows some safety applications to be developed which can potentially save lives the problem is that these two goals are mutually contradictory a sensor that only supports two axes will not have a high dynamic range the economics of the industry work against it the example goes to coverage we must ensure that all Android sensors can eke out the maximum visibility from the hardware whilst ensuring that the application developers seek consistent API is across all of em Rock over the years which steadily added new formal sensor definitions while the major physical sensors like accelerometer find very early on we've continued to define new capabilities like sensor fusion rotation vector geometry geomagnetic rotation vector and new sensor types for completely new form factors like Android wear these new capabilities increased coverage to develop awareness applications because most sensor types mean that we can get more data about the world around us and derive richer inferences we are adding a few things this year for the Android enemies let me walk you through some of them let's start with something really exciting all orientation sensors thus far was centered around orientation they provided the orientation of the device in terms of roll pitch and yaw with reference to either true north of gravity or an arbitrary initial point we are adding the three axes of translational freedom by introducing the sixth off sensor type apps will not only know the orientation but the precise displacement of the device in the world this means that motion control in games can go just beyond tracking where the user is looking now they will be able to move the player in the virtual world just as the player moves in the real world you can walk and not just pin in place augmented reality experiences will be even more immersive now that they can mirror the users movements just to set expectations this sensor is not likely to be widely pervasive across Android but we believe that the introduction of a formal sensor type is required to give the system a push towards augmented reality applications on the variables form factor we extending the capabilities of what it is what is available through the heart rate sensor thus far you have been able to get an average snapshot of the heartbeat measurement which is great but it leaves a lot of information on the table we will expose a much more finer grained event for every heartbeat which should more or less more or less correspond to the systolic peak for each heartbeat this will enable a whole new class of fitness and wellness applications like reporting and monitoring mental stress physical stress / training exercise recovery and sleep analysis etc and you guys will be writing all one of the most often repeated requests from app developers to us was to expose the hardware capabilities to determine the motion state of the device we adding these sensors to the Android sensors API as one-shot events will we shall expose 202 of them one shall fire if the device enters the stationary state and one that fires if the device starts moving we hope that this will allow the applications to be more judicious in the use of system resources and developers will use this ability of motion determination in low-power hardware to get the resource usage finally this year we realized that applicant processors and display controllers are now smart enough to enter low-power States if only we did not keep pinging the application processor with the accelerometer data to compute the device orientation so we're formalizing the screen orientation as a new sensor type that can be implemented by the OEMs to get the low-power domains to compute the device orientation however developers can and should continue to use the Android activity and window manager API is Android will ensure that regardless of the underlying sense of support your app will continue to work seamlessly across all addressable devices and if this sensor is supported you get all the power benefits finally we are adding capabilities to 'allah applications on an Android device to discover and subscribe to external sensors through the sensors API by external sensors here I mean any sensor that is not present on the device when the device boots up now this can be an accessory for the device that physically attaches to the device or a truly external device that connects wirelessly to your phone this is a nod to the central rolls their phone have in our lives and the realization that a single device cannot be all things to all of the people all of the time device capabilities can be augmented and we want to provide the standard way to access such data what is coming up is exciting and I cannot wait to see what cool things the developers will come up with these new sensor types having decided how we want to increase coverage how do we establish accuracy and consistency how do we ensure that when a new API version is released what makes it to the app developers is consistent across Android this is a complex process and involves the hard work of hundreds of thousand people across the Android ecosystem let me walk you through some of the challenges Android is diverse just on the harbor side there are 300 plus OEMs 200 plus carriers and operators hundreds of component vendors dozens of CPU or SOC vendors with a range of products there are dominant players that have huge market shares and there's a long tail that caters to very specific niche cases somehow we have to ensure that we listening to everyone and helping everyone in the ecosystem succeed and then there's a sheer range of devices that under can support all of the devices on this slide are Android devices the same specifications rule them all more or less there are few restrictions on Monday that Android stipulates as an example it's completely okay to make an Android device without a single sensor type not even next kilometer that sounds bizarre if one thinks of Android purely as a cell phone platform but as immediately obvious once you think of Android TV products and convince yourself that determining the device orientation of a TV would be an edge case at best so what does it really mean to support a sensor what sensor should a device have what guarantees if many are made explicit to the developer and how these are difficult questions that keep us busy and what is agreement on these goals how does its diverse ecosystem take excuse what tools do we used to make this happen in a nutshell we drive this process by creating standards for Android producing exemplary Android devices at Google and working in tight collaboration with our partners the very first step is the definition of Android compatibility and under device is supposed to meet 100% of the specifications as defined in the Android compatibility document the Android CDD it goes into lots of gory details about what kind of sensor should be on device how they should be announced and they've announced what bars they must meet with tune the CDD language to advise the ecosystem on what is coming down the road and mash them into more aspirational requirements the CD is enforced through the CTS a compatibility test suite all Android devices must pass the CTS there are no exceptions or any waivers for any device we keep adding CTS tests to patch holes in our testing note however that the CTS is an enforcement mechanism only it cannot possibly cover the entire specifications exhaustively the artifact of record is the CD what do we expect of developers please take a look at the Android CD if you're looking to deploy an app across a large user base it will help set you expectations on what is standardized and what to expect Google devices are another tool we use all Google devices are great devices they also serve to set a bar for the rest of the OEMs a nexus device is subject to a lot of scrutiny outside moving the OEM scrutinize it to make sure that their devices in that class are as good as or even better than the Nexus they also use that opportunity to figure out the holes in our specifications if something is ambiguous a reasonably good assumption would be to go along with what the Nexus is doing the implication here is obvious do teste applications on Google devices I think most of you follow this practice already but please however go beyond just the Nexus that launches this year do teste applications on previous Nexus devices and extend your testing to say Android one devices on the sensor side we go even further last year Google wrote the center stack completely by itself and we realize we are releasing they the complete sense of stack source for the Nexus devices in open source you can get the code for the sensor sub software at the repository on this slide we have two primary reasons to do this the first is to make life easier for the ecosystem they now have access to the source code that runs all the Nexus devices and should be able to comply with all the requirements we are putting forth and hopefully even do better the second is guidance we hope that simply by releasing the source code we convey our intents and expectations very loudly in fact if you look at the source code released you will find the beginnings and the genesis of some of the future work that we shall release in Android n we have not written a ton of formal documentation yet but we are finding that our partners are reading the release source and coming up with a very good idea about what to expect finally we must continue to build internal expertise to keep abreast of the developments in the space we learn constantly and continue to invest in internal capabilities to learn more about sensors and algorithms let me show you how here at Google we are innovating new groundbreaking methods to test our solutions and improve the quality of essential Gardens can you play the video please in this video you'll see Matt programming one of our robotic arms to simulate unfriending swinging for pedestrian dead reckoning solutions in this example you see two things happening first Matt will start by using a motion capture system the motion capture system is able to records mash.sam swings with centimeter level process precision we then translate the exact motion of that capture system into a robotic arm the robotic arm then repetitively tests a specific scenario on an automated basis again and again that gives us the ground tools for us to test our algorithms and sensor solutions let's watch that was cool and Matt seem to be having way too much fun on there the final the final tool in a chest is collaboration with our partners we work very closely with our partners throughout the year to make sure that they are aware of what we are doing this is a symbiotic relationship the partners want to make sure that their product roadmaps are aligned with ours so they are not caught by surprise by any of the announcements we make we on the other hand want to make sure that we are not shouting into thin air and have realistic expectations about what is achievable we meet them constantly throughout the year every year we hold an Android boot camp and give them a glimpse of our plants on the census side last year we conducted many summits with sensor vendors Assoc vendors and the OEMs if we are successful nothing that I'm saying today would be a surprise to any of our partners they should be sitting back and saying we've got this taken care of she's the cool thing for you why do you think they listen to us other than our charming personalities the reason they is they do is because they want to reach you the developers some of the best and the brightest on this planet are developing incredibly exciting technologies and they want them to be exposed to you we are merely facilitating this meeting of immensely creative and smart people from both worlds the ecosystem is what we derive by inference from the ecosystem is what we serve take away reach out to us let us know what you're looking for what is missing what needs improvement what is truly exceptional and you want more off you'll find plenty of ways to reach us at the end of the presentation the animal sense of the API has not only become richer by adding more sensor types it also gives great control for the developers to optimize system performance in the next few minutes I would like to talk about some recommendations that'll make your apps scale across different tiers of devices and should under the hood optimized for the hardware that your app is using I will use power as an example as to the extent possible you want to spend as little power for your apps let us look at some ways that will help you achieve this on many devices there's a multi-layer approach to processing at least as far a sense of the concern at the very source there are sensors like the accelerometers and the gyroscopes sensors today have come a long way from years ago and themselves consume very little power they're really power efficient accelerometers and gyros today take less than a milli watt to sample data they they embed smart algorithms like gestures and motion detection some even have rhodium entry programming frameworks for customization then there's the application processor this is your big honking chip that you're using when you're running apps or playing games running the application processor takes a substantial amount of power so if every time you try to process a single measurement you wake up the application processor it becomes limiting very quickly all the power savings from the sensors are dwarfed if the application processor wakes up too often it consumes a lot of power and is incredibly powerful competition but it's also very power hungry it has to stay on for a few seconds even if it needs to do a little bit of processing it needs time to wake up and sleep if every time you get a sensor measurement you wake up the application processor it never gets a chance to go to sleep so getting the application percent of sleep is a key power settings technique modern Hardware adds an additional layer for sensor processing we show that as a sensor hub in this diagram this low-power computer domain may be part of the sensor itself a special power domain on the SOC or a discrete chip its goal however is the same to offload computation from the application processor and save power there are a few hooks on the sensors API that lets you achieve this code for example batching you can use the sensors API to specify the maximum delay your application can tolerate to receive sensor data this means less frequent updates to the application processor the events can be grouped and processed together the bigger the storage buffers on the hubs the more power is saved another design pattern is to specify the goal that you want to achieve and then we've woken up when that situation becomes true then the low-power hardware can monitor for the decision to become true instead of the main application processor this allows always-on monitoring of events that would have been prohibitive by looking at the raw sensor data all the time and then as an example if you wanted to change the app behavior when it looks like the user has moved from one place to another you can use a combination of geofences the significant motion sensor type to get you're processing for the N release we have added new sensor types reject motion and stationarity by using the right triggers you will consume power only when needed and not miss a moment of interest for you a less obvious design pattern is to use the highest level of abstraction possible while the api's may look like they are all exposed at one level if you become a little familiar with how the system is built you will be able to see where in the system the API is are tapping into take for example the sensors accelerometers gyros and magnetometers if TX is the raw data the system must ensure of its end of the contract and give you accurate reliable and timely data it cannot and should not put in any smarts however if you ask for the rotation vector the same sensors are used and data is fused it will be tempting to roll your own sensor fusion algorithm by processing the raw data directly however you will lose Hardware optimizations on the table as an example most devices today if they computing the rotation vector in Hardware sample the underlying heart sensors and an extremely hurried sometimes in the order of kilohertz there is no way we can support that frequency of updates on the application processor and if by rolling your own sensor fusion algorithms you will be leaving power and performance on the table so this is pretty complex I hear you I hear we have we have been trying to come up with a specification the less the application developers know that the sensors on a device are tough notch in the MNC release we exact we added exactly such a signal for the developers we call it high five sensors if the feature is defined you will know that the sensor are really accurate there are stringent requirements on the resolution range and performance of all the expected sensors you will know that a large batch size is supported allowing you to conserve a battery you will know that the timestamps on the sensors will be very highly accurate for many applications that integrate sensor values over time for example dead reckoning and error in the timestamps is the same as an error in the sense of value itself and often worse finally you will know the sensors are low latency Wi-Fi sensors have stringent latency requirements to support all possible interactive use cases that we have run into checking for the high-five sense of support is very easy this code snippet shows you how to do it we're simply looking for a feature string to be declared so with the quick query to the package manager you can be assured that your device has great sensors and and will support all the fundamental pillars of contextual awareness sensor capabilities are improving getting more ubiquitous and we're seeing a huge demand for these additional capabilities we are the location and context team we're constantly working on integrating sensors into everyday use experiences to make Android devices useful and a delight to use location is the other huge and complex sensor that a services consume I will hand it over to David to walk you through that part of the story thanks Ashutosh hi I'm David and I work as a software engineer on the Android location team you've just heard about a lot of the exciting capabilities we have with low-level sensors and now I'm going to talk about our higher level location api's and how they do a lot of the sensor fusion work for you the fused location provider is our primary API for producing locations it combines many of the sensors Ashutosh just talked about including GPS Wi-Fi cell accelerometer magnetometer and gyroscope it's used by Google Maps to power the current location blue dot and by Google now to provide invisible assistance like reminding you where you parked your car Android apps provide input hints on accuracy requirements and frequency and the SLP decides which sensors to turn on in order to manage both power and accuracy within the constraints of user settings in the next few slides I'll dive into the details of some of these sensors and show how the SLP uses them GPS is one of the most mature location sensors that we use and when it works it usually works great it shines outdoors when you have a clear view of the sky but it has some trouble in cities with tall buildings and when you're indoors it often doesn't work at all it's relatively power hungry and eats the battery pretty quickly when active the SLP will use GPS when app requests the highest accuracy location available it will also sometimes use GPS for balance power app requests on newer devices that support GPS batching Wi-Fi is another staple location technology that has become more and more useful as Wi-Fi access points have proliferated around the world unlike GPS it works well indoors where people typically have Wi-Fi the accuracy usually isn't as good as GPS but it's good enough to tell you what part of the building you're in and often what floor you're on like GPS the power isn't great but it can be improved by reducing the location frequency and knowing which cell tower you're connected to also gives a very coarse estimate of location it can usually tell you what city you're in and sometimes which a neighborhood unlike GPS and Wi-Fi it works both indoors and out and the battery life is great given that the cell modem is continually reconnecting to cell towers anyway so GPS Wi-Fi and cell are the foundation of location sensors surface through the FLP but one of the best parts of using the FLP in your applications is that you continually take advantage of new Google developments in both Hardware adoption and algorithm improvements most of these improvements become active in your apps without any code changes and the next few slides I'll highlight some of the improvements we've recently made and show how to take advantage of them in your apps as I mentioned before dense urban environments are very challenging for GPS when driving near tall buildings it's common for the location to drift or jump to the wrong block at Google we test GPS on a variety of phones and environments this image shows a set up for driving tests that we perform in San Francisco which is one of the harshest GPS environments on the west coast we use a military-grade inertial navigation system that provides centimeter level accuracy in order to collect ground troops location for comparison we also take video imagery to compare position jumps and legs when all devices are in navigation mode this gives us the automated ability to count and measure each device's position jump or leg and then produce a report listing the exact moment when a device experienced these legs and jumps this is an example test drive in downtown San Francisco you can feel a lot of issues with this track positions clump together poor latency and even some positions inside buildings in order to solve some of these problems we've worked with our GPS chipset partners to incorporate their sensor fusion algorithms that combine accelerometer gyroscope magnetometer and barometer with rod GPS pseudo ranges the chips that can fall back to these other sensors when the GPS signal is weak providing a much better user experience this image shows a before-and-after result of this GPS chipset fusion working on a nexus 6p you can see the blue line provides a much more stable position resulting from this fusion as the user drives in an urban environment we've also tackled accuracy in our Wi-Fi location models over the last year we've improved indoor median accuracy by 40% in Google Maps this can mean the difference between knowing it you're in a shopping mall and knowing how close you are to the actual store you're trying to find we've added significant use of the inertial sensors accelerometer gyroscope and magnetometer these sensors don't tell us where the user is directly but they describe how the user moves so we can use them in conjunction with GPS and Wi-Fi to improve location one example of how the SLP uses the accelerometer is by activating it when the device is stationary in order to save power rather than performing costly Wi-Fi scans over and over again we write rely on the low-power accelerometer to tell us when the device moves at which point we resume Wi-Fi scanning we also use all three of these sensors to improve the accuracy of Wi-Fi location if you open Google Maps indoors and walk around you may notice the blue dot follow you around the building as I mentioned before Wi-Fi accuracy alone is not that great but by using sensors we can count your steps detect when you turn and combine this information with Wi-Fi dehonian more accurate location now Ashutosh already talked about sensor batching where we can accumulate sensor data on low-power hardware and process it later likewise we've added support in the FLP for location batching where it collects location data at lower power and processes the batch of locations after the fact if your application doesn't need immediately immediate location updates you can save significant battery by requesting batching this allows the SLP to utilize sensor batching under the hood and also to consolidate network queries an example of this use case might be a sickness app that tracks the user's activity the user will periodically want to check their progress throughout the day but doesn't necessarily need constant updates unlike the other SLP improvements I've mentioned location batching requires code changes in order to activate the power savings in your app here's a code example to activate batching you use a location request object to provide your apps constraints to the SLP the set interval method tells the SLP how frequently to compute locations to activate batching the magic wait call is set max wait time which tells the SLP how frequently due to deliver locations to your app so in this particular example the FLP will compute a location every minute but will deliver them to the app every five minutes typically in batches of five locations now back to the fitness app example let's say it's batching the users locations and then the user opens the app and wants to see their progress immediately the app can call flush locations which tells the FLP to interrupt a batch in progress and deliver the most up-to-date locations as soon as possible the user will see all locations collected in the app and then batching will continue as previously requested so by making a few code modifications the fitness app can take advantage of location batching with significant power savings while still providing the same responsive user experience now that we've seen the latest features of the fused location provider I want to talk briefly overview another set of location api's geofencing the geofencing API is notify your app when the user enters or exits predefined areas of interest such as their work or home it's built on top of the FLP so it inherits all the new features we just talked about that improve accuracy and conserve power in addition geofencing uses activity recognition to dynamically manage power based on the user speed and proximity to the geo fences of interest this code sample shows how to incorporate geofencing into your apps you define up to a hundred circular regions of interest and register for notification I've highlighted the set notification responsiveness call because it's very important for minimizing power consumption it specifies the delay you can tolerate and receiving notification when the user enters or exits a geofence so your app should set this to be as high as as reasonable for your use case so I've shown a brief overview of our location api's and how they do the work of fusing the latest and greatest sensor technologies for you now I'll hand it back to Steve who's going to talk about her upcoming hardware features and how we plan to incorporate them into our api's great awesome thanks David we just heard how we're moving more and more algorithms down into our low-power processing domains or how we just want to offload as much as we can from the main application processor next I'm gonna take a deep dive and talk to you about the Android sensor hub and the cool new things that we're building for it ok our mission with the Android sensor hub is to use all sensors in all wireless radios on a mobile device so we can have a better understanding of the state of the users context and location we want to simplify our users interactions for example pickup to wake the device we want to augment human memory and knowledge like we did with Google Google now with where's my car and we want the users to have a better understanding of themselves like google fit so what did we launch in marshmallow we launched a standalone microcontroller runs always on processing we did this on the Nexus 5x and 6p and we open sourced it to the ecosystem why did we need this separate processor today the cost of computing activities in an always-on fashion for location and activity recognition is really cost prohibitive because of power by introducing this tiny 3×3 millimeter chip we could reduce latency x' improve accuracy x' without hardly affecting the users battery life I'd like to highlight though that sensor hubs are not anything new they've been around for many years however at Google we wanted to standardize on this output so that there's more consistency in the ecosystem this gives us better assurances that when you write your applications they'll work the same across all Android so what did we do how did we do this we connected the following into the Android sensor hub accelerometer gyroscope magnetometer barometer proximity and ambient light sensor by establishing these connections we were able to achieve the following I'm not going to read through this whole list but I want to highlight some examples it's running everything in sensors dot H this includes things like game rotation vectors gravity we're running significant motion detector which is the primary use case for Android doze mode all of our activity recognition models are in this hub and we introduce new gestures like double twist which is a private sensor that you could register your application to today so what are we savings in terms of power it's quite simple when you run activity recognition on the application processor it consumes quite a bit of power when we introduce sensor batching we were able to cut that number down in half running that same activity recognition model on the android sensor hub costs us just a fraction of the power and this is just the start so here's a sneak peak on what we're working on this year I'm gonna highlight some of the big items we're introducing GPS sudo ranges this is groundbreaking as is the first time in history a mobile application will have direct access to the raw GPS measurements this is beneficial to many but especially the phone makers because they could use these measurements to help them in their GPS performance testing and if you ever had a bright idea on how to use GPM at GPS measurements now's your time to shine we're also introducing personalized activity recognition models because we all know my step lengths different from your step length by having a personalized model we'll be able to better our accuracies for activity recognition with the Android sensor hub will introduce downloadable code in marshmallow when we launched activity recognition we lost the capability of updating our algorithms frequently but gained the power savings by running those in the low-power domain in Nexus n or Android n will have the best of both worlds low-power activity recognition with the capabilities are continually updating our algorithms will also connect GPS Wi-Fi and cell directly into the sensor hub which I'll talk about in the next slide so by adding these additional connectivity signals we're gonna move our entire location engine into the hub I noted earlier that power has two sub pillars those were latency and accuracy by moving the location stack down into the low-power domain will improve geofencing because location engines running there so well latency will become better because we could run more often and accuracies will become better because we will have access to more signals so those are the main themes for this year let's touch a bit in the future in order to truly understand the state of a user's context and location we need to answer the following three key questions where's the user examples GPS latitude longitude geofencing which is within your circle or not and semantic location like where am i I'm in Starbucks for activity we need to answer what is the user doing are they walking running on a bike and for nearby we need to answer which devices can I connect to which chromecast devices are nearby so in order to achieve or answer these three key questions we need to push down everything listed here into the low-power compute domain for a location we're already working on the fused location provider and geofencing this year for activity recognition we'll continue to add new sensors and new activity recognition algorithms down into the low-power domain and for nearby we'll work on adding nearby connections and messages into the low-power domain so by answering these three key questions the Android sensor hub will bring the best of Google's databases machine learning algorithms and Google infrastructure at the lowest power this will allow us to bring you always on location and personalized context we'll continue to add more and more signals down into these low-power domains we've already achieved activity recognition and and sensors in marshmallow and soon are about to launch location and dynamically downloadable code in n the future will become more and more contextually aware and by adding these signals will help make you deeper richer experiences in your applications we're truly excited about these possibilities and are looking forward to uh shirring a new wave of api's for your applications thank you so I have one last slide if we could go we have a new developer site this is a nice refresh site go to developers.google.com slash location – awareness we've placed all of our api's in a fresh new look for you guys to acces our location and context information we also have a quick survey that we'd be really interested in getting your feedback on our sensor API so please go to the second link and and fill out this quick and easy survey you could do it at your leisure or do it right now and then the lastly David Ashutosh and I will hang outside for a little while if you guys have any questions or we have office hours on Thursday and Friday at these times thank you very much