![robots]()
Last March, I was in Costa Rica with my girlfriend, spending our days between beautiful beaches and jungles full of exotic animals. There was barely any connectivity and we were immersed in nature in a way that we could never be in a big city. It felt great.
But in the evening, when we got back to the hotel and connected to the WiFi, our phones would immediately start pushing an entire day’s worth of notifications, constantly interrupting our special time together. It interrupted us while watching the sunset, while sipping a cocktail, while having dinner, while having an intimate moment. It took emotional time away from us.
And it’s not just that our phones vibrated, it’s also that we kept checking them to see if we had received anything, as if we had some sort of compulsive addiction to it. Those rare messages that are highly rewarding, like being notified that Ashton Kutcher just tweeted this article, made consciously “unplugging” impossible.
Just like Pavlov’s dog before us, we had become conditioned. In this case though, it has gotten so out of control that today, 9 out of 10 people experience “phantom vibrations”, which is when you think your phone vibrated in your pocket, whereas in fact it didn’t.
The Digital Revolution
How did this happen?
Back in 1990, we didn’t have any connected devices. This was the “unplugged” era. There were no push notifications, no interruptions, nada. Things were analog, things were human.
Around 1995, the Internet started taking off, and our computers became connected. With it came email, and the infamous “you’ve got mail!” notification. We started getting interrupted by people, companies and spammers sending us electronic messages at random moments.
10 years later, we entered the mobile era. This time, it is not 1, but 3 devices that are connected: a computer, a phone, and a tablet. The trouble is that since these devices don’t know which one you are currently using, the default strategy has been to push all notifications on all devices. Like when someone calls you on your phone, and it also rings on your computer, and actually keeps ringing after you’ve answered it on one of your devices! And it’s not just notifications; accessing a service and finding content is equally frustrating on mobile devices, with those millions of apps and tiny keyboards.
If we take notifications and the need for explicit interactions as a proxy for technological friction, then each connected device adds more of it. Unfortunately, this is about to get much worse, since the number of connected devices is increasing exponentially!
![rand]()
This year, in 2015, we are officially entering what is called the “Internet of Things” era. That’s when your watch, fridge, car and lamps are connected. It is expected that there will be more than 100 billion connected devices by 2025, or 14 for every person on this planet. Just imagine what it will feel like to interact manually and receive notifications simultaneously on 14 devices. That’s definitely not the future we were promised!
There is hope though. There is hope that Artificial Intelligence will fix this. Not the one Elon Musk refers to that will enslave us all, but rather a human-centric domain of A.I. called “Context-Awareness”, which is about giving devices the ability to adapt to our current situation. It’s about figuring out which device to push notifications on. It’s about figuring out you are late for a meeting and notifying people for you. It’s about figuring out you are on a date and deactivating your non-urgent notifications. It’s about giving you back the freedom to experience the real world again.
When you look at the trend in the capabilities of A.I., what you see it that it takes a bit longer to start, but when it does, it grows much faster. We already have A.I.s that can learn to play video games and beat world champions, so it’s just a matter of time before they reach human level intelligence. There is an inflexion point, and we just crossed it.
![rand1]()
Taking the connected devices curve, and subtracting the one for A.I., we see that the overall friction keeps increasing over the next few years until the point where A.I. becomes so capable that this friction flips around and quickly disappears. In this era, called “Ubiquitous Computing”, adding new connected devices does not add friction, it actually adds value!
![rand2]()
For example, our phones and computers will be smart enough to know where to route the notifications. Our cars will drive themselves, already knowing the destination. Our beds will be monitoring our sleep, and anticipating when we will be waking up so that we have freshly brewed coffee ready in the kitchen. It will also connect with the accelerometers in our phones and the electricity sockets to determine how many people are in the bed, and adjust accordingly. Our alarm clocks won’t need to be set; they will be connected to our calendars and beds to determine when we fell asleep and when we need to wake up.
All of this can also be aggregated, offering public transport operators access to predicted passenger flows so that there are always enough trains running. Traffic lights will adjust based on self-driving cars’ planned route. Power plants will produce just enough electricity, saving costs and the environment. Smart cities, smart homes, smart grids.. They are all just consequences of having ubiquitous computing!
By the time this happens, technology will have become so deeply integrated in our lives and ourselves that we simply won’t notice it anymore. Artificial Intelligence will have made technology disappear from our consciousness, and the world will feel unplugged again.
I know this sounds crazy, but there are historical examples of other technologies that followed a similar pattern. For example, back in the 1800s, electricity was very tangible. It was expensive, hard to produce, would cut all the time, and was dangerous. You would get electrocuted and your house could catch fire. Back then, people actually believed that oil lamps were safer!
But as electricity matured, it became cheaper, more reliable, and safer. Eventually, it was everywhere, in our walls, lamps, car, phone, and body. It became ubiquitous, and we stopped noticing it. Today, the exact same thing is happening with connected devices.
Context-Awareness
Building this ubiquitous computing future relies on giving devices the ability to sense and react to the current context, which is called “context-awareness”.
A good way to think about it is through the combination of 4 layers: the device layer, which is about making devices talk to each other; the individual layer, which encompasses everything related to a particular person, such as his location history, calendar, emails or health records; the social layer, which models the relationship between individuals, and finally the environmental layer, which is everything else, such as the weather, the buildings, the streets, trees and cars.
For example, to model the social layer, we can look at the emails that were sent and received by someone, which gives us an indication of social connection strength between a group of people.
![rand3]()
The graph shown above is extracted from my professional email account using the MIT Immersion tool, over a period of 6 months. The huge green bubble is one of my co-founder (which sends way too many emails!), as is the red bubble. The other fairly large ones are other people in my team that I work closely with. But what’s interesting is that we can also see who in my network works together, as they will tend to be included together in emails threads and thus form clusters in this graph. If you add some contextual information such as the activity I was engaged in, or the type of language being used in the email, you can determine the nature of the relationship I have with each person (personal, professional, intimate, ..) as well as its degree. And if you now take the difference in these patterns over time, you can detect major events, such as changing jobs, closing an investment round, launching a new product or hiring key people! Of course, all this can be done on social graphs as well as professional ones.
Now that we have a better representation of someone’s social connections, we can use it to perform better natural language processing (NLP) of calendar events by disambiguating events like “Chat with Michael”, which would then assign a higher probability to my co-founder.
But a calendar won’t help us figure out habits such as going to the gym after work, or hanging out in a specific neighborhood on Friday evenings. For that, we need another source of data: geolocation. By monitoring our location over time and detecting the places we have been to, we can understand our habits, and thus, predict what we will be doing next. In fact, knowing the exact place we are at is essential to predict our intentions, since most of the things we do with our devices are based on what we are doing in the real world.
![rand4]()
Unfortunately, location is very noisy, and we never know exactly where someone is. For example below, I was having lunch in San Francisco, and this is what my phone recorded while I was not moving. Clearly it is impossible to know where I actually am!
![rand5]()
To circumvent this problem, we can score each place according to the current context. For example, we are more likely to be at a restaurant during lunch time than at a nightclub. If we then combine this with a user-specific model based on their location history, we can achieve very high levels of accuracy. For example, if I have been to a Starbucks in the past, it will increase the probability that I am there now, as well as the probability of any other coffee shop.
And because we now know that I am in a restaurant, my devices can surface the apps and information that are relevant to this particular place, such as reviews or mobile payments apps accepted there. If I was at the gym, it would be my sports apps. If I was home, it would be my leisure and home automation apps.
If we combine this timeline of places with the phone’s accelerometer patterns, we can then determine the transportation mode that was taken between those places. With this, our connected watches could now tell us to stand up when it detects we are still, stop at a rest area when it detects we are driving, or tell us where the closest bike stand is when cycling!
These individual transit patterns can then be aggregated over several thousand users to recreate very precise population flow in the city’s infrastructure, as we have done below for Paris.
![rand6]()
Not only does it give us an indication of how many people transit in each station, it also give us the route they have been taking, where they changed train or if they walked between stations. Combining this with data from the city — concerts, office and residential buildings, population demographics, … — enables you to see how each factor impacts public transport, and even predict how many people will be boarding trains throughout the day. It can then be used to notify commuters that they should take a different train if they want to sit on their way home, and dynamically adjust the train schedules, maximizing the efficiency of the network both in terms of energy saved and comfort.
And it’s not just public transport. The same model and data can be used to predict queues in post offices, by taking into account hyperlocal factors such as when the welfare checks are being paid, the bank holidays, the proximity of other post offices and the staff strikes. This is shown below, where the blue curve is the real load, and the orange one is the predicted load.
![rand7]()
This model can be used to notify people of the best time to drop and pickup their parcels, which results in better yield management and customer service. It can also be used to plan the construction of new post offices, by sizing them accordingly. And since a post office is just a retail store, everything that works here can work for all retailers: grocery stores, supermarkets, shoe shops, etc.. It could then be plugged into our devices, enabling them to optimize our shopping schedule and make sure we never queue again!
This contextual modeling approach is in fact so powerful that it can even predict the risk of car accidents just by looking at features such as the street topologies, the proximity of bars that just closed, the road surface or the weather. Since these features are generalizable throughout the city, we can make predictions even in places where there was never a car accident!
![rand8]()
For example here, we can see that our model correctly detects Trafalgar square as being dangerous, even though nowhere did we explicitly say so. It discovered it automatically from the data itself. It was even able to identify the impact of cultural events, such as St Patrick’s day or New Year’s Eve! How cool would it be if our self-driving cars could take this into account?
If we combine all these different layers — personal, social, environmental — we can recreate a highly contextualized timeline of what we have been doing throughout the day, which in turn enables us to predict what our intentions are.
![rand9]()
Making our devices able to figure out our current context and predict our intentions is the key to building truly intelligent products. With that in mind, our team has been prototyping a new kind of smartphone interface, one that leverages this contextual intelligence to anticipate which services and apps are needed at any given time, linking directly to the relevant content inside them. It’s not yet perfect, but it’s a first step towards our long term vision — and it certainly saves a lot of time, swipes and taps!
One thing in particular that we are really proud of is that we were able to build privacy by design (full post coming soon!). It is a tremendous engineering challenge, but we are now running all our algorithms directly on the device. Whether it’s the machine learning classifiers, the signal processing, the natural language processing or the email mining, they are all confined to our smartphones, and never uploaded to our servers. Basically, it means we can now harness the full power of A.I. without compromising our privacy, something that has never been achieved before.
It’s important to understand that this is not just about building some cool tech or the next viral app. Nor is it about making our future look like a science-fiction movie. It’s actually about making technology disappear into the background, so that we can regain the freedom to spend quality time with the people we care about.
This post Came from Rand Hindi.
SEE ALSO: Why advertisers and publishers are in an uproar over Apple's latest move
Join the conversation about this story »
NOW WATCH: How Elon Musk can tell if job applicants are lying about their experience
This week, Google 
Elon Musk, CEO of Tesla and SpaceX, doesn’t trust artificial intelligence, which he once likened to 
A robot killed a factory worker at a Volkswagen plant in Germany, 
asked about how the world will look from a technology and social media perspective, Zuckerberg answered, "We're working on VR because I think it's the next major computing and communication platform after phones...I think we'll also have glasses on our faces that can help us out throughout the day and give us the ability to share our experiences with those we love in completely immersive and new ways that aren't possible today."
When the body's immune system senses an implant, the brain's defense mechanism creates scar tissue around it to protect the brain. When a probe becomes too engulfed in glial scarring, it loses functionality.
An intrepid user can emphasize particular features in an image, by running it through the network or even a single layer multiple times.
With Winiger's help, I was able to test the network on a photo of myself drinking tea in an antique shop.
This lower level on the AI network seems to be primed to search for holes and eyes, inadvertently adding dog faces in the background.
While this image produced by an upper layer looked for faces, pagodas, and birds. Notice the grumpy little man in what looks like a space suit appearing in the bottom right.
Winiger and Pieters both hope that the images from #deepdream will have people talking and learning about AI visual systems as they become more integrated into our daily lives.
But these are more than just computerized curiosities. The process that generated these images can actually tell us a great deal about how our own minds process and categorize images – and what it is we have that computers still lack in this regard.
Starting with random noise, Google's artificial neural network found some bananas.
This is similar to believing that cherries only occur atop ice cream sundaes. Because the neural network operates on correlation and probability (most dumbbells are going to be associated with arms), it lacks a capacity to distinguish contingency from necessity in forming stable concepts.
A stable classification mechanism so far eludes deep learning networks. As described by the researchers at Google:
It also enables you to communicate about abstract objects, despite never having experienced them directly, such as 
"Think of the market size: 87 million knowledge workers in the US alone schedule upwards of 10 billion meetings a year ... you should think of X.ai. as similar to Dropbox, but for meeting scheduling. They save files for 400 million users — we want to schedule meetings for 400 million users. They have a free edition, so will we, they have a pro edition for $9, so will we," Mortensen said.
It might not sound like a revolutionary undertaking. A young child can describe the pictures above easily enough.
"As the machine develops one word after another in the English sentence, for each word, it chooses to look in different places of the image that it finds to be most relevant in deciding what the next word should be," Bengio said. "We can visualize that. We can see for each word in the sentence where it's looking in the image. It's looking where you would imagine ... just like a human would."
It did however, correctly predict what every man and woman wants: to sit at a table with a large pizza.
