The data spiral

The data spiral - a service gathers a data set essential to delivering a newer, higher order service that in turns gathers an even richer data set that continues the upward spiral

The data spiral - a service gathers a data set essential to delivering a newer, higher order service that in turns gathers an even richer data set that continues the upward spiral

Google Earth was great for seeing your house from space. We all had a lot of fun with that. It also turns out to be one of the most important things Google ever did and set them on a path to greatness.

Why mention Google Earth, a 15 year-old technology, in a blog about the future? Because it's a terrific starting point to illustrate the concept of the data spiral. The data spiral is the modern equivalent of Moore's Law, and everyone in business needs to understand it. Because whoever has the data, wins.

Google originally purchased the technology that became Google Earth from a small company called Keyhole Inc. Similarly, Google Maps came from the purchase of a company called Where 2. Boy oh boy, did Google get a bargain. Let me explain.


The data spiral is a lot like the Moore's Law of old. Engineers have kept Moore's Law going for over half a century by using the chips of the day to power computers that helped them design the chips of tomorrow. These new, faster, cheaper chips were then used to create the next generation after that.

This self-sustaining loop of increasing computing performance benefited us all. New fast chips let us all run ever more complex and demanding software. And that new software then created a vibrant market for faster and faster hardware. And so the world turned.

A new equivalent to Moore's Law has recently emerged. This time it's all about data.

Here's how it works: A software product is created as a way to provide some kind of service, but also to collect and store a new data set. This data set is then used to deliver a new product or service, one that may not have been possible before. This new service in turn gathers an entirely new data set that wasn't possible to gather before. And so on and so on.


Google's Waze service aggregates the location and speed data of drivers using the service to build an overall picture of current traffic flow

Google's Waze service aggregates the location and speed data of drivers using the service to build an overall picture of current traffic flow

Google does this all the time. Services like Google Maps and Google Earth rely on a detailed data set of global geography and feature maps. Google uses these data sets to deliver services that let them gather data about you. It then uses that data to make money and develop even better new services. For example, Google logs all the searches you do on Google Maps, tracks your location, and knows your speed of movement. By studying where you are most evenings Google figures out where you live. Similarly it has a pretty good idea where you work. It even knows where you buy your groceries and where your kids go to school. It builds a detailed picture of you as a consumer (which is another juicy data set) that it can then sell to advertisers in the form of advertising services. Google aggregates all that location and speed data to build a traffic data set, which is how navigation services like Google's Waze work. Google wouldn't be able to do any of this without the underlying data set of maps. Google knows that if they invest in creating the right data sets (for example, consider all the effort they put into building images for Google Streetmaps) they will be able to use it to gather even more valuable data sets in the future.

UPS and Fedex optimize their delivery driving routes (saving time and fuel) using data from navigation services. UPS estimate their new ORION routing system will save them 10 million gallons of gas and reduce the distance their drivers travel by 100 million miles annually by the end of the year. 

Uber and Lyft rely on navigation and Google traffic services to create their value (and gather passenger data). Google's self-driving test cars use existing navigation data to drive around and gather even more detailed data of streets and environments. Tesla's cars, bristling with modern sensors, are building highly detailed maps of the road network as their drivers zoom around the streets in them. All that data is used as an input to improve its own autonomous driving systems.


Google's Google Now personal assistant service builds on Google's understanding of your habits to anticipate what you will do next. That allows them to make smart recommendations and target you with offers in the moment. And as your comfort grows with Google Now and you see how valuable it can be to you, you are increasingly more likely to offer up even more personal information and give Google Now access to your travel plans, your email inbox, and more. The data spiral accelerates and even more data is gathered, processed and stored.

YouTube is another huge source of valuable data for Google. Teaching a computer to "see" is vital to the future of robotics and autonomous machines. By creating a service that enables hundreds of millions of people to share billions of hours of video, Google has built up a gigantic video data set. This treasure trove of videos enables them to build visual recognition algorithms that do an excellent job of understanding scenes, objects, and context.  The video data is used as a training set for a new class of deep neural networks able to not just understand what's in a scene, but assess it on aesthetic grounds. Google research has described how they use deep learning techniques to find optimal thumbnails in YouTube videos.

Understanding visual, audio, and other sensory input is a key capability for the future of computers. Expect that even more impressive services (that will of course gather even more data) will be built on top of this ever-improving recognition capability. 


Visual recognition is just one component of the research going on in the field of artificial intelligence. The data spiral is vital to the development of these new artificial intelligence platforms. More data sets lead to more insights, and more data.

The new Moore's Law is the data spiral. The companies that embrace the data spiral in the coming decades will do just as well as those that rode Moore's Law through the 80s, 90s and 2000s. Invest and plan accordingly.

Whoever has the data, wins.