Thought Leaders in Cloud Computing: Ari Zilka, CTO of TerraCotta (Part 4)

SM: There are two trends that we are discussing in this conversation. One is the real-time trend. The other is the big-data trend. So, I would like to do a thought experiment with you that takes both of those into account and use a social media construct on the use case.

Let’s say we are looking at the realm of consumer marketing. Some company … let’s say Amazon is marketing to consumers, and there is a ton of stuff going on in social media around products that Amazon sells. The customers of Amazon are conceivably discussing products that they might be buying and so on and so forth. What, in your opinion, is the future of marketing from the standpoint of an Amazon that wants to tap into all that information that is out there and react to it?

AZ: There’re two ways to answer it, the conceptual future and the tactical future. In the tactical future, they are all flying towards one direction and that’s called Hadoop. I’m sure you’ve spoken to other entrepreneurs who have brought up Hadoop already, but Hadoop is the open source, free manifestation of Google’s original search algorithm. It’s actually guys who just started saying, “How does Google work?” Google published papers but never released source code. So, someone intrepid sat down and said, “I’m going to implement exactly what this paper describes.” He implemented. He ended up working at Yahoo. Long story short, Yahoo reinvented itself doing social media and targeted marketing, just like you’re asking, using Hadoop.

The name of the game is to take all of the Web data and index them, all the Web pages in the world. Crawl the Web, index it all, that’s step one.  Step two is to then look at people’s search activity, their click trails through your search engine and see what it is when they put in a search term like Britney Spears, where do they go? And where do they not go? If they end up somewhere, do they spend money? When they search for Britney Spears, do they then realize that they meant to search for Christina Aguilera or some crazy example like that? Companies go through these people’s activity on their systems and look for patterns. They sell that data. They leverage that data to make your user experience faster, more efficient, give you more relevant searches, give you relevant products when they see you typing in a product name directly. That is being plumbed up by these Web giants today using this open-source framework called Hadoop.

Major enterprises are actively pursuing a Hadoop-oriented strategy. The problem with Hadoop is that it’s batch-centric. It takes batches that traditional Oracle and IBM databases used to take hours and days and weeks to process and does them in minutes. But minutes are not good enough in the real-time world.

Strategically – back to your question of the confluence of big data and real time and a social media type use case – where is a company like Amazon going to bring together big data and real time? The answer is minutes are not going to be good enough anymore. We’re working with a major car manufacturer right now that if you drive a fancy German car – and most cars now have this – if the air bags deploy, the police get called. If you want to pay for concierge service, there could be a button that you press, and over a cellular network, someone will come over like On Star but not On Star itself. You can say, “I’m hungry. Where’s some food? I’m out of gas. Where do I get gas? I have a flat tire. Call AAA for me, please.” This car manufacturer has brilliant idea. I have a tiny system. We only get bothered by our cars when there’s an emergency. So, a few hundred times a second around the world is not a very high volume, very intense use case. But we want to become an application marketplace in the future. We want apps built to our cars’ computers so that we can provide data to Starbucks, to Shell gasoline stations, to Amazon.com, to BestBuy so that as you drive by these retailers, the retailers’ apps can come up and offer coupons if you pull in. Restaurants can tell you there’s seating available.

You could basically run a find-an-available-restaurant application in your car. Whereas there was a vision a few years ago of mobile devices becoming coupons and marketing engines that as you walked around, they would offer you coupons and discounts for the retailers you passed, now they’re saying make it a more active experience that won’t be annoying to the consumer. Make it so that the customer installs a restaurant app into his car, not just his phone. And as he’s driving down the street, the car can say, “This restaurant you’re passing has available seating. Do you want to book a table and park in front of it?” That’s pretty compelling because on the phone, you have a certain class of apps for the mobile user who’s phone based. In the car, there’s a slightly different angle on the applications. The confluence of real time and big data is exactly illustrated by this Terracotta customer in automotive saying, just like Amazon would say, “How do I bring the analysis of patterns and the information that’s available to me on the ‘net and in the social graph, and how do I find a way to deliver new value-added services to the consumer?”

You could imagine a restaurant app that says, I want to go eat at this restaurant I’m standing in front of. It’s not got any tables for two for two hours. So, I press find more like this that my friends enjoyed eating at. And so you bring social media and the explosion of big data together with geographic positioning, positioning data and the Internet and real-time analytics to tie it all together for a brave new consumer world.

This segment is part 4 in the series : Thought Leaders in Cloud Computing: Ari Zilka, CTO of TerraCotta
1 2 3 4 5 6 7