Thought Leaders in Big Data: Franz Aman, Chief Marketing Officer, Silicon Graphics (Part 4)

Sramana Mitra: So, you are avoiding that problem by blowing out the main memory available and then doing the in-memory computing on top of that?

Franz Aman: Correct. Then it is native. It is just available, and your application will just run. We recently did a project with a researcher from the University of Illinois about a Twitter analysis. That was one of these big brain systems where we needed one big, continuous memory space to do some of the analysis in real-time at the rate at which we did it. That is the only way you can do something like that. A lot of times I talk with people about 64 terabytes of space, and they think I have the sizes wrong. They ask if I meant gigabytes and not terabytes, or they think it is disk space and not main memory, but it actually is main memory.

Another way of going about it is the use of scale-out technology and clusters, but then you have many systems. The problem is that you are limited to a maximum of a couple of terabytes, and you have to think about how to break the problem down and manage it across a scale-out system. There are interesting approaches in terms of NoSQL and Hadoop. We are using both in a complementary fashion. If you want to have the first view, don’t try to do it across a couple of notes. If you want to digest massive amounts of data, use a scale-out approach. That is most likely the only way to deal with the petabytes of data you may need to look at. Using the right combination of the two is guaranteed to give you the better answer.

SM: What else do you consider to be interesting that we should cover in this interview?

FA: The other thing we see is that everything is going toward real time and right away. Since we talked about the Twitter analysis, one of the things I find fascinating is that we are increasingly becoming a real-time society. I can launch a marketing message and get 100 answers through Twitter in the next 10 seconds. This opens up a whole new real-time market and lifestyle. I don’t think we are even close to leveraging and embracing that the way we should.

Thinking back, there was a book from [Yale professor] Ian Ayres about number crunches. The fundamental premise was that with the computing capacity today, we shouldn’t limit ourselves to trying the next logical thing or trying a few things that we can think of. Why not go rail to rail? Why not try random and wild things and explore what happens? I think this real life and real-time experimentation is something that is absolutely possible today, yet we limit ourselves to what we can think of. We should embrace going rail to rail much more radically. We should experiment and see what happens from a data perspective. [Even with] all the big data discussion, that isn’t happening yet: how to do business, how to do marketing, and how to approach markets. With the TweetBeep analysis [on Twitter alerts] we showed that you can get at people’s emotions and feelings in real time, using Twitter, for example. If you look at people’s hopes and fears on a global basis in real time, it opens up so many opportunities.

This segment is part 4 in the series : Thought Leaders in Big Data: Franz Aman, Chief Marketing Officer, Silicon Graphics
1 2 3 4 5