Thought Leaders in Big Data: Interview with Michael Wu, Chief Scientist at Lithium (Part 4)

Sraman Mitra: So you are anticipating user queries and caching them into intermediary steps.

Michael Wu: We are not really anticipating. Query time is one thing, but there is also actionability. How are you going to look at what every single user in our community does? We have millions of users in our community, and each of those users performs thousands of actions over the years. People need to aggregate around that and summarize it in some way. A way of summarizing that data is a framework I developed, which basically allows the NoSQL solution to aggregate very quickly around who did what, when and where.

SM: What technologies do you use in terms of visualization?

MW: Right now we are using Highcharts. We are also experimenting with D3. We built dashboards or widgets that people can essentially lower into their dashboard. We predefined a widget that contains visualization, whether it is a pie chart, a line chart, or more advanced visualization such as tree map or more hierarchical mapping. It depends on the type of data. If you want to look at interaction data, it is really nice to look at a graph. If you look at hierarchical data, it is very nice to see it as a tree map. Our community is hierarchical. There is the community that is at the top level and then within the community there are different areas. AT&T, for example, is one of our clients. Some areas are related to their DSL service, then there is another area for their mobile questions and more. Each one of those areas has a pinnacle question area. Another one addresses billing problems. We also do a lot of text analytics to pull keywords out to help identify topics or trends, so you can know what is going on. In essence that is what I mean by aggregation. We aggregate this data and summarize it. In the case I just described, we are summarizing where people participate most heavily in. In the previous example, you aggregate along the user dimension.

SM: What do you see as a major trend that is impacting your space?

MW: With respect to big data, mobile is a huge area because mobile data is personally relevant. Our mobile devices are with us pretty much all the time. We carry them wherever we go. A mobile device can record data that gives us a sense of the environment on which we can focus. So, right now we already track each action that the user took. But we have some context under which they took this action. However, we don´t have data on the environment under which they took the action.

One of our clients is Logitech, and a user may be asking, “Which type of mouse or keyboard should I get?” If we had mobile data, we would know about the environment they are actually in – a shopping mall, for example. Or when someone answers a question, we can know if they are sitting at home or just waiting online to buy something. We have a lot of information on the personal context and the environment in which a person takes an action. This can potentially answer the question of why this person is asking this question or taking a particular action.

This segment is part 4 in the series : Thought Leaders in Big Data: Interview with Michael Wu, Chief Scientist at Lithium
1 2 3 4 5