The Audacity of Tom Costello (Part 2)

SM: Did Anna go to Google after it went public?

TC: She went there before it went public. Google was in a total panic as Yahoo! had come out with a competitive product. Google was in a hyper phase of trying to make things better, which is a terrible thing to do. They got nothing done because they ran around like headless chickens. Larry and Sergey moved 12 people into their office because they thought it would be more efficient if everyone was in the same office, and Anna was one of them.

There were two ranking groups. One of them was doing web rankings, and she led the effort to build a bigger search engine. Google at the time was about 4 billion pages, and Microsoft was going to come out with a search engine that had 6 billion pages. Anna wrote the backend architecture for Google search, which was launched two years ago.

SM: What were you doing at this time?

TC: The kids had started school by then so I had a little more time on my hands. I went back to Stanford and taught a few courses. Teaching was fine but it was not my life, so I decided to do a startup. When I looked at the areas I could do a startup in, I decided to do it in search. That seemed to be a lucky field.

SM: What year was this?

TC: This was probably around 2006. There were two things I was interested in: one was scale and the other was being creative. One of the problems when working with information is that if you do a small product, nobody cares. If you index 100 million pages, nobody cares. You must do it at scale, which is one of the biggest challenges.

Google and Microsoft were paving over states to build enormous data centers. When Anna had to order memory for her projects at Google it was a $250 million order. Google had absolutely no controls. If you wanted to order it you just had to check with the engineers to make sure they wanted it.

If you are going to do a startup, you cannot get investors to hand you $1 billion. It is not going to happen, so I had to come up with a different way of doing it.

SM: How much was Anna sharing with you about Google’s architecture?

TC: She wasn’t. She would come home and tell me all about people. She would talk about the politics and gossip. She brings stress home, but not work. In terms of Google’s architecture, it may have been very good, but I did not have $250 million to spend on memory.

I came up with a different architecture but realized I would not be able to convince anybody that I was right because of the level of mathematics I used to validate it. There is nobody in this world who understands mathematics and has money to give out. They just do not go together. I went and got some money by taking it out of my wife’s checking account.

SM: What were your key differences in terms of architecture?

TC: Search engines work by having 10,000 machines. Each of those machines is a search engine for 1/10,000th of the web. When you get a request in, you send it out to all 10,000 machines, and each machine provides its top 10. A top ten is then generated from the list of top tens, and that is how the search results work. That means every machine is involved with every query.

I wanted to devise a way to do this so that each query would just go to one machine. Instead of having each machine focus on one particular section of the web, you have one machine be an expert on one subject. Then you only have to go to one machine for your query, which allows you to do things cheaply.

The trick with that is you need to work out what those concepts are. It is complicated, but when it works you have a big advantage. I went and got some machines and surfing space down in San Jose and crawled the web. I crawled 500 million pages and built a demo.

In 2006 I went to KP and met John Doerr. He told me what I needed was somebody who was really good at architecture. He told me there was a woman at Google who I should really try to get, and started describing Anna. I told him I was pretty sure I could get in contact with her. At the time she was on maternity leave. When she finished maternity leave she decided she would quit Google, and she helped raise money.

We actually took money from TugBoat Ventures. We then bought machines, crawled the web, and got a beta up, and then decided we needed to raise more money. We then did a Series B with Madron. After that we launched in July. We launched on an incredibly slow news day, and the following week Russia invaded Georgia. We got a huge amount of coverage. Of course there were bugs the first day, but overall it was good.

This segment is part 2 in the series : The Audacity of Tom Costello
1 2 3 4 5 6 7