1Mby1M Virtual Accelerator AI Investor Forum: With Venktesh Shukla, Founder and Managing Partner at Monta Vista Capital (Part 1)

Venktesh Shukla, Founder and Managing Partner at Monta Vista Capital, discusses why he wants to see Human-in-the-Loop Generative AI startups. We’re hearing this a lot right now.

Sramana Mitra: Venk is a very experienced investor in Silicon Valley, and we’re going to discuss his thoughts on what’s happening in the AI universe – the trends, how he is thinking about investing in AI? Where is the market going? Where is the startup market going? What are the different trends that inform our decisions as entrepreneurs and investors to work in this field?

Welcome, Venk, it’s great to have you.

Venktesh Shukla: Thanks, Sramana.

Sramana Mitra: So, Venk, as you look at the AI universe, how are you parsing all the different developments? It’s all coming at us very fast, obviously.

What are your thoughts?

Venktesh Shukla: It’s interesting that you say it’s coming very fast. In fact, when it comes to generative AI, it’s not very fast. After ChatGPT came out in November 2022, I had expected the pace of innovation to be a lot higher.

We are still stuck in the same underlying technology constraint that transformer architecture uses, which is probabilistic and full of hallucination.

I had expected that by this time, there would be some breakthrough that would take us away from hallucination. That’s what limits its use in enterprise. In enterprise, you cannot have probabilistic answers or wrong answers. You must get it right.

There are some enterprise applications where hallucination or probabilistic answers are okay – if you’re using it as a natural language front end, then it doesn’t matter if the words used are different or the expression is different and if you don’t get the semantics exactly right. That context is okay. But if you are saying that a sales guy exceeded the quota in the last two quarters, you cannot have hallucination there.

Sramana Mitra: Yes, the hallucination remains a big roadblock for sure. We’ve probably done 15 such conversations already with the investors and discussed some what’s happening in their portfolios. One theme that is coming up is the theme of small language models to constrain hallucinations.

In many of the vertical AI solutions, you don’t really need the full power of the large language model. It’s good to have the large language model capabilities to be able to tackle a natural language, but when it comes to domain specific training, a relatively small domain specific knowledge would be relevant for a particular vertical and a particular customer. That may be sufficient and may be a very good way to contain hallucination. Of course, there are architecture issues.

Is this something that is coming up in your portfolio or in your conversations?

Venktesh Shukla: Yes. We have some deep technologists on our team. These are the guys that used to probably get millions of dollars in salaries at Google. Two such guys are on our team. We get deep into technology when companies present to us. It’s obvious that even with the small language model thing, if the underlying architecture is the same transformer architecture, the hallucination issue doesn’t entirely go away.

I think the only way to get over it is to have a human in the loop. Otherwise, it really does not go away. The benefit of having a small language model really is the cost. The cost of training and cost of inferences go down a lot. But it doesn’t get around the hallucination issue.

Sramana Mitra: So you’re seeing small language models are hallucinating.

Venktesh Shukla: It has to. It is the same transformer architecture, which is based on probability.

Sramana Mitra: The human in the loop theme is also coming up a lot in our discussions, and some people are trying to get around the problem by putting a human in the loop. In some cases, it may work, but if you’re really working with very big data. Can human in the loop solve the hallucination problem?

Venktesh Shukla: Well, it’s better than not having a human in the loop. You don’t know if it’s hallucinating or not. If you have a huge data set of 100 sales guys all over the world and want to know which one has given better than quota performance in the last two quarters, if it comes up with a name, some human can immediately verify if this individual did indeed exceed the quota in the last two months. That’s how you verify.

This segment is part 1 in the series : 1Mby1M Virtual Accelerator AI Investor Forum: With Venktesh Shukla, Founder and Managing Partner at Monta Vista Capital
1 2 3 4