ΑΙhub.org
AIhub coffee corner: Is it the end of GenAI hype?
The AIhub coffee corner captures the musings of AI experts over a short conversation. There has been a string of articles recently about the end of generative AI hype. Our experts consider whether or not the bubble has burst. Joining the conversation this time are: Tom Dietterich (Oregon State University), Sabine Hauert (University of Bristol), Michael Littman (Brown University), and Marija Slavkovik (University of Bergen).
Sabine Hauert: There have been a number of recent articles in the mainstream media talking about the fact that AI has not made any money, and that it might be all hype, or a bubble. What are your thoughts?
Marija Slavkovik: There is this article by Cory Doctorow which asks what kind of bubble AI is. I really like his take that a lot of bubbles come and go; some of them leave us something useful and some of them just generate something for a brief moment in time, like excellent revenue for the investment bankers for example. It’s a wave, right? It hits different targets at different times. I’m pretty sure there are areas in which this wave is just hitting now, and there are areas in which it has passed. I mean, it’s a very business question. A lot of research I saw in IJCAI this year, and I expect it’s going to be the same for ECAI, it’s going to be LLMs for this and that. So, I wouldn’t say it has gone away. A lot of researchers are doing fun little projects around LLMs. We also haven’t seen the grants that involve LLMs. That process takes one year, so I think we will see what has been awarded in the grant schemes next cycle. I expect to see a lot of LLM stuff in there. So, the question is for whom [has the bubble burst]? For journalistic interest, maybe.
Tom Dietterich: The storytelling for all these things is always the new wave, then the anti-wave, then the anti-anti-wave, so the anti-AI hype is rising now. And obviously there are partisans who find that useful, so they’re pushing that narrative. There’s a famous thing called the tire track diagram that shows how various research and application trajectories have often oscillated back and forth between government and industry funding before they had a big impact in the marketplace. It seems that it’s usually 20 years between the time of an initial advance on the research side and the time you really see big success in the private sector. And yet the research may have bounced back and forth a couple of times during that process. I think, particularly with LLMs, I don’t know that we really know what the killer app is yet. That’s the question everybody’s been asking – what will LLMs really be useful for? I think a lot of companies are still actively engaged in trying to figure that out. But realistically, we should probably expect another nine years, maybe 19 years, I don’t know. It wouldn’t surprise me if it takes us a long time to figure out, particularly because this technology has a different profile than anything we’ve had before. It’s got this incredible breadth, but shallow understanding. I mean it just has all these different properties that we’ve never dealt with before. I attended a workshop that was run by Princeton which had several people, both from academia and industry, talking about programming frameworks for how you write systems using LLMs as components. That was quite interesting to see the kinds of things they’re doing. But, again, you get the feeling that these platforms are still academic curiosities, more or less. But we don’t know what’s really happening inside the companies, except knowing that, for instance, Llama2 7B has been downloaded 250,000 times or something. So, everybody’s playing with these things.
Sabine: Maria was mentioning that we’ll see what grants get funded. I guess, Michael, the question is, do you still invest in things when you know the hype is going down or is that when industry should take it up? How do you nurture that, basically?
Michael Littman: I feel like there’s a difference of time scale, and this came up in Tom’s response as well, between the academic “digestive system” and business. Business is buy low, sell high, but it turns out you can also make money from buy high, sell low, if you short things the right way. But there’s no such thing as buy medium, sell medium—they only make money when things are changing. So there has to be a story that things are changing and that’s really important to them. If they have to make money every quarter, there has to be a change faster than that. Whereas, in academia, we’re absorbing these ideas, we’re taking them on board. If you look at the conferences, if you look at the grant proposals, the field has very much embraced this and they’re doing, in many ways, the due diligence of trying to figure out what really is going on here. What’s the science here? What’s the capability that we didn’t have before that we do now? How do we strip away all the excess stuff and really get at the nub of the idea (because that’s what lets us apply it broadly and generally)?
What I’m seeing is that it’s great to see what businesses are investing in, but, in many cases, the grant systems need to be a little bit counter to the business cycle because it’s when the companies are off running around looking at something else that it’s really important for the researchers building those foundations to really explore those ideas. Certainly at the National Science Foundation, the director often will talk about the idea that NSF continued to fund artificial intelligence even when it was technically an AI winter. The companies were like “we’re not going to touch this with a 10-foot pole”. However, there was continued support from various government agencies, I’m sure in other countries as well. So yeah, I don’t think this necessarily changes where we are as researchers, except for those researchers who have dual hats and, in addition to doing research, are also trying to make money in business. But, for the field as a whole, I’m not seeing the downswing, I’m seeing a nice, steady upswing of people really grappling with what the issues are.
Sabine: It’s good to see the positivity and the continuity of that funding. What do you think is causing these noises from the industry side? Is it that we’ve promised too much? Is it that they’re seeing the reality of what this technology can do now, or is it just generally positive still?
Marija: Completely uneducated opinion, and I’m guessing, but Amazon sank $4 billion into Anthropic, and it still has not done better than GPT-4. If I was going to guess what the jitteriness is, $4 billion is a lot of money and it hasn’t bought better performance, or, at least, we haven’t seen anything in the media about this.
Tom: Right, we’ve now seen several models that match GPT-4, more or less, but we haven’t seen a step increase the way we did between GPT-3 and GPT-4. Speculation is rampant, but absence of evidence is not evidence of absence.
One question I had for you, Marija, in terms of the waves cresting, in machine learning, computer vision, natural language, it’s definitely been a huge wave. Where are we in, say, robotics and multi agent systems?
Marija: People are adopting LLMs to do little tasks, so they’re going hybrid. So you use a little bit of LLM somewhere and then you incorporate it with business as usual. So, it’s interesting.
Tom: One of my colleagues is Jonathan Hurst, who has Agility Robotics, and he’s watching other robotic startups raise a lot of money just by putting LLM on top of their robot and be able to say, you know, “pick up the red cup”. Basically, we’re back to SHRDLU, but with an actual robot. He feels that’s ridiculous because those people are raising three or four times as much money as he’s raising, but their robots aren’t as good in his view. So, there is a bit of that salesmanship happening with these startups.
Sabine: The robotics grounds things, right, because you need these things to deliver on a specific task. So, rather than going full general, you’re like, “what can this thing be useful for in this little corner of the world that I operate with in robotics”? I think that could be useful for a lot of the real-world applications of LLMs. I think they’ve gone too much for “it can do everything”, but it can be very useful in very specific contexts when it’s framed better. And so maybe that’s something robotics can offer – a bit of the grounding.
Tom: And in the meantime, we’re seeing the ideas behind diffusion models showing up all over the place. And those may turn out to be equally interesting intellectually. For example, people are using diffusion models to represent policies for controllers. There’s also all this work in molecular design, materials design, architectural design. A lot of that’s all based on similar ideas and, if I were betting where the big money will be made, I think it’ll be made in atoms. So, biology and materials science more than in language. With language it’s a sort of universal interface, so it’ll be an interface layer, but the huge orders of magnitude improvements I think are coming from these applications in fluid dynamics, so weather and molecular dynamics simulations. These models are letting people do things at time scales that are a couple orders of magnitude longer than were possible before. There are huge advances there, but they’re kind of under the radar of the press.
Michael: Sounds like you’re optimistic about the use of the Adam optimizer as an atom optimizer.
Tom: Ha! Something like that.
Sabine: Are there any final thoughts?
Michael: A colleague of mine, Eugene Charniak, retired from the department a couple of years ago and then started, in retirement, working on a book, but then unfortunately died. But he had basically finished the book, so I worked with the publisher, MIT Press, to get the book ready to go, and it should be coming out in October. The premise of the book is essentially: now that we have these large AI models trained by machine learning, the field of AI can finally begin. He was very much viewing this moment not as the end of our story, but really like everything up to this point was just prologue, and now the story can actually start. I think that’s a really exciting way of looking at things. I’m not sure I 100% agree with it because I really like a lot of the stuff that’s already happened. But it’s a really interesting perspective that says maybe we have a new conceptual framework for thinking about all the things that we have addressed in AI—learning, planning, decision-making, reasoning. All these things could potentially have a new foundation, and that’s only going to happen if the researchers buckle down and figure out what this thing is and how we put the elements together. And so, I think it’s super exciting.
Marija: I’m just fascinated by the people’s will to believe, right? I mean, like the will to believe that we have reasoning, human-speaking computers out there. It’s just off the charts. I expect it from people who don’t do anything computer sciency. But then I sit at tables with people who understand some computer science and then they go “but it reasons”. So, I would say it ain’t over until it actually reasons.
Sabine: And I’m going to plug us [AIhub] because I think that positive trajectory saying that this is just the start of things, and we need to build on it, we need years of funding to actually reach the real applications that have breakthroughs, it also means we need to communicate our research better. I think that there are inflated expectations about what the technology can do, because of all the things that Marija is saying. And maybe that’s the role we play – grounding a little bit of that.
tags: AAAI, coffee corner
AIhub is supported by:
