We have been a misunderstood and badly mocked org for a long time. Like when we started... We announced the org at the end of 2015. We were going to work on AGI. People thought we were batshit insane. I remember at the time, an eminent AI scientist at a large industrial AI lab was. DMing individual reporters being like, "These people aren't very good, and it's ridiculous to talk about AGI. I can't believe you're here.""I can't believe you're given them time of day." And it's like, that was the level of like, pettiness and ranker in the field that a new group of people say, "I'm gonna try to build AGI." So open AI and Deep Mind was a small collection of folks who were brave enough to talk about AGI. In the face of mockery. We don't get mocked as much now.. The following is a conversation with Sam Altman, CEO of Open AI, the company. ehind GPT-4, Chat GPT, Dolly, Codex, and many other AI technologies which both individually. And together constitute some of the greatest breakthroughs in the history of artificial intelligence, computing, and humanity. Please allow me to say a few words about the possibilities and the dangers of AI in this current moment. In the history of human civilization, I believe it is a critical moment. We stand on the precipice of fundamental societal transformation. We're soon. Nobody knows when, but many including me believe it's within our lifetime. The collective. telligence of the human species begins to pale in comparison by many orders of magnitude to the general system. The general super intelligence in the AI systems we build and deploy at scale. This is both exciting and terrifying. It is exciting because of the innumerable applications we know. nd don't yet know that we'll empower humans to create, to flourish, to escape the widespread power. The poverty and suffering that exists in the world today, and to succeed in that old, all-too-human pursuit of human beings. It is terrifying because of the power that super-intelligent AGI wields to destroy. uman civilization, intentionally or unintentionally. The power to suffocate the human spirit. In the totalitarian way of George Orwell's 1984 or the pleasure-fueled mass hysteria. The new world, where as Huxley saw it, people come to love their oppression, to adore the technology. They adore the technologies that undo their capacities to think. That is why these conversations would. With the leaders, engineers, and philosophers, both optimists and cynics is important now. These are not merely technical conversations about AI. These are conversations about power, about companies, institutions, and political systems. Those systems that deploy, check, and balance this power, about distributed economic systems that incentivize safety. And human alignment of this power, about the psychology of the engineers and leaders that deploy AGI and. about the history of human nature, our capacity for good and evil at scale. I'm deeply honored to have gotten to know and spoken with on and off the mic with many folks who. now work at Open AI, including Sam Altman, Greg Brockman, Ilya Sutskever, Wojciech Zaremba, Andrej Karpathy, Jakub Pachocki, and many others. It means the world that Sam has been told. It has been totally open with me, willing to have multiple conversations, including challenging ones, on and off the mic. I will continue to have these conversations to both celebrate the incredible accomplishments of the AI community and the steel-made community. The steel-man, the critical perspective on major decisions various companies and leaders make. Always with the goal of trying to. help in my small way. If I fail, I will work hard to improve. I love you all. This is the Lex Fridman podcast. Disappointed. Please check out our sponsors in the description. And now, dear friends. Here's Sam Altman. It's a system that we'll look back at and say it was a very early AI and it's slow. It's buggy. It doesn't do a lot of things very well. But neither did the very earliest computers. And they still pointed a path to something that was going to be really important in our lives, even though it took a few decades to evolve. Do you think this is a pivotal moment? Like out of all the versions of GPT 50 years from now, when they look back at an early. Yeah, that was really kind of a leap, you know, in a Wikipedia page about the history of artificial intelligence, which is a. Which are the GPTs with Ip GOT? That is a good question. I sort of think of progress as this continual exponential. It's not like we could say. ere was the moment where AI went from not happening to happening. And I'd have a very hard time pinpointing a single. I think it's a very continual curve. Will the history books write about GPT-1 or 2 or 3 or 4 or 7? That's for them to be. ide. I don't really know. I think if I had to pick some moment from what we've seen so far, I'd. sort of pick Chat GPT. You know, it wasn't the underlying model that mattered. It was the usability of it, both the RLHF and the interface to it. What is Chat GPT? What is RLHF? Reinforcement and learning with human feedback. What is that little magic? So much more delicious. These models are on a lot of text data, and in that process they learn the underlying something about the underlying. Underline representations of what's in here are in there and they can do amazing things but when you first. lay with that base model that we call it after you finish training it can do very well on Evals, it can pass tests, it can do a lot of. There's knowledge in there, but it's not very useful, or at least it's not easy to use, let's say. And RLHF is how we take some human feedback. The simplest version of this is "Show two outputs", ask which one is better than the other. Which one the human raiders prefer. And then feed that back into the model with reinforcement learning. And that process is a good idea. The process works remarkably well within my opinion, remarkably little data to make the model more useful. So RLHF is how we align the model to what humans want it to do. So there's a giant language model that's. A giant data set to create this kind of background wisdom knowledge that's contained within the internet. And then... Somehow adding a little bit of human guidance on top of it through this process makes it... so much more awesome. Maybe just because it's much easier to use. It's much easier to get what you want. You get it right more often than... d ease of use matters a lot, even if the base capability was there before. And like a feeling like a... Understood the question you're asking or like it feels like you're kind of on the same page. It's trying to help you. It's the feeling of alignment. Yes. I mean that could be a more technical term for it. And you're saying that not much data is required for that. Not much human supervision is required for that. To be fair, we understand the science of this part at a much... Earlier stage than we do the science of creating these large pre-trained models in the first place, but yes, less data. Much less data. That's so interesting. This side... The science of human guidance. That's a very interesting science. That's going to be... ery important science to understand how to make it usable, how to make it wise, how to make it ethical. How to make it a line in terms of all the kind of stuff we think about. And it matters which are the. umans and what is the process of incorporating that human feedback and what are you asking the humans? Is it two things that you're asking them to rank things? What aspects are you asking? he humans to focus in on? It's really fascinating. How, what is the dataset it's trained on? Can you kind of loosely speak to the enormity of this dataset? The pre-trained dataset at Polys. We spend a huge amount of effort pulling that together from many different sources. There's like a lot of... If there are open source databases of information, we get stuff via partnerships, there's things on the internet. It's a lot of our work is building a great dataset. How much of it is the memes subreddit? Not very much. Maybe it'd be more fun if it weren't more. Some of it is Reddit, some of it is news sources, all a huge number of news, paper, etc. There's a lot of content in the world, more than I think most people think. Like, too much. Like, where the task is not to find stuff, but to filter outs. Yeah, right. Yeah. What is there a magic to that? Because I think there's, excuse me, several components to solve the, uh, the design of the, uh, you could say, algorithms, so like, the architecture, then you own that works, maybe the size, then you own that work. There's the selection of the data. There's the human supervised aspect of it with, you know, RL with human feedback. Yeah, I think one thing that is not that well understood about creation of this final product, like what it takes to make GBT-4. The version of it we actually ship out that you go to use inside of Chat GPT. The number of pieces that have to all come together. And then we have to figure out either new ideas or just executing existing ideas really well at every stage of this pipeline. There's quite a lot that goes into it. So there's a lot of problems, like you've already said for GPT-4 in the blog post. And in general, there's already kind of a maturity that's happening on some of these steps. Like being able to. edict before doing the full training of how the model will behave. Isn't that so? Markable, by the way, that there's like, you know... There's like a law of science that lets you predict for these inputs, here's what's gonna come out the other end. Like, here's the level of intelligence you can agree with. Is it close to science or is it still... because you said the word "law" in science. Which are very ambitious terms close to us. Close to us, right? I... be accurate, yes. I'll say it's way more scientific than I ever would be. I ever would have dared to imagine. So you can really know the peculiar characteristics of the fully-trained. You know, like any new branch of science, there's, we're gonna discover new things that don't fit the data and have to come up with the data. We have to come up with better explanations and, you know, that is the ongoing process of discovery and science. But with what we know now, even when we hadn't had. We had in that GPT-4 blog post, I think we should all just be in awe of how amazing it is that we can even predict to this current level. Yeah, you can look at a one-year-old baby and predict how it's going to do on the SATs. I don't know. Similarly, in equivalent one, but because here we can actually, in detail, introspect various aspects of the system you can predict. That's. That said, just to jump around, you said the language model, there's GPT-4. It learns in quotes. Something. In terms of science and art and so on, is there within Open AI within like. Folks like yourself and Ilya Sutskever and the engineers, a deeper and deeper understanding of what that something is? Or is it still a kind of beautiful, magical mystery? Well, there's all these different emails. hat we could talk about. And what's an email? Oh, like how we measure a model as we're training it at. After we've trained it and say like, you know, how good is this at some set of tasks? And also just a small tangent. Thank you for opening, sourcing the... Evaluation process. Yeah, I think that'll be really helpful. But the one that really matters is... We pour all of this effort and money and time into this thing, and then what it comes out with, like how useful... How useful is that to people? How much delight does that bring people? How much does that help them create a much better world? New science, new products, new services. And that's the one that matters. And understanding for a particular set of inputs. Like how much value and utility to provide to people. I think we are understanding that better. Do we understand everything about why the model does one thing and not one other thing? Certainly not always. But I would say we are pushing back like the fog of war more and more and we are... You know, it took a lot of understanding to make GPT-4, for example. But I'm not even sure we can ever fully understand. Like you said, you would understand by asking questions essentially. It's compressing all of the web. Like a huge loss of the web. Into a small number of parameters. Into one organized black box that is human wisdom. What does that human knowledge let's say? Human knowledge. It's a good difference. Is there a difference? Is there a knowledge? There's facts and there's wisdom and I feel like GPT-4 can be also full of wisdom. What's the... What's the leap from faster with the... You know, funny thing about the way we're training these models is I suspect too much of... Of the processing power for lack of a better word is going into using the models of database instead of using the model. The thing that's really amazing about this system is that for some definition of reasoning, we could of course quibble about it. There's plenty for which definitions this wouldn't be accurate, but for some definition, it can do some kind of reasoning. And you know, maybe you're not going to be able to do that. And you know, maybe like the scholars and the experts and like the armchair quarterbacks on Twitter would say, "No, it can't, you're misusing the word, you know, whatever." But I think most people who have used this system would say, "Okay, it's doing something in this direction."AEGIS-OBSIDIAN