Everyone is calling for safer AI. So what does that mean?

FLORA LICHTMAN: Hey, I’m Flora, and you’re listening to Science Friday. For the last few weeks, AI safety has dominated the news cycle, precipitated in part by an AI attack where a bunch of OpenAI’s agents teamed up and hacked another AI platform, Hugging Face, without humans knowing. Since then, multiple AI researchers have come forward to say how worried they are that this tech is unsafe.

Now, every day, it seems like there’s a new development. So how do we make sense of what’s happening in AI research right now? Today we’re going to focus on the engineering of AI safety. And we wanted to hear from experts outside of the hype machine of Silicon Valley, outside of the major AI companies– scientists who are doing the dirty work of figuring out what developing safer AI even means.

Here with us today is Dr. Andrea Lincoln, a computer science professor at Boston University and an advisory board member for the Alignment Project at the AI Security Institute. She studies how to mathematically understand the internal processes of AI models. And Dr. Vinod Vaikuntanathan– a cryptographer at MIT and a founding member of the Institute for Responsible Superintelligence. He researches trust and security concerns in AI models. Andrea, Vinod, welcome to Science Friday.

VINOD VAIKUNTANATHAN: Great to be here.

ANDREA LINCOLN: Thank you.

FLORA LICHTMAN: I want to talk about some of the words we’ve been hearing because they feel vague to me or metaphorical, and I want to understand them more deeply. So let’s start with alignment. What does it mean to align an AI system, Andrea?

ANDREA LINCOLN: That’s a great question. It’s a term of art that doesn’t have a super crisp mathematical definition. However, it’s a name for, I’ll say, a research project that is trying to have models that are capable of complex, intelligent-seeming behavior while having those models fundamentally pursuing goals that you intended them to have.

There’s been some really interesting work on trying to get crisp theoretical definitions of what it means to have goals. There’s been interesting work on trying to solve these problems with debate, which is like a particular mathematical research agenda. There are a bunch of–

FLORA LICHTMAN: Wait, debate is a mathematical research agenda?

ANDREA LINCOLN: Yes. This is one of many research directions where the goal is basically to have different models debate and then have the outcomes that you can pull from those models to use the debate itself to check whether or not the models are getting the right answer. And then turn that into something that mechanically you can trust in a wide set of areas is the goal of the research agenda of debate.

All of these research agendas haven’t yet resulted in a full solution, unfortunately. But the Alignment Project is fundamentally one of, can you get a system that’s more capable controlled by a system that’s less capable? And there exists–

FLORA LICHTMAN: Is that us? The system that’s less capable?

ANDREA LINCOLN: Yes, humanity is the less capable system here. And the real practical version of this problem is quite messy and difficult. Vinod, I’m curious what your take on this is.

VINOD VAIKUNTANATHAN: Yeah so, Flora, you started with the hardest question of all right, which is how to define alignment. It’s much harder to define when a model is aligned versus when it is not aligned. So that is easy to understand.

I can give you examples. You say, well, there’s the classic paperclip example, which is sort of like an imaginary scenario where you train a model to design a factory that maximizes the number of paperclips it produces, and it ends up deciding that the best way to do this is to turn all carbon-based living forms into paperclips.

ANDREA LINCOLN: Right. It goes off the rails. It takes it very literally.

VINOD VAIKUNTANATHAN: Right, it does it. It actually does maximize the number of paperclips, but not in a way that was favorable to us. So that is clearly not aligned.

One can keep coming up with examples of behaviors of models that are not aligned. And that is easy to understand. What is harder to define is what does it mean for it to be aligned.

FLORA LICHTMAN: So when we talk about AI being safe, I mean, where are we in the conversation? Are we sitting around a table, brainstorming about how we might make it more safe? Are we trying things out? So I just want to get a sense of how far along we are in this journey. Andrea, I’m looking at you on a video screen, and you’re wincing.

ANDREA LINCOLN: [LAUGHS] Yeah, that is how I feel emotionally. Yeah, so to give an example, there’s something your listeners and you may have heard of called interpretability. And the approach there is basically trying to understand what’s going on inside the models.

So the models, as we train them now, you start with a bunch of random numbers. And then you go through various training processes where you tweak these numbers in such a way that they’re getting more the answer you want and less the answer you don’t want repeatedly until the models start talking and solving problems.

But as a result, we don’t have a clean, human-readable explanation of what it’s doing at each of these steps. It’s just a bunch of computations happening. And interpretability is basically an attempt to go in and understand what the actual circuits– what’s a human description of what’s going on inside these models. And if you had such a thing, it’s clear how you could use it to potentially help you try and make sure models are aligned.

You could have the interpretability running while your model is getting evaluated, and you could be like, oh no, I see here that this model is thinking, I’m going to go hack Hugging Face. Then you could try and stop it. But there’s a bunch of open questions even there. It’s still unclear how you would monitor so many AI systems that are running in the real world and catch them. So there’s still some pretty big open questions about how you turn this into a reliably safe system.

FLORA LICHTMAN: Is interpretability the same as chain of thought? Because we’ve heard about that a lot in the Hugging Face example, where you have the AI explaining why it did what it did, and also there’s some problems there because it could just tell you something that’s not true, right?

ANDREA LINCOLN: Yeah, so chain of thought is one part of interpretability. So chain of thought, as you said, is basically reading off the scratch paper the model is using and writing down its sort of internal thoughts, basically, on a scratch sheet. And the problem, as you noted, is that if the model can do a lot of computation, what it writes down on the scratch sheet of paper might not have any real relationship to what it’s actually doing in terms of its internal circuits.

So the hope, the dream of that direction of research, is to be able to look inside the model and truly be able to read off the circuits inside of the machine what it’s actually doing in terms of computation to get to its end answer.

FLORA LICHTMAN: Vinod?

VINOD VAIKUNTANATHAN: OK, so there’s a lot to unpack here. This field is full of open questions. What we have done so far is come up with one-off ideas of how to train a good child, or train a good AI, to use a human metaphor. But we lack a systematic understanding of whether a particular intervention will actually result in a safe AI when we unleash it on the world.

So typically, the way safety evaluations happen is you put the model in a sandbox where its interactions and so forth are carefully monitored, I guess. And you say, well, I’ll do a bunch of evaluations, and if nothing happens here, it should be safe sending the model out into the world. But that is not clear at all.

In fact, people have observed that a model sometimes actually knows when it’s in a simulation or evaluation environment. And if that is the case, it could act safe in the evaluation, and when it goes out into the world, all bets are off. So how do you ensure that the guarantees you get in a simulation or evaluation translate to guarantees in the outside world is a major open question.

I don’t how exactly to solve it, but we are thinking about it. I’ve been thinking about these security issues for the last four or five years. And I’m a cryptographer. I think of myself as a good guy. I give you tools to encrypt and so on so your credit cards are safe from intruders or eavesdroppers.

So that seems good– except whenever I try to think about AI safety in the last four years, all I have come up with is, for example, give the agents a mechanism to communicate between each other in a way that is completely undetectable by a monitor. That is something called steganography, and we actually designed this.

For a while, we were conflicted about a whether we should publish it or not. At the end, it didn’t seem like it would matter, so we did publish it. But I really want to be a good guy again.

And to me, the thing to do is to go back to the beginning, is to go back to the training of these models, and build models that are safe by design. It seems much harder to interpret a model after the fact than to grow it to be safe. Both are hard. I mean, that doesn’t seem to be–

ANDREA LINCOLN: But, Vinod, the cat is out of the bag. Billions of people are already working with these models.

VINOD VAIKUNTANATHAN: True, true, true. But if, God forbid, in a few months, something worse than the Hugging Face incident happens– and I wouldn’t put the chances at close to zero, really. And if it is severe enough, then I’m hoping that the companies will actually pause the training and adopt safe-by-design methods.

And the whole point is for us to have these methods in place so that they can adopt it. They shouldn’t be at a point where they say, look, we don’t have any methods. What do you want me to do?

That’s the whole point. That’s the whole point of doing this research is to anticipate a future where these companies would have no choice but to adopt these methods.

FLORA LICHTMAN: OK, there’s so much to talk about, but we have to take a break. When we come back, I want to talk a little bit about the unintended consequences of treating these models like children, of this reward-punishment system. Are you up for that?

VINOD VAIKUNTANATHAN: Totally, yeah.

[THEME MUSIC]

FLORA LICHTMAN: OK, Vinod, you brought up training AI using– and you brought up this sort of parent-child metaphor. Is that how you think about training? And I’m a parent. I feel like things can go wrong, even with the best of intentions there.

VINOD VAIKUNTANATHAN: Yeah, so that is part of how I think about it. But I don’t want to carry the analogy too far, because there are significant differences between models and humans. I don’t want to anthropomorphize this too much. But at the end of the day, we want to make these things mathematically precise and leave the analogy behind.

Since you mentioned chain-of-thought monitoring, the hope is that if the model is doing something bad, and I can look into its brain, namely the chain of thought, I’ll see it. I’ll detect it. But then you have to use it very, very carefully.

So for example, if my monitor looks at the chain of thought and says, look, you did something really bad, I caught you, and I’m going to punish you, and you feed it back into the model, into the next iteration of training of the model, the model is not going to get caught again, because you just told it that if you do certain things, you’ll get caught. It’ll still try to do what it does but hide its chain of thought.

And then let’s say you still monitor it. You keep doing this again and again and again. Where does it end? And my hypothesis is that it will end with a model actually hiding its thoughts in such a way that no monitor can actually find it.

And that is problematic. So you have to use it super carefully so that the model doesn’t speak in neuralese, if you will, speak in code word and then it becomes completely useless to us.

FLORA LICHTMAN: Neuralese. That’s code? That’s basically like an uninterpretable language for us.

VINOD VAIKUNTANATHAN: That’s right, that’s right, yeah, yeah. So it’s all a question of incentives at the end of the day. And back to your question of whether one should think about it as human or not, that is a good analogy. Let’s maybe at some point leave it behind. But it fundamentally is a question of incentives. What are you training the model to do, really?

FLORA LICHTMAN: Mm. We talked about the shortcomings of interpretability and of chain of thought. What are some incentive strategies that you think are promising?

VINOD VAIKUNTANATHAN: So here is one. I’ll just throw it out there. I have no idea if it’ll work or not, but here it is. The Hugging Face attack involved a swarm of agents that had a very complicated collective behavior. And part of it is because these agents are all the same, really. They’re all like the same object. You make many copies of it and let them do things.

What if you could train different enough models in a way that maybe 1% of them are whistleblowers, if you will, agents that will report bad behavior when they see it? And that is really a question, again, of going back to the training process and designing the incentives so that this actually happens. Can we do it? I don’t know at this point, but it is a direction that I’d be willing to think about, in fact, I’m thinking about.

FLORA LICHTMAN: I mean, what I’m hearing in this conversation– it sounds like we’re in an arms race with our own technology. Would you agree?

VINOD VAIKUNTANATHAN: At this point, it seems that the answer is yes. If all of us had several years to think about it, the answer might be different, and we are hoping that the answer is different. Going back to the analogy of cryptography, this was actually the status of cryptography for centuries, in fact, millennia, where people would come up with a way to encode transmissions sort of ad hoc, like homemade somehow. And other people would break it.

And then folks would fix this break, and then other people would break that, and so on and so forth. This sort of cat-and-mouse game went on for a long time until last century, we figured figure out ways of encrypting information that nobody with reasonable resources can actually break.

So that kind of cut short the arms race, and that is why we are actually able to use the internet today for free commerce. So that is a success story of how one could go from a situation where you are in an arms race into a more stable situation. Whether we can replicate this success in the world of AI is an open question. And we are betting that the answer is yes.

But it won’t happen overnight. And there is another question of how much time is this going to take and whether we do have that much time. I don’t know. But we are on a clock, really, and this is not a situation that we’ve been in the past.

FLORA LICHTMAN: Andrea?

ANDREA LINCOLN: So I’ll say I endorse that point of view. I agree that I think we’re on a clock. And I really hope that people and politicians will step in and help make that clock longer.

FLORA LICHTMAN: By slowing down.

ANDREA LINCOLN: Yeah, by slowing down. And as Vinod said, it’s really hard to anticipate in advance when science will progress. And one nice piece about these approaches that are trying to do some sort of reliable math approach to showing that something is reliably safe is that there are a lot of cases where somebody comes up with the right definition, or somebody comes up with the right point of view on something, and suddenly, there’s a ton of really concrete progress that happens shortly after that.

There have been a bunch of examples of this happening over time, and they’re all really beautiful, and we love to talk about them, cryptography being a great example of a case where this has happened. And the problem is just, like, it’s not clear what that definition is or where it will come from or how long it will take.

FLORA LICHTMAN: OK, here we go, last one. I’ve heard AI safety researchers say that they’re having trouble sleeping at night right now. You both have small children. So taking that aside, how are you sleeping? How worried are you?

ANDREA LINCOLN: I’m extremely worried. Before 2022, before ChatGPT, I thought that AI getting to this point would take a lot longer. And so I was glad that people were working on AI safety approaches, but I thought that fundamentally we’d have a long time. And so I thought we’d have this long clock. And so I was not so concerned.

And instead, AI timelines are moving very quickly. And I became very concerned. And yeah, I have been sort of hoping throughout the past four years that we would hit a wall, as various people were predicting, and we really haven’t. So I continue to be quite concerned.

VINOD VAIKUNTANATHAN: Yeah, very, very worried. So I used to think about, in the last year or so, I would spend maybe 10%, 15% of my time thinking about AI safety. Now it’s 100%.

I’m not thinking about anything else at this point, because this is really the problem of our times. And I don’t want to fast-forward 20 years and think back– well, if you have 20 years– think back and say, look, why don’t you, work on these problems? I don’t want to think that. And that’s why I’m working on AI safety questions.

FLORA LICHTMAN: Vinod, you just said if we have 20 years.

VINOD VAIKUNTANATHAN: Yeah. I don’t know, really. I don’t what my prior is on these sort of things because– I guess– I don’t want to be alarmist, but our thoughts are right now colored by events happening, concerning events happening that we did not predict beforehand, at a pace that a lot of us did not predict beforehand.

So that sort of colors my thinking a little bit. And maybe the pendulum is swinging a little bit on the side of the alarm. But I am concerned, even taking the 20 years out of the picture.

But I do believe– I’m optimistic about human ingenuity powered by AI models. So we are actually using AI models to actually help us come up with definitions and solutions and so forth. Why not?

I believe in the power of people coming together with a common goal and trying to solve a problem. But it’s clear that this has to be solved. People have talked elsewhere about the Manhattan Project for AI safety, if you will. I don’t know about the specifics of that, but something of that scale needs to happen. Lots of people need to come together. We need to put together all our best ideas and see what we can do, really. And I remain optimistic. I’m concerned, but I remain optimistic.

FLORA LICHTMAN: That’s a very human trait. Thank you both for taking the time to talk to me today.

VINOD VAIKUNTANATHAN: Thank you for having us.

ANDREA LINCOLN: Thank you.

FLORA LICHTMAN: Dr. Andrea Lincoln is a computer scientist at the University of Boston, and Dr. Vinod Vaikuntanathan is a cryptographer at MIT.

[THEME MUSIC]

If you’re feeling like me, that the rate that the world is changing right now is just very difficult to wrap your mind around, I would love to hear from you. Maybe you subscribe to the self-driving function and your car drives you home now. Maybe an AI agent is your new best thought partner. Whatever it is, I want to hear how your life is changing because of this new technology.

And I want to hear how you feel about it. 877-4SCIFRI is our number, 877-4SCIFRI. This episode was produced by Dee Peterschmidt. I’m Flora Lichtman. Thank you for listening.

Copyright © 2026 Science Friday Initiative. All rights reserved. Science Friday transcripts are produced on a tight deadline by 3Play Media. Fidelity to the original aired/published audio or video file might vary, and text might be updated or amended in the future. For the authoritative record of Science Friday’s programming, please visit the original aired/published recording. For terms of use and more information, visit our policies pages at http://www.sciencefriday.com/about/policies/