Transcript of Who Is Responsible When AI Agents Make Decisions | Matt Mills x Shekhar Natarajan

One of the ways people gain access today, bad guys do, right? They pay somebody for their password.

Not even Geoffrey Hinton, the guy who's actually spoken today, the father of, like, godfather of AI, even he doesn't know how, like, these systems really work.

80 to 90% of these breaches which has come via stolen credentials, uh, via identity.

Can you punish an agent?

And, and, you know, we, we like to say, you know, you're not going to put a Waymo in jail.

Welcome to yet another episode of Tomorrow Today with Shekhar Natarajan. Today I have an amazing guest with me. I'm here at AI4, and I'm really thankful for AI4 for giving me this opportunity to speak to some amazing people. I'm going to talk to Matt. Matt comes from SailPoint. Yeah. He's the president of SailPoint, and they are into identity management. Matt, welcome to the show.

Thank you, sir. It's a pleasure to be with you.

Absolutely. So, Matt, like, explain to me first, who's Matt?

Yeah, who's Matt? So, I've been with SailPoint for 7 years, right? I act as president, really spend a bunch of my efforts around the go-to-market space. And prior to that, I probably, my Next biggest thing, I spent over 20 years at Oracle, right? So that was, you know, a trip before this. But so yeah, so this is me now.

And you were in security and identity management and governance prior, or?

Well, I mean, it was part of my portfolio. I led a good portion of the North America business. And it's funny, I competed against SailPoint even when I was at Oracle, right? So I was a little bit familiar with them when I got the opportunity to come over and be a part of this.

So, for people who don't know much about governance, let's talk about governance first and then we'll get to identity and like why, like, you know, AI is fundamentally going to reshape this entire world. Identity being one of those key issues to kind of like grapple with. So, let's step back and basically let's talk about This is not like a— it's not a sexy part of the organization. It's considered more like compliance, you know, do no harm and preventative and so on and so forth. So, it's a governance layer. So, how do you first explain what governance is? What is the cost of governance? And what if governance is not conducted appropriately in a company?

Yeah. So, look, it's evolved quite a bit.

Yeah.

Since the onset of SailPoint some 20 years ago. You know, 20 years ago when they, you know, it was back office, right? When you think of governance, you typically think of like audit, right? Compliance. I think all of that stuff is still relevant, but it's about basically giving identities, humans up to now, access to things, applications, and then entitlements, what they can do in those applications, right? And then governing it, meaning certifying it, provisioning it, right? So I gotta make sure that you have the ability or the access to those entitlements, and then I've gotta provision them and give them to you, and then I gotta monitor it, right? So that's kind of it in a nutshell, what traditional governance was. Back office, typically static, right? Not a lot of change. We had traditionally episodes where somebody leaves the company, And if nobody actually makes them a leaver, like leaving the company, they keep all their provisions and they could go out and theoretically still have access to email and all the systems and whatnot, right? So it's always been an issue, right? But now it's kind of moved, I think, to a very different place because, as I was sharing with you earlier, if you look at the breaches today, 80 to 90% of these breaches come via— People.

stolen credentials via identity, right? So now we find ourselves smack dab in the middle of trying to secure the enterprise, right? And that's really what it's about. And historically, you had humans, and now you have these non-humans, right? You have these agents, you have these bots, you have the RPAs, you have just tens of thousands of different types of agents that are out there acting on behalf, typically, of a human. So now we find ourselves commingling and pushing them together because, you know, our view of it is that if you're gonna secure your enterprise, if you're gonna have a lot of these non-human identities, you gotta be tethered to something.

So there is like many things like MDM, like mobile device management, and like, you know, I really appreciate what the guys have to do from the security safety standpoint, phishing, emails, like, you know, ransomware, like all these attacks. So from a, when you talk about governance, what are you really talking about? Are you talking really about identity management? Are you talking about basically like intrusion detection, mobile device management? Like there's so many different aspects of like how enterprise think about safety, security, and governance. Are you playing in a sliver? Are you playing as a whole or what are you doing?

Our view is this, right? At the end of the day, we're protecting an enterprise. And if you think about what is it we're protecting, it's not necessarily agents and it's not necessarily humans, right? They're a path to be able to get at where the goods are, and the goods are the data.

Yeah.

Right? And so you have different methods of accessing that data. Could be from an iPhone, could be from a PC. And what we will do is now with context, be able to say, if you have access to, let's say, your forecasting system, salesforce.com, right? And if it's 1 o'clock in the morning and you're at your home on your PC, we're gonna let you do that. That's the context. But if it's 1 o'clock in the morning and you're in a bar in Tijuana, right, we're gonna add that context and say, this is a little bit risky and we're not gonna allow you to do that, right? So we're kind of that preventative layer, right, that gives you permission, gives you access, And let you get— I mean, the differentiator for us is really this fine-grained entitlement. We can go very, very deep and handle these very complex use cases at scale. And that's kind of one of our unique callings.

So, you guys bought Entro.

We did.

Okay. So, what does Entro do?

Entro is— we're pretty excited about it, right? It's all about non-humans. And it's a discovery tool. It's a visibility tool. It categorizes. It takes a very unique approach to actually finding those entitlements, endpoints, if you will, right? But today, it'll be— in fact, I think we announced it today, right? The integration of our Intro already with our, what we call the SailPoint Agentic Fabric. And so I think it's really going to be a significant difference maker for us. Just in terms of its discovery and visibility capabilities.

So what are you discovering?

Non-human identities.

How do you do that?

Well, that's what this tool does, right? So it'll go out—

How do they do that?

Typically, it'll go out to the endpoint, right? And gather up all the endpoints and then come back and attach them to an identity. I mean, one of the big problems you got in all this is you have a lot of these identities, non-human identities that are orphaned. They don't have a human owner, right? And that's one of the biggest challenges these companies have. Not to mention, you know, with a human owner within the company, outside the company, inside the company, inside the company.

So the ratio of non-human to human, right, is growing significantly, right? I mean, if you listen to some of the folks like—

But like when you leave a company, when you're working for a company and you leave the company, then they turn off your access immediately and all of your credentials and like, you know, you don't have, like, that's standard practice.

That's how it should happen.

That's how it should happen. You're saying it doesn't happen that way?

Oh, I don't think it does. I mean, there's a process in most companies called—

How big is your business?

Our business? Yeah. How big is it in revenue?

Yeah.

We're over a billion dollars.

Wow. So people screw up so much.

Yeah. Well, I mean, look, we have 3,500 customers, right?

3,500 customers?

Yeah. And so when you think of this— Is this more like a risk insurance

Or is it like really preventative?

I think early on, right, it was risk, you know, I don't know if I'd call it insurance, but risk mitigation, right? But now it's come, it's front and center to you being able to secure your enterprise. I mean, if you go back and look at all these breaches that have happened just in recent weeks, and I get asked the question all the time, well, if they were using SailPoint, you know, could you have prevented that? And the answer is, yeah, right? But look, there's a lot to it. You have to make sure that you're using the technology, you have it organized correctly, you've got the right operational orchestration, right? But if you do that, the answer is 100% yeah. And we do that because, number one, our platform brings the non-humans and the humans together.

So, we'll get to that in a minute. But so now help me understand, I left my identity in the company.

Yeah.

And you forgot to turn off my identity. So, how does that identity get, like, hacked by someone when you have been denied the access when you left the company?

Well, if you're turned off, right, and your credentials are taken away, then they won't.

Then it's not a problem.

They won't. But it's funny. Right? I mean, one of the ways people gain access today, bad guys do, they pay somebody for their password, right? That's a pretty difficult thing to prevent with technology. And so you put things—

How do they ask for that?

I mean, they ask somebody who's typically low-level, who has access to, like, say, service accounts and passwords, and they bribe them.

But is it not ethical? Is it not a crime to actually give away passwords? I put a company at jeopardy.

It's terribly unethical. Yeah.

And then what happens to those people?

If they get caught, they get prosecuted. Then the question is if they get caught, right? So yeah, I mean, it's— look, this is not a problem unique to us. It's a problem that's plaguing, you know, I think all of society and all of business, right?

Yeah, absolutely.

I mean—

So now let's step back and basically talk about The world you lived in and the world we are entering into. We lived in a very deterministic world where the rules and access and everything, like, you know, you could determine like, this is a human, this is not a human, da da da. Like, you know, his access code, his SSO is so much. Like, you know, this fellow has MDM access, doesn't have it. You know, all of the security protocols, safety protocols that you have. And that was a rather deterministic world where authority, access, and basically your ability to activate certain accounts, sort of like you could manage. Now in the world of AI, you moved into more a probabilistic world, right? So how do you then, when you ask, I'll just simply ask a simple question. When ChatGPT, when you go and ask ChatGPT a question, It doesn't give you the same answer twice. How would you hinge identity of an agent to a system which cannot be controlled? How would you do that? Like, how would you—

See, we're not governing or trying to control the quality of the output, right? What we're trying to do is making sure that people have access to the right things in the right environment. So, for instance—

I get it. I get it. But like, you know, like, let's say like, because I come from the AI world as well. And so now part of my agency, my work was actually now being managed by an agent. Okay. So, I gave some of the attribution of what I do how I do my work and what it means for me to complete that work. And I've given that agency to an agent. Now, agent doesn't work in isolation. It draws its knowledge and its memory from models. And it basically creates emergent properties. Not even Geoffrey Hinton, the guy who's actually spoken today, the father of, godfather of AI, even he doesn't know how these systems really work.

Yeah.

He knows the input and he knows the output. He doesn't know what the transformer technology really does. Sorry. So, how do you solve the identity problem of an agent which assumes the properties in a very probabilistic way? And then agents can spawn agents. And so, how do you manage and control all of those loops?

Well, look, here's what I would tell you. We put things in, place foundationally. And we believe today that there needs to be a human—

In the loop.

In the loop for every agent, right? And that human ultimately, ethically, morally is responsible for that agent, right? And so, my predictions—

Is it the authorizing person or is it the model provider? Is it the The guy who actually trained the model? Is it the guy who gave the knowledge to the model? Or is it the guy who actually provisioned the agent? Who's responsible?

Yeah, I think at the end of the day, the human that's responsible for that agent is ultimately responsible. Now, you're gonna get into a bunch of things here like policy, right? There's a policy engine, there's a risk engine, there's an intent engine, right? All of these things are gonna play into a risk analysis that you're gonna get For these agents. Now, the authorization still comes from the person, the human.

The behavior didn't come from

the— The behavior, the more complex these models get. So you talked about nested, right? I think there's always going to be these unintended consequences that come out of this on things that they do that were completely unintended. And I don't think you're going to get away from that. I think that the more and more complex these get, you're gonna be dealing with that. And I think at the end of the day, you have to have some level of accountability. Yeah. Right? And, and, you know, we, we like to say, you know, you're not gonna put a Waymo in jail.

Yeah. Right? Yeah. See, so, okay. That's where I was headed.

Yeah.

So, you, you definitely cannot put Waymo in jail. Neither you can fire the algorithm which powers Uber, which discriminates drivers sometimes. Right? So, let's get that right, right? So, in the world of AI, identity is so crucial, but the consequences for that bad action doesn't seem to me like, you know, you're not punishing someone for doing something wrong. Can you punish an agent?

I don't think you could punish an agent, you know, but you can punish the person who is authorizing that agent, who's certifying that agent. And that's— and look, I—

How do you certify something even like the godfather of AI doesn't know how to certify?

Look, I hear you. It's a real problem. And it's something that—

So human in the loop is the best way you can handle that?

Today. Today. Look, we have customers that are very, very advanced. They would argue with me today, right? But it's— I mean, when you look at the adoption of this AI, this agentic world. Right now, people are screaming for structure. Yeah. I mean, they want regulatory to tell them this is how it should happen. And they love when we talk about blueprint, when we start talking about ethics and morals for agents. So as I build these, I build them with the intent of them being ethical and having morality. And Look, it's still not going to work out 100% of the time, and you're still going to have to deal with these unintended consequences, right?

Yeah. So where do you see the world in 5 years from now?

Oh man, I don't know. I mean, we, you know, look, I think in this world it's tough to project 3 years out. I mean, we think agents are going to be a big— they're not going away anytime soon. And what we're trying to do is provide this foundation that allows us to grow. You think back, you know, just 12 months, We weren't talking about any of this, right? And so here we are. And in another 12 months, I think it's gonna continue to evolve. I think you're gonna be into real-time authorization, right? You're gonna be running— I mean, think about the thousands of agents are gonna be out there that you're gonna have to be able to manage. I think it's gonna continue to get complex. I think today, I think one of the big challenges for companies is that everybody says they can do it. Everybody says they can solve this problem. And the fact of the matter is it's a really, really short list of people who can. And when you look at the buyers today, it's big-time buyer beware. And we're kind of back to the future a little bit in that people want to see this.

I don't want to listen to you anymore. I'm skeptical. I want to see it, put it on my network, run it with my data. I want to actually see it work. And that's where we're at today.

Got it. Got it. Got it. Interesting. So, when you think about like where the world would go with agents and humans working together and sharing the work and sharing the space, how do you build trust? Yeah,

that's a big part of it.

A big part.

I think, look, I think it starts with, you gotta have some regulatory, you gotta have some rules, you gotta have some structure. And then you've got to adhere to them, right? Yeah. And that's a big thing. It's a challenging thing. I think there's a lot of companies today and people that are buying CISOs, CIOs, they're skeptics, right? And so, I've kind of always been a believer that you do a couple of things right, you start to build credibility. You do it right over a longer turn of time, you start to build trust. And I think that's what we're all up against. We've got to start building that credibility and then you earn that trust.

Yeah. So, you're selling identity as a service to companies?

Yeah. Yeah.

It's a paradox. You're in the business of securing the identity, but you're selling identity as a service to people. So, in that world, Previously, like, you know, the actions were not monitored, right? With AI, you could monitor the actions of CT. Yeah. And basically now you begin to understand the intent of what that person was trying to do, right? Who actually has control of that data?

Yeah. I mean, again, Our premise is, over time, you're going to have a policy engine, right, that's probably AI-driven because it's going to be too much for humans to keep up with, right, that drives a lot of that.

Why does policy need an AI engine?

Well, because if you're going to get to this where everything's real-time, and when you walk in the building, I know you're in the building, and here's the things that I'm going to make available to you right now, or maybe You're going to check out capability.

External vulnerabilities.

Yes, nothing static. I mean, look, we've been talking about these ideologies like zero trust, zero standing privilege, least standing privilege forever, and they're ideologies. They're great, but they're really hard to get deployed. I think where we're going now, we announced here today our Agentic Business and Agentic Business Plus suites. And they're tailored to be able to drive those kinds of outcomes. So, Agentic Business is built to deliver least standing privilege, right? And our Business Plus is zero trust and zero standing privilege. And you have the advanced capabilities to be able to automate and deliver those kinds of things to companies. And look, there's still going to be a transformation component to this, right? But I think all of a sudden when you start being able to deliver that, You start taking a ton of risk out of your business for breach and for bad behavior and for being over-permissioned, right? Because now you've got a set of eyes on it. And the other thing I'll tell you is this. When you're talking with auditors, right, and you think they're not going to really care about this, they care a lot about it. They want transparency in everything.

They want to be able to see it. They want to be able to drive some level of consistency in how decisions are made. Even if they're made by AI, right? And so, that's a big part of it. And that's what we've had to add to our product. It's available today. And I think that's what allows companies that are buying to feel comfortable, right? There's a way to look at this and make sure that we're getting, for the most part, the intended outcomes.

Do you think companies are morally and ethically right?

Do I think they are?

Morally and ethically, right? Because you're trying to solve the moral and ethics problem. Are they the best custodians of morals and ethics?

Yeah. I mean, I don't know. I mean, look, at the end of the day, I deal with, you know, I get to talk with a lot of people, and I think they come from the right place. But I think these are complex problems, right? With layered technologies. And so, I think, you know, probably more often than not, They're

not aware of what is right

here. Yeah. Are they conscious or unconscious? I think they know right and wrong. I just think the problems are so complex that they might not know that wrong's happening right down here, or that I was doing right and these agents decided I'm going to do something else and they didn't catch it. I think that's probably more likely.

Well, it was a fun conversation.

Yeah, yeah. I appreciate it.

I really appreciate like your time here.

Yeah, yeah. Well, if you got a hole to fill sometime, gimme a call. I'll jump back on with you. So appreciate it, man. Thank you. All right.