Rethinking Risk: Data-Driven Decisions for Modern CISOs

Raj Krishnamurthy (00:01.232)
Hey, hey, hey, welcome to Security in GRC Redecoded. I’m your host, Raj Krishnamurthy. Very rarely do I say that we have this multifaceted guest. Today we have one of those guests with Tony Martin-Wigg. Tony has been a risk practitioner for a long time. He’s a professional speaker. He goes on professional circuits, talks about cybersecurity risks. He has run cybersecurity risk management, even cyber crime. And Tony, if I’m saying this right, for companies like Gap.

Netflix, he’s also co-chair of many of these forums. He’s with the FAIR and he’s also with SIR. So Tony, welcome to the show. It is fantastic having you on the show.

Tony Martin-Vegue (00:42.392)
Thank you for having me Raj. It’s a pleasure to be here.

Raj Krishnamurthy (00:46.31)
Tony, I’ll, I want you to, what brought you to risk management?

Tony Martin-Vegue (00:51.598)
That is such a great question and I think my origin story goes so far back. So I started out, like a lot of people do in information security or risk management, just doing run of the mill IT work. My very first job was first line phone support for a computer manufacturer. And I’m gonna date myself a little bit here, it was Windows 95. So.

For the listeners that aren’t familiar with how troublesome Windows 95 was when it first came out, it was pretty terrible. So I think that being the first line phone support for that product really gave me my thick skin, which comes in handy with risk management. So fast forward many years later, I wanted a career change, but I still wanted to hold on to all the skills and all the learnings that I had from networking and

computer support and learning about system administration. So I chose information security. And then a really great mentor of mine told me to break into information security, you really need to choose a niche. Are you going to be red teamer? Or are you going to be blue team or incident response or risk management or cryptography? What are you going to choose? I’ve always been really fascinated with economics. I have an economics degree.

And once I heard about risk management, I thought, okay, this is it. This is my calling because I can use so many different aspects of my brain outside of technology and outside of security, finance and just so much and apply it to this one field and really help organizations and companies execute their value proposition. And that’s what attracted me to it. And I’ve loved it ever since.

Raj Krishnamurthy (02:42.118)
That’s a killer combination, right, to come from economics into cybersecurity. I think, no wonder I think it’s a perfect intersection of lot of what you, the seminal work that you’re doing around risk, I would assume. Tony, I forgot to mention this in the preface, that you are also an author and you have a book coming soon. Tell us about the book.

Tony Martin-Vegue (03:03.371)
Yes, so the book is called From Heat Maps to Histograms, a Practical Guide to Cyber Risk Quantification. And it comes out in early 2026. I don’t have a release date yet, but it will come out early next year. And I’m really excited about writing this. I’m excited to share this with just the general risk management community. And it’s I’ll just give you a quick TLDR. It’s basically about

cyber risk quantification, but written from the point of view of a risk analyst. I’m a risk analyst. I’m not a mathematician. I’m not a statistician. Some of this stuff really comes hard to me, to be honest with you. And I wanted to write the book that I wish I had 15 years ago when I was trying to get started and really trying to understand risk quantification.

And I didn’t understand basics like Monte Carlo simulations. It just seemed so mystical to me. And for other people, it just seemed like a natural to them. that’s who my book is for. It’s for the beginners, the people who think Monte Carlo, they hear Monte Carlo and think, that just sounds way too hard. I can’t do that. But you can do it.

Raj Krishnamurthy (04:19.228)
And Bayesian statistics is another term. Totally, totally. And I think you did a great job in one of your recent posts. I saw that you called out on Monte Carlo analysis as well, making it simple, digestible to your readers, right? So that was super cool. The cyber risk quantification, it’s a sort of a domain in its own. What is one heart take or a controversial opinion, if I can ask you, that you may have on this space, on cyber risk quantification?

Tony Martin-Vegue (04:49.907)
this is a hot take. It’s controversial, but I hope that your listeners will connect with it. It’s not as hard as it sounds. You don’t need to boil the ocean. You don’t need to collect all the data. You don’t need it to be perfect or absolutely precise. You just need it to be directionally correct, heading in the right direction. And a lot of us can use the data that we have today.

And I’ve always said that I if I spend an hour with you I can get your first quantitative risk assessment done today and one one hour and then tomorrow you can do the next one and the next one and the next one just use what you have and the reason why I think that’s controversial is I Think CRQ kind of has this reputation of just it being out of reach or you need to have a huge team or a lot of money expensive tools

and you don’t, you can do it in Excel. You can do it with publicly accessible reports, just stuff that’s available right now, just with internet searches, Google searches, and you can get that started for yourself right away.

Raj Krishnamurthy (06:01.168)
Let me, I want to break this down for our listeners, Tony. Let’s break this down. To a see-saw, I’m going to give you a few profiles and I want you to react to it, right? If you are the Chief Information Security Officer, what questions does CRQ answer for me?

Tony Martin-Vegue (06:18.357)
Okay, if I could compare it to qualitative risk, and then I’ll tell you what CRQ gives you additionally. So qualitative risk, what I mean by that is traditional heat maps, risk matrices, red, yellow, green, high, medium, low, that’s qualitative risk. That is effective for risk prioritization and ranking. It helps you rank your risks.

I have 10 risks, which one should I do first? Let’s do the ones colored red. And it does help you with that. So I have to give credit where credit is due. Where risk quantification adds on to that, what additional things you’re gonna be able to give that persona that you mentioned. Risk quantification can also do risk ranking and prioritization, but now we can do additional add-ons. We can do cost benefit analysis. So.

I need to mitigate data breach. need to turn that red into a yellow. With qualitative risk, that’s all you’re doing. Okay, we just need to mitigate it. With quantitative risk, with numbers, I can tell you that there’s two different projects we’re considering, Project A and Project B. Which one gives us the best bang for our buck? Which one reduces the most loss exposure? clearly it’s Project A. We’re gonna reduce $50 million.

Project B only reduces by 10 million. And then also if you know how much each project costs, I can give you return on investment ratios. So this one is such a good deal and this one’s not. If you have a negative return on investment ratio, and what I mean by that is, if we’re gonna reduce risk by $5 million, but the project costs us 7 million, we don’t wanna do that. That’s not a good.

not a good return on investment, let’s go rethink the project or go think of something else entirely. You don’t get that with qualitative risk with Red Yellow Green. my whole.

Tony Martin-Vegue (08:23.85)
My whole point with this is it enables such richer and deeper conversations. And there’s so much more you can do besides return on investment conversations or comparing A versus B projects. There’s so much more you can do, cyber insurance conversations. There’s just so much around it. that’s probably the number one thing that people ask me for.

Raj Krishnamurthy (08:49.948)
Got it, got it. And I think the comment that you made that you don’t have to wait to boil the ocean, right? You can start with where you are. So I’m assuming that is very true, especially in the quantitative world where you start with a number and your understandings gets better and your number gets more and more accurate, right? What is typically the time it takes for you to start a program like this and get to a point where it is a reasonably good quantification program?

Tony Martin-Vegue (09:19.882)
I think that’s a great question. think, as I mentioned a little bit earlier, you can do your first quantitative risk assessment today.

Raj Krishnamurthy (09:32.988)
Yes, I can see it. But Eric, I think we can keep going on. I don’t know why that is flickering, but I don’t see it now. But I don’t know if in the post he’s going to. We can cut all of this out, Tony. Eric, I don’t know if you’re there. Well, let’s go on. I don’t see it flickering now anymore. Maybe it is. Do you want me to restate the question, or can you?

Tony Martin-Vegue (09:59.093)
Yeah, sorry about that. just I wanted to make sure that. Let me unplug my camera really quick. And then and then we’ll OK.

Raj Krishnamurthy (10:02.16)
It is flickering again.

Okay, sure. Take your time.

Tony Martin-Vegue (10:58.028)
Okay, I really hope that that fixes it. Otherwise, I’m just gonna ignore it and we’ll just keep going. It’s just really slight, hopefully.

Raj Krishnamurthy (11:04.774)
No, I don’t see the flicker anymore, but we’ll see if I say it. Maybe we’ll start. My question was around the quantification and how long does it take for you to get to maturity? That was the question. And then give it a pause, Tony, so that we can edit this in post and we can go from there.

Tony Martin-Vegue (11:27.392)
So as I mentioned just a couple minutes ago, think all of us can get started with a quantitative risk assessment today. You can do your first one in an hour. And let me explain that a little bit more. It’s both fortunate and unfortunate, I think, for our community, for just the way things are today. We have a lot of data breach information. Data breaches have happened.

at such a frequency, it’s really disturbing actually. But a lot of the consequences, the impact and the frequency, how often data breaches happen and how much they hurt when they do, that is very noble. That’s all out in the public there in 10K reports with the SEC filings. We have a lot of insurance data. We know what claims look like, we know how much they cost. So if you work in a particular sector like manufacturing or

or something like transportation or utilities, you can look up what data breaches look like in the utility sector or the agriculture sector, for example, and you can get that information and do your first quantitative risk assessment today with that information. And that’s really easy, that’s low-hanging fruit. And then to build on your next one, I would recommend your second risk assessment being…

Go out in your company, your organization, and find someone that has a really big problem. Something that can’t really be solved with traditional risk management methods, red, green. I was able to find those people at Netflix, and there were a few people with just these really burning questions. Like, I really want to onboard this vendor, but I can’t get budget for it.

I really want to get additional headcount, but I can’t get budget for it. So I did a risk assessment. So here’s your current baseline risk, what it looks like today. And here’s hypothetical risk with risk reduction if you bring this new vendor on and they were able to compare and contrast the two and show a very positive ROI. And then something amazing happened. They were able to get their budget and headcount.

Tony Martin-Vegue (13:45.654)
So just building on from that, I think that that’s about six months. You could do your first risk assessment today in one hour, about six months to get all those big problem ones done. And then at the six month point, I would say, now you tackle your risk register. Now you start converting your risk register from red, green over to quantitative. So I would give it about a one year runway.

Raj Krishnamurthy (14:10.619)
got it. And I think what I learned something very interesting through this conversation is that, it all starts with the questions. That’s what you’re saying. We need to understand the framing of the question seems to be half the problem here, right? Is that correct? Now, one of the challenges as an engineer that I’ve always struggled with in cybersecurity

Tony Martin-Vegue (14:26.749)
Exactly. Yes.

Raj Krishnamurthy (14:33.657)
is that unlike financial systems where you have a baseline, right, there is a generally accounted accounting principles that are chart of accounts and there is a way that you record these transactions and you’re layering your metrics and quantification on top of it. How do we do that for cybersecurity?

Tony Martin-Vegue (14:54.345)
I think that we’re missing that measurement piece. We’re missing, there’s a whole discipline called metrology and it’s just about measurements. And I think that that’s what we’re missing. We need to get better metrics and better measurements around the things that we care about. And maybe you’re at a company, you never moved to quantitative risk.

But if you can take some of those principles and implement them and get better at counting things, get better at metrics, KRIs and KPIs, I think that we can just make much better decisions and answer some of those questions in a much more intelligent way.

Raj Krishnamurthy (15:36.294)
Got it. I came across a blog. You’re a terrific blogger, Tony, and I think some of your blogs are awesome. And I strongly recommend our listeners to go take a look at your blogs as well. We’ll maybe put a link. You wrote a blog, I think this is a couple of years ago, correct me if I’m wrong, on how to write good risk scenarios and statements. I thought it was so fundamental and foundational that I want to spend some time on that. Maybe unpack that a little bit. How do we write good risk scenarios and statements?

Tony Martin-Vegue (16:06.057)
Yeah, first I’ll start with why we need that. So if you don’t have a really well-defined scope of the risk that you’re talking about.

It can just lead to misunderstandings and risk assessment efforts that go awry, that waste a lot of time. And you’re not really answering questions for the people that are asking you these things. I’ll give you a really quick example, a story from my past. So I used to work at Gap Inc, you the clothing manufacturer that owns Gap and Banana Republic and Aflata and a couple other brands. And at the time they were rolling out

hand-held product scanners just like you know just like an iPhone so they’re kind of ubiquitous now in most stores but back then they were a little bit new so you would take your your scanner that you had and you can scan products it had a credit card swiper on it and you could check people out immediately if someone says hey do you have this sweater in a large you could look it up yeah we do have it large it’s in the back or maybe

My Walnut Creek store has it. Let me order it for you right now. So it was connected to our central databases. And I was looking at a risk register and a risk workshop meeting with a lot of different folks. And we had a risk register item that said lost or stolen handheld store devices. And the risk was red. The risk was high and the likelihood was medium.

And we started going around the room and I realized there’s a huge disconnect about what we mean by the risk of lost or stolen handheld store devices. The CISO immediately thought that this is data breach risk. If somebody steals this device, they can get customer PII, they can get credit card information, they can get all of that. The store managers,

Tony Martin-Vegue (18:14.847)
They weren’t thinking data breach. They were thinking, one of my store associates doesn’t have their device. This is an availability issue. This is almost a store outage if enough of them get stolen. That’s hitting my bottom line. That’s not enabling us to give good customer service to the customers. So there was just so much of a disconnect there. The privacy person saw privacy issues with this.

and the person that was in charge of IT, the person that was in charge of provisioning these, they just saw this. That’s a thousand dollar loss. That’s a low risk because I have to go replace it. I have to go buy a new iPhone with a new credit card swiper. So there was just so many different people that saw this risk in a completely different way. And so the…

The symptom is misunderstanding in that room that I was in, misalignment on the risk, and the cause is not a good well-formed risk scenario. So to have a really good well-formed risk scenario, you wanna clearly define the asset. What is it of value that we care about? Next, you wanna define the threat actor that can act against it. In this case, it was a thief or…

It could be a untrained store associate or maybe just someone just did something accidentally because maybe you just you leave it on the counter and walk away and then now it’s lost. And then you also want to start describing the effects. What are the bad things that can happen to your organization if the first two things happen? If a thief gets a hold of the in-store device, what are all the bad things? And that’s when you start breaking out separate scenarios.

So what ended up happening is we took that one scenario and broke it out into three. So one of the effects is you’re actually losing the asset. So now we need to replace it. There’s replacement costs. Another one is there is a chance of a data breach scenario if the phone’s unlocked and they get access to customer data.

Tony Martin-Vegue (20:29.43)
and so on and so forth. So it ended up being three scenarios there. And that’s why it’s so important to get this one foundational thing nailed before you do any type of risk assessment. And this foundation applies to qualitative risk and quantitative risk. It’s we just, all of us, we need to get this one skill nailed down.

Raj Krishnamurthy (20:47.494)
Blended.

Raj Krishnamurthy (20:52.892)
Now I think that’s a fantastic, beautiful example, Tony. And I think I’m able to sort of relate to this very clearly. The question though is that when this information gets surfaced, very different perspectives, the CISOs view different than the store managers view, what view does the board get?

Tony Martin-Vegue (21:11.294)
They wanna see aggregate view typically. The board doesn’t necessarily care that we lost six devices out of our entire fleet and it cost us $30,000 to replace them and shut off accounts and do all the incident response. They might not care about that. I think that boards typically care about aggregate risk.

So this is how much risk exposure we’re caring for data breach risk, or this is how much risk exposure we’re caring for all of our in-store devices, including cash registers and all of that. And I think that if you can start to tie it to really interesting conversations the board cares about, CISOs will find themselves really successful in these types of encounters. So for example,

Here’s our total exposure for data breach and this is what we’re doing to mitigate it. But more importantly, these are the trade-offs we’re making during mitigation. These are the opportunity costs. This is what we want to do but we’re not able to do. And if you can put it in financial terms, then you’re just having so much richer conversations.

Should we mitigate data breach risk or should we put out a new product line? And then people can start making really good informed trade-offs about some of these things.

Raj Krishnamurthy (22:41.102)
And we talked about the Monte Carlo simulation before. I want to bring that back in. How does Monte Carlo analysis fit into what you just talked about on this scenario?

Tony Martin-Vegue (22:53.384)
I know it just sounds so mystical and so confusing, I think, maybe to a lot of folks, including myself when I first started getting started. Monte Carlo simulations, in true mathematician form, the person that invented it came up with a really esoteric, confusing name for something that’s incredibly simple. So a Monte Carlo simulation is just a simulation of events.

So I’ll make, just give a really quick example. If you have a coin, I wish I had one, I would actually flip it. It’s either gonna show up as heads or tails. And you can flip it 100 times and record the results. Now let’s simulate that. You can open up an Excel spreadsheet and just use the Excel random generator, rand, and set it to randomly.

show up in a cell as heads or tails. And then drag yourself all the way down to 1,000. And now you’re simulating 1,000 trials of.

coin flipping heads or tails. And now you count how many were heads and how many were tails. If you do enough trials, it’s probably going to be 50 50, at least very close to 50 50. I just described a Monte Carlo simulation, and your listeners can go do that right now in Excel. And that’s your very first Monte Carlo simulation in your life. So I would congratulate everyone for doing that. Now, just take that and apply that to cyber risk.

Raj Krishnamurthy (24:23.115)
I it.

Tony Martin-Vegue (24:31.818)
We know how much, generally we know how often data breaches happen, and we know generally how much they cost in our sector. just take banking, for example. We know how much they cost and how much they happen. What we want to do is simulate 50,000 2027s, because a data breach is kind of like earthquakes.

They kind of happen all the time. And once in a while, you have one that really hurts, like that 9.0 earthquake. So we want to simulate 50,000 2027s so we could see the distribution of that. How many of them do we get that really hurt? And how many don’t hurt very much at all? How many were just security incidents but didn’t really graduate to?

a reportable data breach because you know, maybe the attacker didn’t get PII or maybe they hit the wrong database or maybe they caught it in time. So you get to see the shape of some of these things. So for example, we’ll find out that maybe 40,000 of 2027s didn’t hurt at all. And there was a good portion we didn’t get a data breach at all. But maybe there’s 5 % that it really hurt.

And that’s going to give decision makers the ability to figure out, where should we apply our mitigations? Should we mitigate some of these low hanging fruit problems that cause system incidents or data breaches while keeping an eye on that 5 %? Maybe we need to do cyber insurance on that 5%. What should our policy limits be? It just answers a lot of those questions.

Raj Krishnamurthy (26:13.839)
Got it. I think one challenge I see when I try to do this is that in the case of a coin flip, these are independent probabilistic events. One coin flip has no bearing on the other, at least in theory. But in cybersecurity, risks are interconnected. They are dependent. They are not independent. So how does that factor into your modeling, Tony?

Tony Martin-Vegue (26:39.497)
Yeah, it can get complicated. You can start out by modeling independent events, but you’re completely right that a lot of these are interconnected and there really is a butterfly effect with a lot of these things. So I think that you can do really simple Monte Carlo simulations with cyber risk quantification today. You can do a lot of that. To do some of the more complicated interdependent stuff.

I’ve always looked to software. So I think that Excel does a really good job of getting yourself started getting some of these initial questions answered. And I’ve always looked to more complicated software. My brain doesn’t really scale to those really complicated ones. So I looked to software for that. But there’s a lot of really good vendors out there, a lot of really good companies that have tackled some of these hard problems.

and they keep track of some of these interdependent interconnected risks and just help you see the entire picture of the risk landscape.

Raj Krishnamurthy (27:44.028)
And as a risk practitioner, I think you talked about this earlier in the store example, right, on the handheld device. Everybody has their own perspectives. And when you go talk to people, right, and even that is true even when you present to the management of the board, they have a feel.

of what the risk is. And I think you came up with, I saw that in a blog post recently that you did a great job of articulating a diagnostic framework as you have this conversation. Can you describe that for our users?

Tony Martin-Vegue (28:16.403)
Sure, so you walk into a boardroom and then you present ransomware risk is $50 million. The most likely value is 50 million. When it really hurts, it could be up to 100 million. And then somebody raises their hand and says, that feels too high to me. And it’s a really good.

Raj Krishnamurthy (28:36.249)
Ha ha ha ha.

Tony Martin-Vegue (28:40.883)
conversation to start to have. It can be frustrating because you’ve spent, you know, presumably months on this.

Raj Krishnamurthy (28:44.603)
You don’t get offended when they say that. You say, I’ve been spending three weeks on this stuff or four weeks on this stuff. And you say, it feels good or it slow. Do you get offended? OK.

Tony Martin-Vegue (28:55.653)
I never get offended by, I don’t get offended by anything in risk management. You just can’t, you can’t, you gotta, you gotta be able to have a thick skin and really stand behind your numbers and speak exact. So there’s a few different diagnoses for this. When someone says it feels high, it could be their bias and bias creeps in in a lot of different ways.

Maybe they felt that it should be a lot higher or it should be a lot lower and you need to ask them So what would a comfortable number be to you? And I’m not asking that question to change our risk analysis because we have to be very honest and ethical and data-driven I’m trying to get to the root cause of why it feels that way. So if it feels too high maybe they think that

They need to dig deeper into the data and understand some of the ripple effects. Well, we counted some of the legal ramifications of this, shareholder lawsuits, of extended outages, and that’s why it’s 50 million. And then they’ll get it. maybe they think it’s too low. OK, so why do you think it’s too low?

Hey, you didn’t count this system over here and they can teach you something. Maybe you need to go back and recalculate some of your data. So sometimes you got something wrong. Sometimes they’re not understanding it correctly. You need to really walk it through. You need to walk through some of the assumptions that you made. But I always like it when people say it feels too high or feels too low or that seems off because it means that they’re engaged and interested in the risk analysis results.

gives me an opportunity to dig in deeper and really try to have a good conversation about the nature of risk, which is all I’m really trying to do in the first place anyway.

Raj Krishnamurthy (30:52.092)
And how do you, from that qualitative aspect, how do you generate a very quantitative, maybe a scientific number or a range like an annualized loss expectancy or a single event loss expectancy? What is the thought process? How do you take from this bucket of things that you’ve got from people to this number?

Tony Martin-Vegue (31:13.501)
That’s a great question. So we start with the risk scenario, which is the asset that we care about, the threat actor that can act against that asset, and then the effect. What does the consequence look like when this happens? Now we have a risk statement, a good risk scenario of what we’re trying to measure.

Now you go out and start to measure things. of course risk, whether you’re qualitative or quantitative, it doesn’t matter. It’s the same basic equation. Risk is made up of two things. How often the bad thing happens.

And then when it does, how much it hurts. It’s those two things. It’s likelihood and impact. Let’s go out and measure likelihood. How often does it happen? You can talk to your incident response teams. Hey, have you seen ransomware here in the last five years? You can take a look at your competitors.

You can take a look at other companies in your similar technology stack or revenue or geographic location or employee size. If they’ve had a bad enough ransomware event, it’s going to show up in the news. So you can take a look at those and you start to build your frequency. So based on all the information that I’ve gathered, both internally and externally, we estimate that ransomware happens between, you know, one

to three times every 10 years, something like that. And now you start to look at the magnitude side. So how bad does it hurt when it actually does happen? And then it’s just measurements. You just break down a really big problem into measurable bite-sized pieces. It’s like Lego blocks.

Tony Martin-Vegue (32:56.337)
So incident response team, how many hours do you think you would spend on a ransomware event? Forensics team, how about you? Would you ever hire an external forensics team? you would? How much would that cost? the typical engagement is $30,000. Hey, legal team, do you think we could get sued for this? What does a typical lawsuit look like? And then you just work your way through it. It’s just these Lego blocks.

It’s a lot of conversations. And then once you’re done, you just basically calculate it and throw that into your risk model. And now you have an annualized number. Now you have a number of what ransomware could look like at your company.

Raj Krishnamurthy (33:40.752)
It looks like a lot of investigation, right? You’re more an investigator and sort of a detective role in uncovering a lot of. was a that’s a good one. Netflix released in open source the Risk Quant project, which essentially tries to provide you some framework for the loss expectancy determination.

Tony Martin-Vegue (33:46.831)
I always tell people you’re a detective, you’re a risk detective.

Raj Krishnamurthy (34:07.612)
Are there other projects in open source, Tony, that people can build their own models, bring their own models towards this quantification for cybersecurity?

Tony Martin-Vegue (34:16.774)
Yes, so starting at the easiest end, the most accessible, there’s some spreadsheets out there that you can just Google FAIR spreadsheets. So FAIR is a framework for cyber risk quantification. There’s a lot of frameworks that you can use. FAIR is just one of them. So you can download FAIR spreadsheets. You can also download other risk quantification spreadsheets.

There’s an author of a series of books around just risk management in general. His name is Doug Hubbard. So if you Google him, you can find his spreadsheets and those are great. And then moving on from that, there’s packages in R, there’s packages in Python. There’s also some web-based applications that are really good that you can start out with risk quantification.

But all of them have a few things in common. They all use Monte Carlo as the engine behind it because Monte Carlo allows us to simulate many futures, many 2027s. And they also always use the same basic risk equation. Likelihood times magnitude equals risk. And they all have that in common. So I would encourage your listeners

to just find what works for them. You could use fair. You don’t have to use fair. You can use anything you want. Just find what works for you, what works for your company.

Raj Krishnamurthy (35:45.594)
Got it. You also talk about the six levers that impact risk management. Can you describe that for us?

Tony Martin-Vegue (35:55.153)
Yes, so when you’re assessing or reassessing risk, what factors come into play? What influences risk? What moves risk up or down? And I think that a lot of us risk managers, and I’m guilty of this too, whenever somebody asks me, well, what moves data breach risk up or down? I only think risk mitigation. Hey, so we turned these five reds into yellows.

because we put all these projects into place or we mitigated $10 million of exposure. We reduced $10 million of loss exposure because we did all of these things. We got better incident response. We got better EDR. We got better detections. But there’s a lot more to that. You can move the frequency down through mitigation, but frequency can go up.

if your attackers get smarter or they get better resourced or they develop new tools or maybe one of your controls that worked yesterday doesn’t work today. I remember way back when we wanted to mitigate data breach or password compromise by creating long passwords.

So we increased the password requirement from six to 10 and introduced complexity. And that worked for a little while. And then we had to put an MFA. And MFA worked for a little while until there’s MFA bypass. So that’s a really good example of the attacker arms race where controls that worked yesterday don’t necessarily work today. On the magnitude side, how much does something cost us when it happens?

We can try to drive that cost down by putting in retainers or getting more efficient incident response or Doing a lot of things, but there’s also things that drive costs up An obvious one is inflation. I went through and updated all of my risk models When we had really bad inflation a couple years ago Why is that legal firms will charge us more? When an incident happens when they have to defend us in court?

Tony Martin-Vegue (38:14.92)
forensic to get retainers for friends outside forensics that cost more incident response that cost more so I had to go through and update all of that so identified six of them and then each six of those is broken out to broke it out into subcategories but those are just a few examples of both internal and external factors that can drive risk up and down and it’s a constant thing it’s a constant thing that happens

Raj Krishnamurthy (38:42.053)
Got it. So it looks like you are actually doing a lot of data mining to get to needle sort of in a haystack problem to get to what you want. And one of the questions that I have is that the traditional notion of compliance is that you start with operations and you end with audit, right? And you’re proving to some external body.

about the things that you promised that you would do. The question I want to ask you is that all that enriched data that you collect in compliance, whether it is about users and their security posture, about your systems and their security posture, how do you feed that back into the risk? Or do you see a path to feeding back into risk? What should happen there?

Tony Martin-Vegue (39:23.944)
100 % I I love my compliance teams and they’re my best friends Along with my my auditors and my governance people They are such key partners because If you just go back to that basic risk equation how often something happens and how much it hurts when it does who knows All the things that are happening around security controls

Raj Krishnamurthy (39:30.411)
Okay.

Tony Martin-Vegue (39:52.922)
or around the crown jewels, the assets that we care about, the things that we’re trying to protect. Compliance, compliance knows a lot about this, especially how much it hurts. So I’ll just keep going with the ransomware example. I can get my compliance team in a room. Ransomware event has happened. Just put yourself in those shoes. Let’s just pretend that it happened.

What are all of the compliance events, all the downstream things that can cost us, start costing us money? Well, if a ransomware event happened, that meant we’re probably out of compliance with these three things. And here’s the regulators that are going to be upset with us. And then you just start pulling threads. Okay, so which regulators? FTC and SEC and maybe some banking regulators. And now you have more information.

you can take that and bring it to your legal folks and get more information around that. Then I go back to the compliance team. So tell me about the controls that are working and maybe not working so well. And how does that influence risk? it’s just having all these conversations, writing it down, getting more information and going back to that basic risk equation and plugging that in and just getting better and better data every day. So.

I love working with compliance. They’re my best friends.

Raj Krishnamurthy (41:14.895)
Got it. And maybe it is a rhetorical question. Continuous controls monitoring is all about brushing teeth and doing your exercise daily, right? How much of this hygiene thing is important for risk management?

Tony Martin-Vegue (41:30.81)
I think it’s incredibly important. So it’s important in two ways. So we need to have really good hygiene in organizations, good security hygiene. If you don’t, you’re going to find your risk numbers go up. And I’ve measured it. I’ve seen it. And it’s very provable. If you can get the basics down, you can just see your risk exposure start to level out.

Of course it goes back to the six levers that influence risk. You need to stay on top of that and constantly be redefining what it means to be Hygienic what good security hygiene means you need to redefine that all the time because the world changes so fast The second aspect of that is you’d have to have really good risk hygiene Do you sit in a room and play pick a color?

You know, hey, what’s data breach? It’s red today. I’ve been in companies that have done that and it’s really frustrating because it’s not data driven. When you go out and measure risk, are you writing those things down? When you make a forecast about something, hey, I think data breach is going to be $70 million next year. After 2027 has happened, do you go back and check your forecast and see if it was accurate or not?

And if it wasn’t, do you try to figure out why? So that’s risk management hygiene. So there’s security hygiene and then there’s hygiene for yourself. You really need to be disciplined about it.

Raj Krishnamurthy (43:00.327)
That’s beautiful. But Tony, why do then many companies don’t do this? There seems to be a big chasm between walk and the talk. Why?

Tony Martin-Vegue (43:11.047)
So here’s another controversial take that I have. I think that we’re disincentivized to do it. We’re disincentivized to do it well and correctly. I think that there’s enormous pressure on security teams and risk management teams to move very quickly. And to do that, we take shortcuts. Those shortcuts are, what’s the bare minimum risk management program? We…

can put in place to pass an audit. And you have the auditing firms come in and they audit you on the same program that they built, that they sold us. And the bare minimum is going to be red, yellow, green, sitting in a room, playing pick a color every quarter. And I find that just really inefficient. It creates a system where we’re incentivized to do the bare minimum that doesn’t really work.

we’re incentivized to take our risk management activities and tie it to an audit instead of tying it to a business decision or a business objective. We do risk management after an event has happened and that’s what risk registers are typically in companies. There are a list of things that have already happened. You shouldn’t be doing that. We should be doing risk management on things that haven’t happened yet but can really negatively impact

our value proposition. And that’s what the risk register should be instead of things that have happened. But doing it the other way passes an audit. It really does. So in some ways, it’s really depressing. If I think about it, I get bummed out because we could just be doing so much more to become business enablers. But we’re stuck. I feel like we’re stuck. And I want us to get unstuck.

Raj Krishnamurthy (45:02.811)
If I’m the CISO listening to this podcast right now, Tony, what are the two or the three things that I can do to change that behavior?

Tony Martin-Vegue (45:14.021)
I’m not even going to mention risk quantification because I think that there’s some foundational things that we need to put in place before we can think about risk quantification. And the first one is CISOs should expect more from their risk management teams. Go look at your risk register. And if it’s a list of findings, a list of gaps, a list of control deficiencies, and a list of incidents, you’re recording things that have already happened.

you should challenge your risk management teams to really try to tie their risk management activities to business objectives and things that haven’t necessarily happened yet. And then you can start to have much better decisions around that. The next thing is how are you assessing your risk? Do you sit in a room and play pick a color on a risk matrix?

Tony Martin-Vegue (46:35.778)
Those are the two things I would recommend to see so as to go out and start doing start demanding more. Don’t settle for bread crumbs. Demand more from your risk management teams.

Raj Krishnamurthy (46:46.564)
God, no, that is beautiful. What is the role of generative AI and large language models, the way in which companies adopt and use them? What is that effect? What is the effect of those technologies on risk management?

Tony Martin-Vegue (47:05.208)
I love that question because it’s like an onion, right? You can start to peel it and there’s so many layers and the layers make you cry. So I want to answer the question by just going around a little bit with the onion super quick. So I see a few different risks of AI. And the first risk is using it.

There’s privacy issues. We’re increasing the chances of unintended data disclosure, of accidentally plagiarizing or really bad results from AI. And that’s a huge risk. It really is. Another risk is not using AI. AI is such a huge unlock for productivity.

For the risk analysts, we can do so much more today than we were able to do just a couple years ago with generative AI. And it’s amazing. I use Geni every day, but you have to use it safely and smartly. So there’s a risk of not using it. I think that as security teams, if we’re not at least having these conversations, we’re going to be left behind by our competitors very, very soon, especially for risk managers.

So that’s the second risk. And then the third risk is using it, integrating it into our workflows. And then it goes away. And it could be something really small like an outage. AI has an outage for eight hours. And now all of a sudden we found out that we’re over reliant on it. We don’t know how to do this stuff with traditional methods and we’re stuck. Or it could be something really big where…

AI is regulated out of existence. And I don’t want to sound too paranoid, but what if it’s like a Terminator Skynet-like scenario where we, as humanity, we get really scared of it and we decide to pull the plug. And now it feels like we’re going back to Stone Age technology. So I don’t want to get paranoid, but we have to have these conversations about these things as we start to rely on it.

Tony Martin-Vegue (49:19.492)
So going back to the second one, the risk of not using it, I just want to dig in really quick of how I use it. You can use it for scenario building. I was talking earlier about the asset, the threat, and the effect. I have a really short prompt that can do that. You can point it at really anything, a Jira ticket or the Verizon Data Breach Investigations Report, which is a really good report put out by

Verizon, you can point it at that, you can point it at an internal audit finding or an SEC filing, and it can give you a list of risk scenarios. So I really encourage your listeners to explore that. And if anyone wants the prompt, just find me on LinkedIn, I’ll give it to you. It’s also really good at research.

finding the frequency and the magnitude of these things that are happening, events that are happening. You have to double check everything because AI hallucinates and it can lead you down a wrong path. So double check, but it can shave hours, days, weeks off of your research efforts. I think CRQ is more accessible now than ever because of these things.

Raj Krishnamurthy (50:39.192)
How do you say that?

Tony Martin-Vegue (50:42.766)
You can speed up your quantitative risk assessment by a factor of 100 by harnessing generative AI to build risk scenarios, to do your research, to parse PDFs, to parse incident response logs.

Raj Krishnamurthy (50:58.34)
Beautiful. This investigation that you are doing with people, you can do it much more effectively, even offline is what you’re saying. That’s beautiful. Especially within cybersecurity, how do you view GR and C? And the reason I’m asking that question is,

In many cases, when I use the word GRC, most people hear the word compliance. They don’t hear GNR, right? So how do you see this integrated approach of GRC, especially for cybersecurity?

Tony Martin-Vegue (51:32.262)
It’s just incredibly important. If you work in a security team right now, or you work in security team adjacent and you don’t know your GRC team on a first name basis, please go fix that tomorrow. Go get to know them because they see a different point of view from not only security operations, but the things that can really hurt your company. The governance people.

The compliance people and the risk people they just have a different point of view It’s not better or worse, but it’s just a different point of view from security operations and I think understanding that understanding the work that they do Understanding the types of things that they analyze and assess can really help you get a much

well-rounded view of not only just your company, but the things that threaten the company. And that’s all, that’s why we’re all here, right? We’re all here at our organizations to see the company succeed. Security teams don’t exist to secure the company.

That’s another hot take of mine. We don’t exist to secure the company. We exist to help the company succeed in a secure way. Move fast, safely. Execute our value proposition safely. How do we do that? And your GRC team knows how that’s happening. So please go make friends with them.

Raj Krishnamurthy (53:01.914)
I love it. I love it. I think that’s a very clear call for action for all those engineers particularly listening to this. I would absolutely echo that. What do you see as the current state of GRC tools? I think you went into a great depth about the analysis, the investigative aspects of risk. But how good are today’s GRC tools to do what you’re saying?

Tony Martin-Vegue (53:24.845)
It is so exciting right now. I think it’s such an exciting time to be in this field. It’s such an exciting time to be alive just in general, I think, because technology is enabling so much. You if you go back 15 years ago, there were only a handful of GRC tools out there.

And they didn’t do it very well I’m not gonna name any but there were a couple that just caused me so much heartburn because If you wanted to do anything in GRC, you had to use these tools. You had to use one of these vendors and it was so hard And now they are they are great they with just the interface is so clean and clear and the API’s You can suck in so much data from so many different sources and get instant analysis

instant feedback, instant insight, and generative AI. The companies that are integrating gen AI into some of these GRC products are really pulling ahead, especially on the compliance space. I’ve seen this and on the risk management side, the CRQ space. We could do something in minutes, which used to take us, I’m not joking, months to do.

Raj Krishnamurthy (54:38.642)
I think it’s super exciting. We are approaching the end of the segment, Tony, and I want to ask you two questions. One, for the person who’s graduating from college, or maybe even in college, who wants to focus on cyber risk or cyber risk quantification.

What would the advice be? What should they focus? Where is the puck moving? What should they focus on today so that they become valuable in the next two years?

Tony Martin-Vegue (55:11.845)
So if you’re graduating now, the number one thing I wanna tell you is continue your learning journey. Be a curious and constant learner. I’m still a learner. I learn something every single day and I read every day constantly, constantly trying to keep my skills fresh and I think that’s going to be really important.

I think that a lot of the tools and skills that you are going to need to survive and thrive in 2020s and 2030s in this field, you’re gonna find inside of security and inside of CRQ, but also outside.

You need to be able to take a lot of the lessons learned and a lot of techniques and tools from other fields like actuarial science or behavioral economics or just economics in general. A lot of the problems that we face in cybersecurity are not really unique to cybersecurity. They’ve been solved in the past in fields like actuarial science and economics and stuff like that. So if you start.

looking at those fields for how they solve things, you’re gonna have your mind just opened up to a completely different field and a completely different way of doing things and you’ll be able to apply those lessons here and you will find yourself being invaluable at any company if you can do that.

Raj Krishnamurthy (56:38.167)
And are there community groups, open source groups that I can be part of that can help me understand this better and also contribute?

Tony Martin-Vegue (56:47.087)
There’s two great societies, I think, that your listeners can join. And the first one is called CIRA, the Society for Information Risk Analysts. They’re a vendor neutral and model neutral risk quantification society. So as I mentioned earlier in the podcast, FAIR is really popular as a framework.

But there’s other models that aren’t fair and they’re great. So if you join CIRA, you can learn about those. And there’s a conference, an annual conference that’s coming up in September. It’s very reasonably priced. There’s virtual and in-person options. So your listeners can check that out.

And the other one is the Fair Institute. And this is a nonprofit, a society that’s built up around the Fair framework, which is a CRQ framework. And it’s also awesome. I love it. There’s a great community. There’s also a conference that’s coming up in September.

September, October, I can’t remember. But if you Google FairCon, you’ll get the date. Yeah, so the conference is called FairCon. And it’s also a very vibrant community of CRQ professionals.

Raj Krishnamurthy (58:02.125)
Got it. And I want to call out that you were a board member of SIRA, right? Are you still a board member? For a moment. OK. And then you currently are a co-chair at the San Francisco chapter for the Farr Institute.

Tony Martin-Vegue (58:09.284)
I former board member. Yes.

Tony Martin-Vegue (58:19.138)
Yes, we have a local chapter here and we have local meetings. So if you’re in the Bay Area, please contact me and I’ll give you the information for our next chapter meeting.

Raj Krishnamurthy (58:29.043)
That’s beautiful. Thank you. I want to end with this question. I want to give you the next 60 seconds or so, Tony, to shout out the book. I see a bookstore behind you. Books you read, people that have influenced you, your mentors, whatever you want to talk about, right? Something that you want to shout out for who made Tony who he is today.

Tony Martin-Vegue (58:52.484)
Oh, great. Thank you for that question. I love that. Okay, so I’m going to be vulnerable here for just a moment here. As I mentioned earlier, this stuff doesn’t come easy to me. It really doesn’t. And I keep this on my desk. It’s called probability without tears. And the reason why I have Yes. probability without tears by Derek Roundtree.

Raj Krishnamurthy (59:13.281)
Okay. Can you show the book? Can you show the book one more time, please? Okay, got it.

Tony Martin-Vegue (59:22.732)
And the reason why this is on my desk is because a lot of times I don’t understand this stuff. So if you don’t understand it, you don’t understand probability or conditional probability or Monte Carlo, you’re the same as me. I don’t either sometimes. So I’m constantly reading a beginner’s book. So this.

That’s really important to me. So that’s the number one thing for your listeners is you can check out that book or also, you know, don’t be shy about picking up beginners books on some of these concepts. One of the most foundational books that I’ve read is called The Failure of Risk Management by Doug Hubbard. And that is an amazing book. It opened my eyes up to.

better risk management techniques and how to go out and measure things in a way that you can feel proud of that really helps your company. And the last book I’ll plug is, I must have read this book so many times, the pages are falling out. It’s called Measuring and Managing Information Risk by Jack Jones and Jack Freund. And that’s actually a fair book.

And it really made me who I am today. It is it’s an amazing book and it taught me so much about CRQ and all these books are They’re available anywhere books are sold and they’re all very affordable. You can pick them all up use now

Raj Krishnamurthy (01:00:46.051)
And all of those are fantastic people, like Jack Freud, Jones, and Doug. yeah. And so Tony, this was a fascinating conversation into the world of cybersecurity risk management and risk quantification, I should say. Super thanks for joining us here. And thank

Tony Martin-Vegue (01:00:51.458)
Yeah, they’re great guys. Yeah.

Tony Martin-Vegue (01:01:07.534)
Thank you for having me Raj. I really appreciate you having me on and letting me talk for so long. And thank you to the listeners.