AI Can Write Patches. So Why Does Vulnerability Response Never End?

— What 20 years of vulnerability management reveal about the real wall: change management

The technology for finding vulnerabilities has advanced dramatically over the past 20 years.

And now, AI is making vulnerability discovery faster still.

For the first installment of our series "Where Defenders Stand Now," we spoke with Ryoji Kanai, who has spent many years developing cybersecurity products and, as CTO, still oversees technology across the board as well as the product business. We asked him what has changed over the past 20 years, and what challenges defenders face now, in the age of AI.

"What hasn't changed in 20 years is the cat-and-mouse game"

Hiradate: Twenty years ago, just finding a vulnerability was enormously valuable, wasn't it? Find one and you were a hero.

But the situation has changed a lot, and now too many vulnerabilities are being found. It's not only the companies using the software. Even the people who assess and sort through the vulnerabilities being found can no longer keep up. Add the prioritization and remediation that come after, and the system as a whole can't handle the volume anymore.

I think the process from finding a vulnerability to responding to it has changed dramatically over these 20 years. At the same time, how much has the core of attack techniques and defensive technology actually changed?

Kanai: Twenty years ago was when what we would now call targeted attacks were starting to become a problem around critical infrastructure and government.

Back then, we didn't even really understand what was happening. You would analyze the malware and work out what techniques were being used. Through that kind of work, I think attack methods and the overall picture have been mapped out fairly thoroughly.

By around 2006, I'd say the core security technologies, including pattern matching and behavior-based detection, had more or less all appeared.

Of course, there have been innovations such as fuzzing, and recently AI is a very big change.

Beyond that, it has been a cat-and-mouse game between defensive technology and the attack techniques that evade it.

Once there is antivirus, firewalls, and IDS, techniques to evade them appear. When behavior-based detection comes along, something emerges to get around it. These days, attackers disable EDR, evade EDR, or go after network devices where EDR can't be installed in the first place.

Against today's threats, which existing technologies do you revisit, and how do you combine them? That combination itself is hard, and I think that's where innovation comes from.

Because countermeasures are never going to be 100% perfect.

So is the real bottleneck today still "finding" vulnerabilities?

"Change management can't keep up with how often we have to patch"

Hiradate: Patches not getting applied has been a problem for 20 years.

When it comes to actually applying fixes for the vulnerabilities that are found, what is the biggest bottleneck?

Kanai: To begin with, the number of vulnerabilities has been rising steadily since well before AI. So the sheer volume that has to be dealt with keeps growing.

But when it comes to applying patches, I really think it's change management.

You have to sort out the dependencies and confirm that everything still runs normally. There are reviews and approval processes for that, too.

In the end, change management can't keep up with the number of times you have to patch. I think that's what it comes down to.

A patch doesn't remove only the vulnerability

Hiradate: For my part, I've spent a long time on the side that uses and operates software, watching how hard it is to apply patches.

From the side that builds the products, though, what makes it difficult to get from discovering a vulnerability to shipping a fix?

Kanai: Ideally, you would fix only the vulnerability, with no effect on other functions and no change to the specifications.

If you could say, "Apply this and only the problem gets fixed," it would be easy.

But real products aren't that simple. A regular release may include new features, and if the problem lies in the design itself, closing the vulnerability may require changing the specifications. Even with OSS and middleware, a patch-only version isn't always released. You may have to move to the latest version, and as a result of that update, a lot of other things get upgraded along with it and the behavior changes.

So verification is something you can't skip.

Whether to fix a defect is also a business decision

Hiradate: Does your company also have standards and governance for software release management?

Kanai: We do. But it isn't as simple as shipping a fix for every version each time a defect is found.

Fixing and releasing has a cost in itself, and how far you go is also a business decision.

If development of the next version is underway, one option is to put the fix there and ask customers to update. If, instead of fixing it only in the next version, you also release a fix for the current version, and then for the previous version still under support, you end up releasing the same fix on three lines.

Then each of those needs testing, too.

The cost of the response simply multiplies.

At our company, too, there have been times when we said, "This will be fixed in the next version." How far back to support older versions depends on each product's circumstances. It isn't standardized.

What does Patching as a Service actually solve?

Hiradate: Recently, services such as Patching as a Service have appeared that use AI to analyze source code, find vulnerabilities, and even prepare the fix patches.

That means producing a proposed fix can now be automated to a considerable degree.

On the other hand, whether that patch actually gets applied is still left to the customer's own change management and verification processes.

If vulnerabilities had been left unaddressed simply because remediation work couldn't keep up, I think this would make a big difference.

But if the bottleneck was "we don't know whether it's really safe to apply this patch," then the hardest part still remains.

So in the end, what does Patching as a Service solve, and what doesn't it solve?

Kanai: If applying a patch removed only the vulnerability and affected nothing else, and that were guaranteed, you could just apply it.

But there's no such guarantee. Nobody guarantees that. And a patch written by AI doesn't come with that guarantee either.

With AI in particular, quality standards differ from company to company, and you can't always see fully how far, and with what kinds of tests, a patch has been tested.

Whether a business system will keep running normally after you apply a new patch or version is, in the end, something you have to verify in advance.

Of course, there's also the option of taking the risk and applying it as is. But fundamentally, I think you need to verify that things still work.

What Patching as a Service solves is "the cost of producing patches."

But AI being able to write fix code is not the same as being able to apply that change safely to production.

Figure 2: Where has the bottleneck moved in 20 years? — Figure 2 Where has the bottleneck moved in 20 years?

Even if AI can find vulnerabilities and write the fix code, that doesn't mean the change can be applied safely to production.

So can we hand even that verification and change management over to AI?

In Part 2, we look further ahead, taking test automation and Human in the Loop as our starting points.