Automated Test Coverage for Legacy Applications

Overview

A security patch only protects you once it is in production, and for most legacy applications that is where it stalls. The update is ready, but no one can prove it won’t break something, so it waits for a manual regression test or the next release. Automated tests close that gap. A test suite that exercises what the application does today proves that the software that worked yesterday still works after the patch, so the patch can ship the same day.

We have taken on dozens of applications written by teams who are long gone, in different languages and different generations of those languages. Building their test coverage is how we make them safe to patch.

Why Tests Decide How Fast You Can Patch

We have argued since 2017 that a critical patch has to reach production within minutes or an hour, and that fully automated testing is the only way to get there. In 2023, Frank Rietta made the case to security executives in GRC Outlook: How to win the race with hackers when new vulnerabilities are publicly disclosed. It starts from the 2017 Equifax breach, which Equifax confirmed came through a known Apache Struts flaw it had failed to patch, and concludes that you need to patch within hours, and that extensive automated tests are what give you the confidence to deploy.

In 2026, an attack proved it. On July 29, Ruby on Rails shipped a fix for a remote code execution flaw in ActiveStorage, CVE-2026-66066, scored 9.5 out of 10 by that evening. That night we updated each affected client application, ran its full automated test suite locally and in CI, and deployed only once every suite passed. For one State government client, the patch was in production at 11:09 PM. The first exploit attempt against that application arrived at 7:10 AM the next morning, eight hours and one minute later, before business hours. It failed, as did every attempt that followed through August.

Without those tests, the patch would have waited for a manual regression check and a morning approval, and the application would have been unpatched when the attack arrived. For the full timeline, see Government Rails Site Hit Hours After CVE Patch.

Start with a Test Coverage Assessment

The first step is a Test Coverage Assessment of one application, starting at $2,500. You receive:

• A coverage baseline: which tests exist, what code they reach, and whether they run in CI.
• The application’s change points, the code that patches and upgrades are most likely to touch, ranked by risk.
• A test plan for covering them, in priority order.
• The first smoke tests and characterization tests around the highest-risk change point, merged into your repository.

Larger applications cost more, and we confirm the price on the scoping call. From the assessment, we quote building out the tests, or your team can carry out the plan with our coaching.

Where This Fits

Test coverage is one part of our vulnerability management and remediation solutions:

• CVE monitoring finds the flaw in software you depend on.
• Test coverage proves the patched application still works.
• Secure CI/CD pipelines run the tests and the security scanners on every build.
• Our application maintenance and remediation program ships the fix to production, and keeps shipping them.

How We Build Coverage

Some applications start with truly no effective test coverage. Then we crawl before we walk, and walk before we run:

1. Crawl. Smoke tests prove the application starts, serves its main pages, and completes its most important workflows. They are quick to write, and they catch the update that breaks everything.
2. Walk. Characterization tests pin down what the code does today around the change points, so a patch or upgrade that changes behavior fails a test instead of reaching production.
3. Run. New code is written test first, the suite runs on every build, and it becomes trusted enough to deploy on.

For each change, we follow the legacy code change process from Michael Feathers’ Working Effectively with Legacy Code, the process the 2023 article recommends:

1. Identify change points. Where patches, upgrades, and fixes will land, starting with the code that would hurt most to break: authentication, access control, data access, and the paths that framework and library upgrades change.
2. Find test points. Where a test can observe what that code does.
3. Break dependencies. Where code is too tangled to test, we break its dependencies at seams so it can be tested.
4. Write tests. Characterization tests record what the code does today, before anything changes.
5. Make changes and refactor. With tests in place, we make the change and improve the code in small steps, following Robert C. Martin’s Clean Code: A Handbook of Agile Software Craftsmanship, so the next change is easier and safer.

Three more practices keep coverage growing:

• Test what must not happen. We write abuser stories for the threats the application must block and turn them into negative tests, so a later change can’t quietly remove a protection.
• Test first for new code. New features and fixes are written test-driven, the discipline Martin teaches in Clean Code: the test first, then the code that passes it.
• Run on every build. The suite runs in your CI pipeline, so every change is checked the same way.

Our goal is an application that can be deployed to production without human intervention when its test suite passes. Most legacy applications aren’t there yet. We help you get closer with every release.

Review and Coaching

If your developers will keep working on the application, we review the tests they write and coach them on these techniques as we go, so coverage keeps growing after our engagement ends. For structured training, see our developer security training.

Languages and Test Frameworks

We build tests in whatever your application is built on, including legacy Java applications on Apache Tomcat, Ruby on Rails, Python, and JavaScript or TypeScript. We work with the test frameworks your application already uses, such as JUnit, RSpec, Minitest, and pytest, and add browser tests with Selenium for the workflows your users depend on. If your application is built on something else, ask us.

AI-Assisted Test Writing

With your authorization, we use AI assistance to draft tests faster, which saves time and money. You decide where your code is processed: with our contracted U.S.-based AI providers, on offline models, or exclusively within your organization’s own AI infrastructure. We can also work without AI assistance, which may increase the timeline and cost of the engagement.

Before any AI is used, we analyze your repositories for secrets and other sensitive data, in the latest code and throughout the commit history. A characterization test records what the code does, including any bug it has, so our team reviews every test before it is merged and decides whether the behavior it pins down is correct.

What You Receive

• From the assessment: a coverage baseline, the change points ranked by risk, a test plan, and the first smoke and characterization tests.
• Automated tests merged into your repository and running in your CI pipeline.
• Review and coaching for your developers, if you want it.
• A path toward deploying automatically when the suite passes, through our application maintenance program.