•
Drizz raises $2.7M in seed funding •
•
Featured on Forbes
•
Drizz raises $2.7M in seed funding •
•
Featured on Forbes
Key takeaways
Test coverage metrics measure how much of your software has been exercised by tests. That's definition. The problem is that most of what teams measure tells them almost nothing about whether their software works.
You can hit 90% line coverage with tests that verify nothing. You can hit 100% and still ship bug that costs your company money. Every post on this topic says "coverage isn't quality" and then goes back to explaining how to hit higher coverage numbers.
Most coverage metrics measure activity, not quality.
Once teams separate code from test coverage, next step is deciding which test coverage metrics actually matter. Because most of them don't.
Metrics that look useful but usually aren't:
Each of these gets tracked because they're easy to measure. Each of them gets gamed when it becomes a goal.
This is Goodhart's Law in action. When a metric becomes a target, it stops being a good metric. Set an 80% line coverage target and your team writes tests that hit lines without asserting behavior. The dashboard goes green. Production bugs don't decrease.
The classic study by Inozemtseva and Holmes at Waterloo tested this directly. They analyzed 31,000 test suites across 5 open source projects and found correlation between code coverage and bug detection was 0.4 to 0.5, dropping to under 0.3 when you control for suite size. Coverage predicts something, but not much.
Every SERP article lists these without explaining what they actually measure. Here's real distinction.
Statement coverage
Percentage of statements executed at least once.
function calculatePrice(item, discount) {
let price = item.basePrice;
if (discount > 0) {
price = price * (1 - discount);
}
return price;
}One test with discount = 0.1 hits all 4 statements. Statement coverage: 100%. Easy to game.
Branch coverage
Percentage of decision branches (both true and false paths) executed.
Same function. One test with discount = 0.1 hits true branch. Branch coverage: 50%. You need a second test with discount = 0 to hit false branch.
Function coverage
Percentage of functions called at least once. Easier to hit than either of above.
Condition coverage
Every boolean sub-expression evaluated as both true and false.
if (user.isPremium && item.inStock) {
applyDiscount();
}Condition coverage requires 4 test cases: (T,T), (T,F), (F,T), (F,F). Statement coverage requires 1. Big difference in test rigor.
Path coverage
Every possible path through function. Explodes combinatorially in real code. A function with 10 branches has up to 1,024 paths. Nobody achieves 100% path coverage on non-trivial code. It's a theoretical ideal, not an operational goal.
What most teams measure vs what matters:
If your CI dashboard shows "coverage" without saying which type, it's showing statement coverage. That's least useful of four practical types.
Total codebase coverage tells you about all code. Change-based coverage tells you about code that just changed in this PR.
Which one matters more?
How change-based coverage works:
Why this works better than total coverage:
Tools that support this: Codecov, SonarQube (with "new code" definitions), GitHub's built-in coverage reports on PRs, and Jest's built-in coverage thresholds for JavaScript teams.
Broader QA metrics include a few test coverage metrics, but coverage deserves its own conversation because it's metric most likely to be gamed and least likely to be understood.
These are different metrics measuring different things. Teams conflate them and it causes real problems.
Code coverage measures what fraction of your code is exercised by tests.
Requirements coverage measures what fraction of your specified requirements have at least one test case validating them.
You can have 100% code coverage and 60% requirements coverage. Every line runs during tests, but 40% of specified behaviors don't have a test that explicitly validates them.
You can have 100% requirements coverage and 45% code coverage. Every spec has a test, but implementation has code paths that never get triggered by those tests.
Which matters more depends on what you're trying to prove:
Most engineering teams optimize code coverage because it's automatic. Most product teams care about requirements coverage because it maps to features. QA needs to track both and understand where they diverge.
Not all code needs same coverage. Setting one target across whole codebase is why teams either over-test unimportant code or under-test important code. Trimming suites without watching test coverage metrics is how teams end up with fast pipelines that miss real bugs.
Tier 1: Life-critical, financial, security-critical code
Payment processing. Authentication. Cryptography. Medical calculations.
Tier 2: Core business logic
Checkout flows. Order management. User account handling. Core feature logic.
Tier 3: Standard features
Search, filters, notifications, non-critical UI logic.
Tier 4: Cosmetic, disposable, or experimental code
Marketing landing pages. A/B test variants. Prototype features.
Setting a flat "we need 80% coverage" ignores that some code needs 90% and some needs 20%. Tiered coverage lets you actually invest where it matters.
If you only measure one coverage metric, this is one.
How mutation testing works:
What mutation testing catches that line coverage doesn't:
Consider this test:
test('calculates total', () => {
const result = calculateTotal([10, 20, 30]);
expect(result).toBeGreaterThan(0);
});Line coverage: 100%. Branch coverage: 100%. This test would pass even if function returned 1. Or 42. Or -50. It asserts on nothing meaningful.
Mutation testing catches this. If you mutate function to return 1 instead of actual total, test still passes. That's a surviving mutation. Score drops. You know test is weak.
Tools by language:
Why teams don't use mutation testing:
None of these are fundamental problems. They're setup investment. Teams that make investment ship measurably better software. Teams that don't stay on vanity metric treadmill.
Mobile testing has coverage dimensions web doesn't:
Device coverage
OS version coverage
Screen size coverage
Feature flag combinations
Network condition coverage
Mobile coverage metrics need to include these axes explicitly. Reporting "80% code coverage" for a mobile app without device coverage numbers is misleading. Your code might be tested. Your product isn't.
The paths users actually take matter more than paths they don't.
Critical path coverage
Error path coverage
How to measure:
Most teams find critical path coverage is 60-80% and error path coverage is 20-40%. That's why bugs users hit aren't ones QA catches.
Research on what predicts test suite effectiveness:
If you're measuring bottom two and not top four, your dashboard is telling you a story that doesn't match reality.
The metrics teams should actually track:
These are harder to measure than "we're at 82% coverage." That's point. Anyone can hit 82% coverage. Not everyone can ship stable software.
Coverage metrics for mobile teams are directly affected by test framework choice. Selector-based frameworks (Appium, XCUITest, Espresso) often produce coverage numbers that don't match reality because flaky tests inflate false positives (tests marked as passing when they didn't actually verify anything).
What flakiness does to coverage metrics:
Drizz tests are written in plain English:
Vision AI reads screen instead of relying on locators. When tests run reliably enough to trust (typical Drizz suite pass rates run 94-97%), coverage metrics they contribute to actually reflect verified behavior.
Before, on Appium:
After, on Drizz:
The framework choice determines whether coverage metrics on your dashboard match coverage you actually have. A reliable framework doesn't just save maintenance time. It makes metrics honest.
Test coverage metrics only tell you something useful when tests they measure actually verify what they claim to verify. Get that layer right, and metrics start meaning something. Skip it, and you're managing a dashboard, not shipping quality.
Statement, branch, function, condition, and path on code side. Requirements, critical path, and mutation score on behavior side.
Depends on code. Payment and auth need 90%+ branch coverage. Marketing pages might need 30%. Flat targets are wrong.
Yes for line coverage. Nearly impossible for path coverage on non-trivial code. Meaningful coverage matters more than percentage.
Code coverage measures which lines/branches ran. Test coverage is broader, including requirements coverage, user flows, and error path coverage.
Mutation score and change-based branch coverage predict quality better than total line coverage. Most teams don't measure first two.
Code coverage tools work on unit tests. UI coverage tracks critical paths, device matrix, OS versions, and error states separately.
©Drizz 2026 • All Rights Reserved
