Your actuarial report arrived. The number is different from last year, and the forty-page appendix does not quite explain why. Or the number did not change, and you expected it to, because you know your claims operation shifted. Either way, you are looking at a projection and wondering: could something be wrong with it?
The answer is almost certainly yes, in some dimension. Reserve estimates are not measurements. They are projections built on assumptions about how your claims will behave in the future, calibrated from how they behaved in the past. When the past stops being a good guide to the future (because your operations changed, your claims mix shifted, or your data has problems), the projection drifts. The question is not whether the number is perfect. It is whether the actuary identified the ways it could be wrong and accounted for them.
That is what a diagnostic review does. It is the difference between an actuarial analysis that mechanically runs triangles and one that asks, before selecting a single development factor, whether the data still supports the assumptions behind it.
The stability principle: what a good triangle looks like
Every development technique starts from the same premise: if the environment that produced your claims has been stable, then the development pattern from historical experience should be a reliable predictor of future development.
Friedland frames this concretely (p. 65): in a stable environment, the values at a given maturity age should be roughly consistent across accident years when you look down any column of a loss development triangle. If your 12-to-24-month reported development factor has been between 1.30 and 1.40 for six consecutive accident years and then jumps to 1.65 for the most recent year, that jump is a signal. Something changed.
The diagnostic question is: what changed? Is the 1.65 telling you that losses genuinely emerged faster in the most recent year, or is it an artifact of a case reserve practice change, a TPA transition, or a shift in claim mix? The answer determines whether the actuary should ride the historical pattern (treating the 1.65 as noise) or adjust for a structural change (because the old pattern no longer applies).
This is the core logic of diagnostic review. Before selecting development factors, the actuary should look at the triangle expecting to see the effects of known operational changes, and investigate when those effects are absent or when unexpected deviations appear.
Factor selection is judgment, not arithmetic
A point that most reserve reports gloss over: the selection of development factors from a triangle is inherently subjective. Two qualified actuaries working from the same triangle can produce different selected factors, and both selections can be defensible.
Friedland addresses this directly (p. 89): the selection process involves judgment, and different actuaries will make different selections, sometimes with material impact on the projected ultimate. This is not a failure of the method. It is a feature of any estimation process that requires interpreting noisy data. But it means that the number in your report is not the answer. It is one defensible answer among several, and the quality of the analysis depends on how well the actuary justified the choices.
For a buyer, this has a concrete implication. If your actuary selected a five-year weighted average for the 12-to-24-month factor but the most recent three years show a clear upward trend, the weighted average may understate the development. A different actuary might select a three-year average and produce a materially higher ultimate. Neither is wrong in isolation. The question is which selection is more consistent with what you know about your claims operation.
This is why diagnostic review matters more than method selection. The method is a framework. The diagnostic review is what makes the framework’s output match reality.
The failure-mode matrix: mapping operational changes to reserve errors
Friedland’s diagnostic framework can be distilled into a matrix that maps specific operational changes to specific consequences in the triangle. Understanding this matrix gives you a structured way to think about what could be wrong with any reserve estimate.
The matrix has four dimensions:
- What changed in your claims operation
- Which triangle is affected (paid, reported, or both)
- Which direction the error runs (overstated or understated ultimate)
- What adjustment corrects for it
Here is a simplified version covering the most common failure modes for self-insured programs.
Case reserve strengthening. Adjusters started posting higher initial reserves (after a new claims director, a regulatory audit, or a deliberate adequacy initiative). The reported triangle is affected: inflated development factors in recent diagonals overstate the projected ultimate. The paid triangle is unaffected. The correction is the Berquist-Sherman case adjustment, which restates historical case reserves (also called case outstanding) to a consistent adequacy level (Friedland, p. 283).
Case reserve weakening. The opposite: adjusters are setting lower initial reserves, often because of caseload pressure or reduced experience. The reported triangle understates the ultimate for affected years because development factors look artificially low. The same correction applies, in the other direction.
Settlement speed-up. Claims are closing faster than the historical pattern (because of a TPA initiative, a litigation management program, or a change in settlement authority). The paid triangle is affected: paid development factors increase in recent diagonals, and the paid chain ladder overstates the ultimate unless the actuary recognizes the shift. The reported triangle is less affected because incurred losses are relatively stable through the life of a claim. The correction is the Berquist-Sherman disposal-rate adjustment, which restates the paid triangle on a constant-closure-rate basis (Friedland, p. 287).
Settlement slowdown. Claims are closing more slowly. The paid triangle understates the ultimate because less has been paid at each maturity. This commonly follows TPA transitions (processing delays during onboarding), staffing shortages, or litigation expansion. The same disposal-rate adjustment applies.
Change in claim mix. The composition of your claims shifted (more litigated claims, a new exposure type, geographic expansion). Both triangles are affected because the blended development pattern no longer represents the current book. The correction is segmentation: build separate triangles for distinct claim types and project each independently. Friedland recommends reorganizing data before reaching for quantitative adjustments when the operational change allows it (p. 283).
Change in retention or limits. If your self-insured retention increased between policy years, newer accident years can develop to higher amounts than older ones. The historical pattern from lower-retention years cannot predict development at the higher retention. Both triangles are affected. The correction is to analyze by policy year rather than accident year, or to cap claims at the prior retention and analyze excess development separately. For more on how retention structures affect the reserving problem, see Self-Insurance, Captives, Large Deductibles, and SIRs.
For a detailed walkthrough of how to diagnose the three primary drivers (case adequacy, payment patterns, and claim mix) using data your TPA already produces, see What’s Actually Driving Your IBNR Higher?.
The diagnostic conversation: what should happen before factor selection
Friedland describes a diagnostic process that should occur before the actuary selects any development factors. The actuary should meet with claims management, learn what has changed in the claims operation during the experience period, and then run the triangles expecting those changes to show up in the data.
This is the piece most self-insured buyers never see. The triangle work happens in the actuary’s office. The claims conversation, if it happens at all, is often cursory. And the diagnostic step (looking at the triangle through the lens of known operational changes) is the step most likely to be compressed or skipped when the review is a routine annual engagement.
When that step is skipped, the actuary is reading the triangle blind. A jump in the 12-to-24-month factor could be real development, case strengthening, or a TPA transition effect, and without context from the claims operation, the actuary has to guess. The guess is usually to average over it, which is exactly the wrong answer if the jump reflects a structural change.
The self-insured amplification effect
Everything in the failure-mode matrix is amplified for self-insured programs, for one reason: thinner data.
A carrier with 10,000 claims per accident year has enough volume that individual claim variation averages out. The development factors are stable, the pattern is credible, and a single outlier claim barely moves the selected factor.
A self-insured program with 200 claims per accident year has no such cushion. A single large claim can move the age-to-age factor for an entire development interval. A case reserve practice change affecting ten claims can shift the reported development pattern for the year. A TPA transition that delays twenty closures can make the paid triangle look like claims stopped developing.
This means diagnostic review is not optional for self-insured programs. It is the primary quality control on the reserve estimate. The chain ladder will always produce a number. The question is whether that number was checked against the reality of your claims operation before it was reported. For more on the leverage problem that thin data creates, see the chain ladder article’s discussion of highly leveraged CDFs.
For programs where broad IBNR represents a large share of the total reserve, the diagnostic stakes are even higher. A development factor distortion on a program where 60% of the reserve is IBNR produces a much larger dollar error than the same distortion on a mature program where IBNR is only 10% of the total.
What a buyer should ask their actuary
These questions test whether a diagnostic review actually happened.
1. Did you speak with our claims team before running the triangles? If the answer is no, the analysis was done without operational context. That is not automatically wrong, but it means the actuary relied entirely on the data to diagnose changes, which is less reliable than hearing about them directly.
2. What operational changes occurred during the experience period, and how did you account for each one? The actuary should be able to list them: TPA transitions, staffing changes, new claims protocols, retention changes, exposure growth. For each, the actuary should explain which triangle was affected and what adjustment (if any) was made.
3. Are the age-to-age factors in the most recent two or three diagonals consistent with the older history? If not, the actuary should explain why. A deviation is not a problem if it is explained. A deviation without explanation means the actuary either did not notice or did not investigate.
4. Where would two actuaries most disagree about your selected factors, and why did you choose the direction you chose? This question acknowledges the subjectivity of factor selection and asks the actuary to identify the judgment calls that matter most. The most important development factor in your report is the one where reasonable actuaries would differ. You should know which one it is.
5. Did you run a Berquist-Sherman adjustment, and if not, why not? If case reserves or settlement speed changed during the experience period, Berquist-Sherman is the standard diagnostic tool. If the actuary did not run it, the report should explain why the unadjusted triangle was reliable.
What to require in documentation
A reserve report that reflects a genuine diagnostic review should include:
- A narrative section describing the operational environment during the experience period: TPA changes, staffing shifts, claims protocol modifications, retention changes, and exposure growth or contraction.
- An explicit assessment of case reserve adequacy trends over the experience period, with supporting data (paid-to-incurred ratios by maturity, average case outstanding trends, or closed-without-payment rates).
- An explicit assessment of settlement speed trends, with supporting data (disposal rates by maturity, average time to closure).
- The selected age-to-age factors alongside at least three averaging methods (all-year, five-year weighted, three-year weighted) so the reader can see the range of reasonable selections.
- An explanation of why the selected factor was chosen for each development interval, especially where the selection departs from a simple average.
- A comparison of paid and reported chain ladder indications for each accident year, with an explanation of material divergence.
- Where operational changes were identified, either an explicit adjustment (Berquist-Sherman or data reorganization) or a documented explanation of why no adjustment was needed.
If the report does not include a diagnostic narrative, the reserve number is a mechanical output. It may be correct, but you have no basis for evaluating it. For more on why a single number without context is the least useful actuarial deliverable, see Point Estimate vs. Range.
Further reading
For the foundational concept of IBNR that every method estimates. For a walkthrough of the loss development triangle that feeds every diagnostic. For detailed diagnostics on the three primary drivers of adverse development, see What’s Actually Driving Your IBNR Higher?. For the leading indicators that predict development before it shows up in a triangle, see Five Leading Indicators of Adverse Reserve Development. For how the five core methods differ and when each one applies. For why a point estimate without diagnostic context is the least useful output.