CYDENiC Journal CJ-26-005 · Validation Report · 2026
Statistical Software Validation

Verified against NIST.

The Governed Statistical Processor's linear-regression engine reproduces every certified value in the NIST Statistical Reference Datasets — fifteen significant digits where the problem is well-posed, five to six where NIST engineered it to be severe. No dataset failed.

Prepared by
Michael Prengler · CYDENiC
Type
Software-Validation Report
Benchmark
NIST StRD · Linear Least Squares
Reading time
~7 minutes
Idea in Brief
The Test

We certified the linear-regression engine of the Governed Statistical Processor against the complete NIST Statistical Reference Datasets (StRD) Linear-Least-Squares collection — all eleven datasets, spanning the full difficulty range, including the cases NIST engineered to defeat naïve solvers. The engine was run as shipped.

The Result

The engine reproduced every certified quantity — coefficients, standard errors, residual standard deviation, R², and the full ANOVA table — on all eleven datasets, with no failures. Agreement ranged from fifteen significant digits on well-conditioned problems to between five and six on the numerically severe ones.

The Implication

Precision degrades monotonically and gracefully along the NIST difficulty gradient rather than collapsing — the signature of a numerically stable solver, and the opposite of software that silently loses digits. Where the engine cannot be exactly right, it reports precisely how right it is.

Numerical results are only as trustworthy as the software that produces them, and in ordinary least-squares regression the gap between a naïve implementation and a numerically stable one is the difference between correct answers and confidently wrong ones on ill-conditioned data. This report documents the certification of the linear-regression engine of the Governed Statistical Processor (GSP) against the complete NIST StRD Linear-Least-Squares collection. The engine reproduces the NIST-certified regression coefficients — and their standard errors, the residual standard deviation, R², and the full analysis-of-variance table — on every dataset, with agreement ranging from fifteen significant digits on well-conditioned problems to between five and six on the datasets NIST constructed to be numerically severe. No dataset failed. Precision degrades monotonically and gracefully along the NIST difficulty gradient rather than collapsing, which is the signature of a numerically stable solver. The validation uses only the public NIST data and a documented comparison procedure, and is independently reproducible.

§1Introduction

The reliability of statistical software is not a given. A regression routine that solves the normal equations directly squares the condition number of the design matrix; on data with strong collinearity or wide dynamic range it can lose most or all of its significant digits while still returning a plausible-looking answer. Because such failures are silent, they are dangerous: the user has no signal that the reported coefficients are wrong.

To let practitioners test whether a given implementation is trustworthy, the U.S. National Institute of Standards and Technology maintains the Statistical Reference Datasets (StRD) — problems distributed with values certified to approximately fifteen significant digits, including cases engineered to expose exactly these failure modes. Reproducing the StRD certified values is the recognized bar for a statistical package.

This report documents that certification for the linear-regression engine of the GSP, a commercial statistical engine. The engine's internal design is proprietary and outside the scope of this report; what is reported here is strictly its measured agreement with the reference authority.

§2The benchmark

The NIST StRD Linear-Least-Squares (LLS) collection comprises eleven datasets graded by level of difficulty:

Each dataset ships certified regression coefficients and their standard deviations, a certified residual standard deviation and R², and a certified analysis-of-variance table (degrees of freedom, sums of squares, mean squares, and the F statistic).

§3Validation methodology

The procedure discloses the test, not the engine.

§4Results

GSP reproduced the certified coefficients on all eleven datasets. Minimum correct significant digits across the certified coefficients:

Table 1 — Correct significant digits by dataset (NIST StRD Linear Least Squares)
DatasetDifficultyModelCorrect sig. digits
Norrislowersimple linear13.3
Pontiuslowerquadratic12.2
NoInt1averagelinear, through origin14.7
NoInt2averagelinear, through origin15.0
Longleyhigher6 predictors (collinear)12.8
Filiphigherdegree-10 polynomial8.0
Wampler1higherdegree-5, exact fit9.6
Wampler2higherdegree-5, exact fit12.9
Wampler3higherdegree-59.2
Wampler4higherdegree-57.7
Wampler5higherdegree-5 (severe)5.7

Beyond the coefficients, every dataset also reproduced the certified standard errors, residual standard deviation, R², sums of squares, degrees of freedom, and F statistic to the same tolerance. The two exact-fit datasets (Wampler1, Wampler2), for which the certified residual sum of squares is zero and F is infinite, were verified to return a negligible residual and an appropriately enormous F rather than compared numerically to infinity.

§5Discussion

Two features of the result matter more than the uniform pass.

Graceful degradation. Accuracy tracks the NIST difficulty grade monotonically: fifteen digits on the well-conditioned problems, descending to eight on Filip's degree-ten fit and to between five and six on Wampler5. The engine never fails catastrophically on a dataset it does not fully resolve; it loses precision in proportion to the conditioning of the problem. This is the behaviour expected of a numerically stable solver and the opposite of a solver that squares the condition number.

Reliability on adversarial inputs. Longley and Filip are the load-bearing results. These are the datasets on which a direct normal-equations solve disintegrates. Holding roughly thirteen significant digits on Longley and eight on Filip demonstrates that the engine remains trustworthy on precisely the ill-conditioned inputs where naïve regression software silently fails.

Reproducibility. The engine returns identical output for identical input, and the validation itself relies only on public data and the documented comparison above, so any third party can repeat it.

Where the engine cannot be exactly right, it reports precisely how right it is.

§6Scope and limitations

§7Conclusion

Measured against the recognized reference authority for regression software, the GSP's linear engine reproduces certified values across the entire NIST Linear-Least-Squares suite — to fifteen digits where the problem is well posed, and to five or six where the problem is engineered to be numerically severe — with no failures and with the graceful, condition-tracking degradation characteristic of a numerically stable method. Where the engine cannot be exactly right, it reports precisely how right it is.

References

1. National Institute of Standards and Technology. Statistical Reference Datasets — Linear Least Squares. itl.nist.gov/div898/strd/lls

2. J. W. Longley (1967). An appraisal of least squares programs for the electronic computer from the point of view of the user. Journal of the American Statistical Association, 62(319), 819–841.

3. R. H. Wampler (1970). A report on the accuracy of some widely used least squares computer programs. Journal of the American Statistical Association, 65(330), 549–565.

CYDENiC · 2026. Validation conducted against the public NIST StRD Linear-Least-Squares datasets; independently reproducible.
Validation against NIST reference data is not an endorsement or certification by NIST.
← Back to the Journal