Skip to content

Fix/inference dof serialization - #81

Open
ArdiaD wants to merge 7 commits into
masterfrom
fix/inference-dof-serialization
Open

Fix/inference dof serialization#81
ArdiaD wants to merge 7 commits into
masterfrom
fix/inference-dof-serialization

Conversation

@ArdiaD

@ArdiaD ArdiaD commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Five bugs in the reporting and persistence layer around the likelihood, plus regression
tests for each. The likelihood itself is not touched: no estimate, log-likelihood,
conditional variance or state probability changes anywhere in this branch. What changes is
what summary() prints, what AIC/BIC count, what predict() returns for a Bayesian
fit, and whether a saved fit can be reloaded at all.

Everything below is reproduced on data("SMI") with the shipped code. R CMD check --as-cran is unchanged by the branch (2 WARNINGs, 1 NOTE, testthat OK — and the version
bump in the last commit clears one of the two WARNINGs).

1. Standard errors used the delta method transposed — R/Inference.R:38

mSandwitch <- t(mJacob) %*% mInvHessian %*% mJacob   # -> mJacob %*% mInvHessian %*% t(mJacob)

numDeriv::jacobian returns ∂f_i/∂x_j, so Var(g(θ̂)) = J V J'. The two orientations
agree only if J is symmetric, and it is not: the working→natural map is triangular inside
each regime (the sGARCH bound on beta is 0.9999 − alpha1; the gjrGARCH and tGARCH bounds
on beta also involve alpha2 and the shape/skew parameters), and the transition-probability
block is anti-diagonal.

On the default MS(2)-GARCH(1,1)-Normal fit to SMI, checked against the observed information
computed directly in the natural parameterisation:

natural-scale H fixed (J V J') before (J' V J)
alpha1_1 0.01606 0.01511 0.03413
beta_1 0.02197 0.02091 0.00958
alpha1_2 0.00431 0.00437 0.00610
beta_2 0.00416 0.00426 0.00049
P_1_1 0.00982 0.00973 0.00059
P_2_1 (at bound) 0.03072 0.50250

Six of eight were wrong, by factors from 0.06× to 16×. Because the P block is anti-diagonal,
transposing exchanged the two transition probabilities' standard errors. Only alpha0_k,
whose map is a plain exp, was unaffected.

2. Pr(>|t|) was one-sided — R/Inference.R:43

1 - pnorm(abs(t)) under a two-sided label; now 2 * (1 - pnorm(abs(t))).

3. AIC/BIC mis-counted regime.const.parsR/Utils.R:420

A regime-constant parameter leaves one free value where there were K, so it removes K − 1
degrees of freedom (cf. f_rename_par, which strips name_2 … name_K). dofMSGARCH
subtracted one, which is right only at K = 2: with a regime-constant shape parameter the df
was 17 instead of 16 at K = 3 and 27 instead of 25 at K = 4, always over-penalising the
constrained model.

4. Saved specs and fits could not be reloaded — R/Utils.R:349-353

R serializes external pointers as NULL, so a saveRDS-ed spec or fit comes back with dead
pointers. f_check_spec exists to rebuild them, but the rebuild branch called
spec$rcpp.func$get_mean() / get_sd() — the very pointer whose failure had just triggered
the branch:

Volatility(fit) : Error in .External(...): NULL value passed as symbol address
State(fit)      : Error in .External(...): NULL value passed as symbol address
predict(fit)    : Error in .External(...): NULL value passed as symbol address

The two values read there are already held on the R side in spec$prior.mean /
spec$prior.sd, which the next two lines were using anyway, so the C++ round trip was dead
code. Removing it makes the rebuild work; a reloaded spec, ML fit or MCMC fit now returns
values identical to before it was saved, with user priors preserved.

This is the ordinary workflow of fitting a model, saving it, and analysing it in a later
session — or shipping a spec to a parLapply worker.

5. predict() on an MCMC fit returned draw #1R/CondVol.R:32

vol is (T+1) × ndraw and the one-step-ahead value was vol[dim(PredProb)[1]] — a single
index into a matrix is linear indexing, i.e. the last row of the first column. On a
100-draw chain fitted to SMI the reported value was 1.019753 (the first draw) against a
posterior mean of 1.039776, with a spread of 0.999837–1.097337 across draws. Volatility()
already averaged correctly, so the two methods disagreed on the same fit. The
single-parameter (ML) path is unchanged.

Tests

test_Inference.R and test_Serialization.R are new; test_Volatility.R gains one block.
11 blocks, 34 assertions, +13s of check time. All 11 fail on the current code.

Two things worth pointing out, since they are what makes the tests worth having:

  • The standard-error test is anchored on a single-regime GARCH(1,1)-Normal, not on the
    MS(2) default. Every parameter there is interior, so a central-difference Hessian of the
    natural-scale negative log-likelihood is well conditioned: it agrees with J V J' to 1e-4
    in relative terms while the transposed sandwich is off by 44% and 85%. That is an
    independent check on the value, not a restatement of the formula.
  • Every block opens with a guard asserting its own precondition — the two sandwich
    orientations really differ for this model; the round trip really did invalidate the
    pointers; the posterior mean really differs from the first draw — so none of them can pass
    vacuously if the surrounding code changes.

The K = 3 and K = 4 legs of the degrees-of-freedom test run against a fit-shaped list
rather than a real fit, because a constrained K ≥ 3 model cannot currently be fitted at
all (see below).

Not in this branch

The last commit (version bump + NEWS) is separable — drop it if the release number should
be decided elsewhere; nothing depends on it.

A review of the package turned up nine further issues that are not addressed here,
several of them more serious than some of the above. The two worth flagging now:

  • src/MSgarch.h:283 computes the ergodic distribution with a raw Armadillo .i() on
    I − P + U, on every likelihood evaluation. When do.plm = TRUE — forced by fixed.pars
    and regime.const.pars, and hard-coded in FitMCMC — the free transition entries are
    mapped into (0,1) independently, so for K ≥ 3 a row can leave the simplex, the matrix
    can be exactly singular, and the uncaught exception aborts the entire run. Reproducible:
    FitML on a K = 3 spec with regime.const = "nu" dies at
    P = [[1, 0.999973, −0.999973], [0,1,0], [0,0,1]]. This is why the tests above cannot fit
    a constrained K ≥ 3 model.
  • R/FitML.R:133 guards optimisation failure with if (llk == 1e+10), but f_nll
    returns +1e10 so llk is −1e10; and f_OptimFUNDefault wraps optim in try(), so
    optimizer$value errors first. Every failure mode — including a single NA in the data,
    which f_check_y lets through — therefore surfaces as
    Error in optimizer$value : $ operator is invalid for atomic vectors.

Happy to open these as separate issues or as a follow-up PR, whichever you prefer.

ArdiaD added 7 commits August 10, 2026 20:42
f_InferenceFun built the delta-method sandwich as t(J) V J. numDeriv::jacobian
returns d f_i / d x_j, so the variance of the natural parameters is J V t(J).
The two agree only when J is symmetric, which it is not: the working-to-natural
map is triangular inside each regime (the sGARCH bound on beta is 0.9999 -
alpha1; the gjrGARCH and tGARCH bounds on beta also involve alpha2 and the
shape/skew parameters) and the transition-probability block is anti-diagonal.

On the MS(2)-GARCH(1,1)-Normal fit to SMI, six of the eight reported standard
errors were wrong, by factors from 0.06 to 16, and the two transition
probabilities had their standard errors exchanged. The corrected values agree
with the observed information computed directly in the natural parameterisation
to within 2%, where the old ones were off by up to 85%.

The Pr(>|t|) column reported 1 - pnorm(abs(t)), i.e. half a two-sided p-value
under a two-sided label.
A MSGARCH_SPEC holds Rcpp module objects. R serializes external pointers as
NULL, so a spec or fit written with saveRDS comes back with dead pointers and
every method on it fails with "NULL value passed as symbol address". That is
what f_check_spec is for, but its recovery branch called
spec$rcpp.func$get_mean() and get_sd() -- the very pointer whose failure had
just triggered the branch -- so the rebuild could never run.

The two values it read there are already kept on the R side, in
spec$prior.mean and spec$prior.sd, which the next two lines were using anyway;
the C++ round trip was dead code. Dropping it makes the rebuild work: a
reloaded spec, ML fit or MCMC fit now returns exactly the same volatilities,
state probabilities, forecasts and information criteria as before it was
saved, with user-supplied priors preserved.

This affects the ordinary workflow of fitting a model, saving it, and
analysing it in a later session, and of sending a spec to a parallel worker.
Setting constraint.spec$regime.const ties a parameter across all K regimes, so
it leaves one free value where there were K and removes K - 1 degrees of
freedom (see f_rename_par, which strips name_2 ... name_K). dofMSGARCH
subtracted one per constrained name, which is right only at K = 2.

With a regime-constant shape parameter the reported df was 17 instead of 16 at
K = 3 and 27 instead of 25 at K = 4, so stats::AIC and stats::BIC over-
penalised the constrained model in every K >= 3 model-selection table.
In f_CondVol, vol is a (T + 1) x ndraw matrix and the one-step-ahead value was
taken as vol[dim(PredProb)[1]]. A single index into a matrix is linear
indexing, so that is the last row of the first column: predict() on a
MSGARCH_MCMC_FIT reported the forecast of MCMC draw #1 rather than the
posterior mean, and mean() of the resulting scalar was a no-op.

On a 100-draw chain fitted to SMI the reported value was 1.019753, the first
draw, against a posterior mean of 1.039776 and a range across draws of
0.999837 to 1.097337. Volatility() already averaged across draws correctly, so
the two methods disagreed on the same fit. Adding the missing comma leaves the
single-parameter (ML) path untouched.
test_Inference.R (new) pins the delta method against an independently computed
observed information. It is anchored on a single-regime GARCH(1,1)-Normal
because every parameter there is interior, so a central-difference Hessian of
the natural-scale negative log-likelihood is well conditioned: it agrees with
J V t(J) to 1e-4 in relative terms while the transposed sandwich is off by 44%
and 85%. It also checks the exact sandwich on a two-regime fit, where the
transition-probability block is the part that gets exchanged, that Pr(>|t|) is
two-sided, and that dofMSGARCH and the AIC/BIC arithmetic drop K - 1 values per
regime-constant parameter for K = 2, 3 and 4.

test_Serialization.R (new) round-trips a spec, an ML fit and an MCMC fit
through saveRDS/readRDS. R restores external pointers as NULL, so this
reproduces the cross-session failure inside a single session, and the tests
require Volatility, State, predict, AIC, summary, DIC and the log-kernel to
return values identical to the originals, with user priors preserved.

test_Volatility.R gains one block that recomputes the one-step-ahead
volatility draw by draw through the public interface -- each such call carries
a single parameter vector, so it cannot depend on how the draws are pooled --
and requires predict() on the MCMC fit to equal their mean.

Every block opens with a guard asserting its own precondition (the two
sandwich orientations really differ; the round trip really did invalidate the
pointers; the posterior mean really differs from the first draw) so that none
of them can pass vacuously if the surrounding code changes. All eleven blocks
fail on the unpatched package.
Also refreshes the Date field, which R CMD check --as-cran flags as stale
together with the unchanged version number. Drop this commit if the release
number should be decided separately; nothing else in the branch depends on it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant