This page is for people who want to read, modify, or extend spDBL. It
maps the workflow functions to the code that implements them. For the
statistical model, see vignette("calibration-details") and
Banerjee et
al. (2025).
Layers
| Layer | Files | What lives there |
|---|---|---|
| Workflow API |
R/func_wrapper.R, R/calibration_prepare.R,
R/calibration_fit.R
|
prepare_data(), emulator_learn(),
emulator_predict(), calibration_prepare(),
calibration_validate(), calibration_fit(),
calibration_predict(), and the S3 methods for
spDBL_calibration
|
| Dynamic linear model kernels |
R/func_FF.R, R/func_BS.R,
R/func_FFBS.R
|
Forward filter, backward sampler, posterior sampling and prediction for each covariance model |
| Out of core |
R/func_FF.R (FF_bigdata_R()),
R/func_bigdata.R
|
Filter, smoother and exact predictor that stream CSV blocks from disk |
| Compiled kernels |
src/FF.cpp, src/BS.cpp,
src/rmn.cpp (RcppEigen) |
One forward-filter step, one backward-smoothing step, matrix-normal draws |
| Calibration internals |
R/calibration_recover.R,
inst/stan/*.stan
|
Stan data assembly, the two Stan programs, R reference log densities, recovery of discrepancy and field draws |
| Distributions |
R/func_MNIW.R, R/func_MNIW_sampler.R,
R/func_lppd_ana.R
|
MNIW/MNIG updates and samplers, matrix-T density, one-step log predictive densities |
| Data and design construction |
R/func_gen_data.R,
R/func_generate_grid_Dan.R
|
Kernels, autoregressive design matrices, episode partitioning, grid traversal, synthetic data |
| Simulator and legacy helpers | R/func_calibrate.R |
The SIR reaction–diffusion simulator (SIR(),
gen_pde()) and helpers from the original research code |
| Plotting | R/func_plot.R |
Heatmap panels and interval summaries |
Covariance models and naming
The emulator is a matrix-variate dynamic linear model. Its right (spatial) covariance has three forms, and kernel functions carry a matching suffix:
| Model | Suffix | Example functions | Available in emulator_learn()
|
|---|---|---|---|
| Inverse-Wishart Σ | none |
FF(), FFBS(),
FFBS_sampling()
|
covariance = "iw" (default) |
| σ² R, R fixed | _sigma2R |
FF_sigma2R(), FFBS_sigma2R(),
FFBS_sampling_sigma2R()
|
covariance = "sigma2R" (required for calibration) |
| Identity | _I |
FF_I(), FFBS_I(),
FFBS_sampling_I()
|
no; kernel functions only |
Emulation: what emulator_learn() does
- Validates the grid (
Nx,Ny), the blocking (episode_windowmust divideNx), and the kernel settings. - Builds two exponential kernels with range
3 / (gp_tune * max distance):Vover simulator inputs andR_episodeover the sites of one spatial block. The full-gridR_field(S × S) is not formed unless supplied; only calibration uses it, and.cal_R_field()builds it at the observed sites. - Splits each time step’s field into
B = Nx / episode_windowspatial blocks (generate.grid.rowsnake()). Every (time, block) pair is one episode of the dynamic model; this is the dynamic transfer-learning partition. - Builds the autoregressive design for every episode with
gen_F_ls_AR1_EP()orgen_F_ls_AR2_EP(), unlessF_ls_trainis given. - Runs forward filtering and backward smoothing:
FFBS()orFFBS_sigma2R(). The inverse-Wishart forward step is in C++ (FF_1step_cpp); the σ² R step is in R (FF_1step_R_sigma2R()); every backward step callsBS_1step_cpp. - Draws
nsamposterior samples withFFBS_sampling()orFFBS_sampling_sigma2R().sampling = "marginal"draws each episode’s smoothed marginal given a shared covariance draw;sampling = "joint"draws whole coefficient paths. - Returns an object of class
spDBL_emulatorwithpara_ffbs(filtering and smoothing moments),res_ffbs(posterior draws), andsetup(every setting and design needed for prediction).
emulator_predict() calls
FFBS_predict_exact() (default) or
FFBS_predict_MC() (MC = TRUE), then maps
episodes back to the time × space layout with
recover_from_EP_exact() or
recover_from_EP_MC(). Given the output of
calibration_prepare(), it predicts with GP-interpolated
design rows instead, so no new simulator runs are needed.
Calibration: from emulator to posterior
-
calibration_prepare()requires a"sigma2R"emulator. It reuses the emulator’s draws or resamples them (sample_calibration_emulator()), thenmake_offline_calibration_input()fits the design-row GP (fit_offline_ar2(); by default a Matern 5/2 kernel with one length scale per input and separate hyperparameters for each time step) and caches Cholesky factors. Result: classspDBL_calibration_input.predict_calibration()turns one coefficient draw into the field mean and covariance at an input. Withdesign_uncertainty = TRUE(the default) the covariance includes the design GP’s error, as a rank-N factor per time (.cal_design_factors()). -
calibration_validate()compares these PDE-free predictions with held-out simulations. -
calibration_fit()validates the settings, compiles or loads the Stan program (.cal_stan_model()), and callsfit_calibration_rstan(). That function runs one conditional Stan fit per emulator draw and pools an equal number of samples from each fit. Result: classspDBL_calibration, withsummary(),print()andplot()methods. -
calibration_predict()recovers discrepancy paths (recover_calibration_discrepancy()) and field draws (recover_calibration_predictions()).
There are two Stan programs in inst/stan/:
-
calibration_marginal.stan, the default (marginal_discrepancy = TRUE), integrates the discrepancy states out with a Kalman filter. -
calibration_offline.stansamples the discrepancy innovations explicitly.
Both integrate out the emulator field analytically. R reference
implementations of the same log densities
(calibration_log_density(),
calibration_marginal_log_density()) let the tests check the
Stan code.
Large grids out of core
For grids too large to hold every episode in memory, three functions
stream CSV blocks from disk (R/func_FF.R,
R/func_bigdata.R):
-
FF_bigdata_R()filters forward and writes each time step’s filtered moments, all blocks side by side, to one set of CSV files. -
BS_bigdata_R()reads those files backwards, one time step at a time, and writes the smoothed moments. -
FFBS_predict_bigdata_R()streams each block’s responses, designs and smoothed means, and writes one prediction file per time step.
Memory holds one time step of moments and one data block at a time.
The tests check all three against the in-memory FF(),
BS() and FFBS_predict_exact(). Prediction
needs every training run in each block (bnrow = fnrow), as
emulator_learn() arranges them.
Reproducibility mechanisms
- Every function that takes a
seedsets it locally and restores the caller’s random-number state on exit (.cal_seed()). - Compiled Stan models are cached in memory and on disk
(
tools::R_user_dir("spDBL", "cache"), orcache_dir). The cache key combines the Stan file’s MD5 hash and the R, rstan, StanHeaders and platform versions, so editing a Stan file triggers a recompile. -
calibration_fit(checkpoint_dir = ...)saves each conditional fit to disk as it finishes. Nothing reloads these files automatically; a rerun starts again from the first emulator draw. -
calibration_fit()recordsexecution_only = TRUEwhen there is no warmup or fewer than two post-warmup iterations, as withiter = 1.iterhas no default, so every analysis states its sampler settings.
Extending spDBL
-
A new covariance model needs a one-step forward
filter, a sampler for the covariance, an
FFBS_*()/FFBS_sampling_*()pair, and a newcovarianceoption inemulator_learn(). The_sigma2Rfunctions are the closest template. -
A new input kernel: the exponential kernel and its
range heuristic are defined inside
emulator_learn();FFBS_predict_exact()and the calibration code read the storedphi_para,gp_sigma2, andgp_tau2. -
A new sampling backend:
calibration_fit(backend = )accepts only"rstan"today. The backend-specific code is.cal_stan_model()andfit_calibration_rstan().
Tests
The tests are in tests/testthat/. The calibration tests
compare the Gaussian conditioning, the Stan log densities, and the
pooled sampler with independent R calculations. Tests that compile Stan
are skipped unless SPDBL_RUN_STAN_TESTS=true; the coverage
workflow on GitHub Actions turns them on. See the contributing
guide for how to run tests and coverage locally.
Known limitations
- The out-of-core path covers filtering, smoothing and exact (mean) prediction. Posterior sampling and calibration work in memory.
- The
delta(discount) argument of the one-step filters is accepted but not used. - Calibration supports the σ² R emulator only, with a fixed
discrepancy covariance scale
U, complete observations, and the training grid and times. - The design-row error is treated as independent across observation
times, although it is in fact persistent; see
vignette("calibration-details").
