Leave-one-out cross-validation (LOO-CV) — fit-method-loo • cmdstanr
Description: The $loo() method computes approximate LOO-CV using the loo package. In order to use this method you must compute and save the pointwise log-likelihood in your Stan program. See loo::loo.array() and the loo package vignettes for details.

# Leave-one-out cross-validation (LOO-CV)

Source: `R/fit.R`
`fit-method-loo.Rd`

The `$loo()` method computes approximate LOO-CV using the loo package. In order to use this method you must compute and save the pointwise log-likelihood in your Stan program. See `loo::loo.array()` and the loo package vignettes for details.

    loo(variables = "log_lik", r_eff = TRUE, moment_match = FALSE, ...)

## Arguments

variables

  (character vector) The name(s) of the variable(s) in the Stan program containing the pointwise log-likelihood. The default is to look for `"log_lik"`. This argument is passed to the `$draws()` method.

r_eff

  (multiple options) How to handle the `r_eff` argument for `loo()`:

  - `TRUE` (the default) will automatically call `loo::relative_eff.array()` to compute the `r_eff` argument to pass to `loo::loo.array()`.

  - `FALSE` or `NULL` will avoid computing `r_eff` (which can sometimes be slow) but will result in a warning from the loo package.

  - If `r_eff` is anything else, that object will be passed as the `r_eff` argument to `loo::loo.array()`.

moment_match

  (logical) Whether to use a moment-matching correction for problematic observations. The default is `FALSE`. Using `moment_match=TRUE` will result in compiling the additional methods described in fit-method-init_model_methods. This allows CmdStanR to automatically supply the functions for the `log_lik_i`, `unconstrain_pars`, `log_prob_upars`, and `log_lik_i_upars` arguments to `loo::loo_moment_match()`.

...

  Other arguments (e.g., `cores`, `save_psis`, etc.) passed to `loo::loo.array()` or `loo::loo_moment_match.default()` (if `moment_match` = `TRUE` is set).

## Value

The object returned by `loo::loo.array()` or `loo::loo_moment_match.default()`.

## See also

The loo package website with documentation and vignettes.

## Examples

    # \dontrun{
    # the "logistic" example model has "log_lik" in generated quantities
    fit <- cmdstanr_example("logistic")
    #> Model executable is up to date!
    loo_result <- fit$loo(cores = 2)
    print(loo_result)
    #>
    #> Computed from 4000 by 100 log-likelihood matrix
    #>
    #>          Estimate  SE
    #> elpd_loo    -63.7 4.1
    #> p_loo         3.9 0.5
    #> looic       127.4 8.3
    #> ------
    #> Monte Carlo SE of elpd_loo is 0.0.
    #>
    #> All Pareto k estimates are good (k < 0.5).
    #> See help('pareto-k-diagnostic') for details.
    # }