Profile Likelihood Methods
LikelihoodProfiler provides a range of methods to profile likelihood functions and explore practical identifiability. The method should be provided as the second argument to the solve function.
Optimization-based profiles
The method computes profiles for each parameter by iteratively changing the value of the parameter and re-optimizing the likelihood function with respect to all other parameters.
LikelihoodProfiler.OptimizationProfiler — Type
OptimizationProfiler{S, opType, optsType}A profiler method that uses stepwise re-optimization to profile the likelihood function.
Fields
stepper::S: The algorithm used to compute the next profile point. Supported steppers include:FixedStep: Proposes a constant step size in the profiling direction (Default).LineSearchStep: Uses a line search to adaptively determine the step size in the direction which is chosen by thedirectionkeyword argument.
optimizer::opType: The optimizer used for the optimization process.optimizer_opts::optsType: Options for the optimizer. Defaults toNamedTuple().
Stepping Options
The stepper argument controls how the next profile point is chosen. For example:
stepper = FixedStep(initial_step=0.1): Use a constant step size of 0.1.stepper = LineSearchStep(direction=:Secant, linesearch=InterpolationLineSearch()): Use a line search with secant direction.
See the documentation for each stepper type (e.g., ?FixedStep, ?LineSearchStep) for more details and customization options.
Example
using OptimizationLBFGSB
profiler = OptimizationProfiler(; optimizer = LBFGSB(), optimizer_opts = (reltol=1e-4,))Integration-based profiles
The method computes profiles for each parameter (or function of parameters) by integrating the differential equations system.
LikelihoodProfiler.IntegrationProfiler — Type
IntegrationProfiler{opType, optsType, DEAlg, DEOpts}A profiler method that uses integration of differential equations system to profile the likelihood function.
Fields
reoptimize::Bool: Indicates whether to re-optimization after each step of theintegrator. Defaults tofalse.optimizer::opType: The optimizer used for the optimization process. Defaults tonothing.optimizer_opts::optsType: Options for the optimizer. Defaults toNamedTuple().integrator::DEAlg: The differential equation algorithm used for integration.integrator_opts::DEOpts: Options for the differential equation solver. Defaults toNamedTuple().matrix_type::Symbol: The type of matrix to be used for the Hessian approximation. Possible options are::hessian,:identity. Defaults to:hessian.gamma::Float64: Correction factor used in integration if full hessian is not computed (e.g.matrix_type = :identity). Defaults to1.0.
Example
using OrdinaryDiffEq
profiler = IntegrationProfiler(integrator = Tsit5(), integrator_opts = (dtmax=0.3,), matrix_type = :hessian)References:
- Chen, J.-S. & Jennrich, R. I. Simple Accurate Approximation of Likelihood Profiles. Journal of Computational and Graphical Statistics 11, 714–732 (2002).
- Chen, J.-S. & Jennrich, R. I. The Signed Root Deviance Profile and Confidence Intervals in Maximum Likelihood Analysis. Journal of the American Statistical Association 91, 993–998 (1996).
Confidence Intervals by Constrained Optimization (CICO)
The method computes intersections (endpoints of the confidence interval (CI)) of the profile with the predefined confidence level (threshold) without restoring the exact trajectory of the profile. Requires using CICOBase package.
LikelihoodProfiler.CICOProfiler — Type
CICOProfilerConfidence Intervals by Constrained Optimization (CICO) method to find the intersections of the likelihood function with the threshold. See CICOBase docs for more details. Requires using CICOBase.
Fields
optimizer::Symbol: The optimizer used for the optimization process. Defaults to NLopt:LN_NELDERMEAD.scan_tol::Float64: The tolerance for the endpoints scan. Defaults to1e-3.
Example
profiler = CICOProfiler(optimizer = :LN_NELDERMEAD, scan_tol = 1e-3)References:
- Borisov, I. & Metelkin, E. Confidence intervals by constrained optimization—An algorithm and software package for practical identifiability analysis in systems biology. PLoS Comput Biol 16, e1008495 (2020).
- Venzon, D. J. & Moolgavkar, S. H. A Method for Computing Profile-Likelihood-Based Confidence Intervals. Applied Statistics 37, 87 (1988).
Quadratic approximation (FIM curvature at optimum)
LikelihoodProfiler.QuadraticApproxProfiler — Type
QuadraticApproxProfilerQuadratic-approximation confidence intervals (Wald approximation) based on local curvature at the optimum. The curvature is approximated by the Fisher Information Matrix/Hessian, so the resulting confidence intervals reflect the local quadratic approximation of the likelihood around optpars. By default this method reuses Hessian logic from OptimizationProblem (user-supplied Hessian or AD backend). The confidence interval is computed as θ̂ ± z * sqrt(Σ[idx, idx]), where - θ̂ is the optpars[idx], - z is the quantile of the chi-squared distribution corresponding to the conf_level and df parameters of the ProfileLikelihoodProblem, - Σ is the covariance matrix obtained by inverting the FIM.
cov_factor controls Hessian/objective scaling conventions. Common choices are:
1.0when Hessian is for-logL.2.0when Hessian is for-2logLand you want covariance on the-logLscale.
Any strictly positive value is allowed (not only 1 or 2), which can be useful for calibrated or robust variance scaling.
Fields
inversion::Symbol: Matrix inversion strategy (:cholesky,:pinv).clamp_to_bounds::Bool: Clip estimated interval endpoints to profile bounds.cov_factor::Real: Multiplicative factor applied toinv(H)to obtain covariance (Σ = cov_factor * inv(H)).resolution::Int: Number of points per branch (left/right) used to sample the quadratic approximation.