API references

function get_endpoint(
    theta_init::Vector{Float64},
    theta_num::Int,
    loss_func::Function,
    method::Symbol,
    direction::Symbol = :right;

    loss_crit::Float64 = 0.0,
    scale::Vector{Symbol} = fill(:direct, length(theta_init)),
    theta_bounds::Vector{Tuple{Float64,Float64}} = unscaling.(
        fill((-Inf, Inf), length(theta_init)),
        scale
        ),
    scan_bound::Float64 = unscaling(
        (direction==:left) ? -9.0 : 9.0,
        scale[theta_num]
        ),
    scan_tol::Float64 = 1e-3,
    loss_tol::Float64 = 1e-3,
    local_alg::Symbol = :LN_NELDERMEAD,
    kwargs...
    )

Calculates right or left endpoint of CI for parameter component. It is a wripper of get_right_endpoint functions for selection of direction and using different transformations for faster optimization.

Return

EndPoint object storing confidence endpoint and profile points found on fly.

Arguments

• theta_init: starting values of parameter vector $\theta$. The starting values is not necessary to be the optimum values for loss_func but it the value of loss_func must be lower than loss_crit.
• theta_num: number $n$ of vector component to compute confidence interval $\theta^n$.
• loss_func: loss function $\Lambda\left(\theta\right)$ the profile of which is analyzed. Usually we use log-likelihood for profile analysis in form $\Lambda( \theta ) = - 2 ln\left( L(\theta) \right)$.
• method: computational method to evaluate interval endpoint. Currently the following methods are implemented: :CICO_ONE_PASS, :LIN_EXTRAPOL, :QUADR_EXTRAPOL.
• direction: :right or :left endpoint to estimate.

Keyword arguments

see get_interval

get_interval(
    theta_init::Vector{Float64},
    theta_num::Int,
    loss_func::Function,
    method::Symbol;

    loss_crit::Float64 = 0.0,
    scale::Vector{Symbol} = fill(:direct, length(theta_init)),
    theta_bounds::Vector{Tuple{Float64,Float64}} = unscaling.(
        fill((-Inf, Inf), length(theta_init)),
        scale
        ),
    scan_bounds::Tuple{Float64,Float64} = unscaling.(
        (-9.0, 9.0),
        scale[theta_num]
        ),
    scan_tol::Float64 = 1e-3,
    loss_tol::Float64 = 1e-3,
    local_alg::Symbol = :LN_NELDERMEAD,
    kwargs...
    )

Computes confidence interval for single component theta_num of parameter vector and loss_func according to loss_crit level.

Return

ParamInterval structure storing all input data and estimated confidence interval.

Arguments

• theta_init: starting values of parameter vector $\theta$. The starting values is not necessary to be the optimum values for loss_func but it the value of loss_func must be lower than loss_crit.
• theta_num: number $n$ of vector component to compute confidence interval $\theta^n$.
• loss_func: loss function $\Lambda\left(\theta\right)$ the profile of which is analyzed. Usually we use log-likelihood for profile analysis in form $\Lambda( \theta ) = - 2 ln\left( L(\theta) \right)$.
• method: computational method to evaluate interval endpoint. Currently the following methods are implemented: :CICO_ONE_PASS, :LIN_EXTRAPOL, :QUADR_EXTRAPOL.

Keyword arguments

• loss_crit: critical level of loss function. The endpoint of CI for selected parameter is the value at which profile likelihood meets the value of loss_crit.
• scale: vector of scale transformations for each component. Possible values: :direct, :log, :logit. This option can make optimization much more faster, especially for wide theta_bounds. The default value is :direct (no transformation) for all components.
• theta_bounds: vector of bounds for each component in format (left_border, right_border). This bounds define the ranges for possible parameter values. The defaults are the non-limited values taking into account the scale, i.e. $(0., Inf)$ for :log scale.
• scan_bounds: vector of scan bound for theta_num component. It must be within the theta_bounds for the scanned component. The defaults are $(-9., 9.)$ for transformed values, i.e. $(1e-9, 1e9)$ for :log scale.
• scan_tol: Absolute tolerance of scanned component (stop criterion).
• loss_tol: Absolute tolerance of loss_func at loss_crit (stop criterion). Restriction. Currently is not effective for :CICO_ONE_PASS methods because of limitation in LN_AUGLAG interface.
• local_alg: algorithm of optimization. Currently the local derivation free algorithms form NLOPT pack were tested. The methods: :LN_NELDERMEAD, :LN_COBYLA, :LN_PRAXIS show good results. Methods: :LN_BOBYQA, :LN_SBPLX, :LN_NEWUOA is not recommended.
• kwargs...: the additional keyword arguments passed to get_right_endpoint for specific method.

function get_right_endpoint(
    theta_init::Vector{Float64},
    theta_num::Int,
    loss_func::Function,
    method::Val{:CICO_ONE_PASS};

    theta_bounds::Vector{Tuple{Float64,Float64}} = fill(
        (-Inf, Inf), length(theta_init)
        ),
    scan_bound::Float64 = 9.0,
    scan_tol::Float64 = 1e-3,
    loss_tol::Float64 = 0.,
    local_alg::Symbol = :LN_NELDERMEAD,
    kwargs...
    )

Interface for current and future methods for endpoint estimation.

Return

Tuple of three values:

• Right end point value: ::Float64.
• Profile points estimated on fly: ::Array{ ProfilePoint, 1}, see ProfilePoint.
• Status of sulution: ::Symbol. One of values: :BORDER_FOUND_BY_SCAN_TOL, :SCAN_BOUND_REACHED.

Arguments

• theta_init: starting values of parameter vector $\theta$. The starting values is not necessary to be the optimum values for loss_func but it the value of loss_func must be lower than loss_crit.
• theta_num: number $n$ of vector component to compute confidence interval $\theta^n$.
• loss_func: loss function the profile of which is analyzed, see get_interval. In this function loss crit is always equal 0 for code simplification.
• method: this value is always fixed. Implemented methods are: Val{:CICO_ONE_PASS}. It is implemented for easy switching between different implemented and future methods.

Keyword arguments

• theta_bound: vector of bounds for each component in format (left_bound, right_bound). This bounds define the ranges for possible parameter values.
• scan_bound: right scan bound for theta_num component. It must be within the theta_bounds for the scanned component.
• scan_tol: Absolute tolerance of scanned component (stop criterion).
• loss_tol: Absolute tolerance of loss_func at loss_crit (stop criterion). Restriction. Currently is not effective for :CICO_ONE_PASS methods because of limitation in LN_AUGLAG interface.
• local_alg: algorithm of optimization. Currently the local derivation free algorithms form NLOPT pack were tested. The methods: :LN_NELDERMEAD, :LN_COBYLA, :LN_PRAXIS show good results. Methods: :LN_BOBYQA, :LN_SBPLX, :LN_NEWUOA is not recommended.
• kwargs...: the additional keyword arguments passed to get_right_endpoint for specific method.

function profile(
    theta_init::Vector{Float64},
    theta_num::Int,
    loss_func::Function;

    skip_optim::Bool = false,
    theta_bounds::Vector{Tuple{Float64,Float64}} = fill((-Inf, Inf), length(theta_init)),
    local_alg::Symbol = :LN_NELDERMEAD,
    ftol_abs::Float64 = 1e-3,
    maxeval::Int = 10^5,
    kwargs... # currently not used
    )

It generates the profile function based on loss_func. Used internally in methods :LIN_EXTRAPOL, :QUADR_EXTRAPOL.

Return

Returns profile function for selected parameter component. Each call of the function starts optimization.

Arguments

• theta_init: starting values of parameter vector $\theta$.
• theta_num: number $n$ of vector component to create the profile.
• loss_func: loss function $\Lambda\left(\theta\right)$ the profile of which is analyzed. Usually we use log-likelihood for profile analysis in form $\Lambda( \theta ) = - 2 ln\left( L(\theta) \right)$.

Keyword arguments

• skip_optim : set true if you need marginal profile, i.e. profile without optimization. Default is false.
• theta_bounds : vector of bounds for each component in format (left_border, right_border). This bounds define the ranges for possible parameter values.
• local_alg : algorithm of optimization. Currently the local derivation free algorithms form NLOPT pack were tested. The methods: :LN_NELDERMEAD, :LN_COBYLA, :LN_PRAXIS show good results. Methods: :LN_BOBYQA, :LN_SBPLX, :LN_NEWUOA is not recommended.
• ftol_abs : absolute tolerance criterion for profile function.
• maxeval : maximal number of loss_func calls to estimate profile point.

scaling(x::Float64, scale::Symbol = :direct)

Transforms values from specific scale to range [-Inf, Inf] based on option.

Return

Transformed value.

Arguments

• x: input value.
• scale: transformation type: :direct, :log, :logit.

unscaling(x::Float64, scale::Symbol = :direct)

Transforms values from [-Inf, Inf] to specific scale based on option. Inverse function for scaling.

Return

Transformed value.

Arguments

• x: input value.
• scale: transformation type: :direct, :log, :logit.

update_profile_points!(
    pi::ParamInterval;
    max_recursions::Int = 2
    )

Refines profile points to make your plot more smooth. Internally uses adapted_grid to compute additional profile points. See PlotUtils.adapted_grid.

Arguments

• pi : ParamInterval structure to update.
• max_recursions : how many times each interval is allowed to

be refined (default: 2).

struct EndPoint
    value::Float64                        # value of endpoint or nothing
    profilePoints::Array{ProfilePoint, 1} # vector of profile points
    status::Symbol                        # result of analysis
    direction::Symbol                     # :right or :left
    counter::Int                          # number of loss_func() calls to calculate the endpoint
    supreme::Union{Float64, Nothing}      # maximal value inside profile interval
end

Structure storing one endpoint for confidence interval.

status values - :BORDERFOUNDBYSCANTOL, :BORDERFOUNDLOSSTOL, :SCANBOUNDREACHED, :MAXITERSTOP, :LOSSERROR_STOP

struct ParamInterval
    input::ParamIntervalInput
    loss_init::Float64
    method::Symbol
    result::Tuple{EndPoint, EndPoint}
end

Structure storing result of parameter interval calculation

struct ParamIntervalInput
    theta_init::Vector{Float64} # initial parameters vector
    theta_num::Int # number of the parameter for analysis
    loss_func::Function # loss function
    loss_crit::Float64 # loss function maximum value, "identifiability level"
    scale::Vector{Symbol}
    theta_bounds::Vector{Tuple{Float64, Float64}} # search bounds for id parameter
    scan_bounds::Tuple{Float64,Float64}
    scan_tol::Float64 # fitting tolerance for local optimizer (default - 1e-3)
    loss_tol::Float64 # constraints tolerance
    local_alg::Symbol # local fitting algorithm (default - :LN_NELDERMEAD)
    fitter_options::Any
end

Structure storing input data for parameter interval calculation

struct ProfilePoint
    value::Float64               # x value of profile point
    loss::Float64                # y value of profile point (loss function at value)
    params::Array{Float64, 1}    # vector of optimal values of loss_func arguments
    ret::Symbol                  # return value from NLOpt.optimize()
    counter::Union{Int, Nothing} # number of loss_func() calls to calculate the value
end

Structure storing one point from profile function. ret values - :FORCEDSTOP, :MAXEVALREACHED, :FTOL_REACHED

Main module for LikelihoodProfiler.jl.

Four functions are exported from this module for public use:

• get_endpoint. Calculates endpoint of confidence interval.
• get_interval. Calculates confidence interval.
• profile. Generates the profile function based on loss_func
• update_profile_points!. Updates interval by profile points.

logistic10(x::Float64)

Function transforming interval [-Inf, Inf] to [0,1] using logistic transformation. Inverse function for logit10.

logit10(x::Float64)

Function transforming interval [0,1] to [-Inf, Inf] using logit transformation.

using Plots
plotly()
plot(pi::ParamInterval)

Plots profile L(theta) for parameter theta_num, identifiability level, identifiability interval. Use update_profile_points!(pi::ProfileInterval) function to refine profile points and make your plot more smooth