Fitting Quantile Regression Models

QuantReg.jl currently contains three methods for fitting quantile regression models: (1) the Barrodale-Roberts simple algorithm, (2) the Frisch-Newton algorithm, and (3) via the industrial solver Gurobi. The first two methods leverage FORTRAN code originally written for the R quantreg package. The third method was developed specifically for this package and only works if Gurobi is already installed on the user's machine.

Generic functions

A QuantRegModel object, named model, can be fit using the generic fit!() function, and indeed, this is the function that most end users will leverage. This functions fits the object in place, modifying the QuantRegFit object stored at model.fit. For example,

julia> model = QuantRegModel(@formula(Y ~ X1 + X2 + X3), df; τ=0.80, fitmethod="fn")

creates an unfitted a quantile regression model at the 0.80th quantile and specifies for it to be fit via the Frisch-Newton. Then,

julia> fit!(model)

fits the model in-place via the specified method.

Additionaly, there is a fit() method that creates a deep copy of the passed model, fits that copy, and returns the new, fitted model.

StatsBase.fit! — Method

fit!(model::QuantRegModel)

Fit model in-place according to model.fitmethod.

source

QuantReg.fit — Function

fit(model::QuantRegModel)

Deep copy model and fit according to model.fitmethod.

source

Ultimately, these two functions simply serve as a one-stop wrapper for fitting a model and calling different subroutines depending on the value of model.fitmethod.

Choosing a fitting method

Choosing the best fit method for a quantile regression model largely depends upon the size of the dataset that you are attempting to fit.

For very small datasets (both in number of observations and number of predictors) the Barrodale-Roberts simplex tended to outperform other fitting methods in limited simulations. The speed difference between the Barrodale-Roberts simplex algorithm and the Frisch-Newton algorithm, however, was negligible (both returned a model well within one second), so either method seems like a good choice. Using Gurobi, however, imposes a signficant speed penalty, mainly due to the fixed cost of initializing a Gurobi environment. While the method still produces results relatively quickly (sub-two seconds), this is substantially slower than the other algorithms.
As datasets grow in size, the Frisch-Newton algorithm and Gurobi catch-up to and substantially overtake the Barrodale-Roberts simplex algorithm in speed. With the exception of extremely large datasets, the Frisch-Newton algorithm still tends to outperform the Gurobi algorithm due to the latter's fixed start-up costs. The speed of the two methods is relatively similar in scale, however.

Going forward in development, the goal for the Gurobi solving method should be to produce a reusable Gurobi environment upon load to eliminate the fixed start-up costs.

Barrodale-Roberts Simplex Method

The Barrodale-Roberts simplex algorithm implemented can be used for both fitting a model and for computing inference via a rank test inversion. When fitting, the following method is called with the keyword argument, ci, set to its default value of false.

To select this method when specifying a quantile regression model, set the keyword argument fitmethod="br" in either the rq function or the QuantRegModel constructor.

QuantReg.fitbr! — Function

fitbr!(model::QuantRegModel; ci::Bool=false),

Fit model using the Barrodale-Roberts method.

If ci is false, model.fit is updated in place to reflect the fit produced by running the Barrodale-Roberts simplex and confidence intervals are not computed. Otherwise, confidence intervals are computed, and model.inf is updated in place to reflect the confidence invervals produced by this method but model.fit is not updated.

This fitting method leverages public domain FORTRAN code written by Roger Koenker for the R quantreg package.

source

This method relies on a subroutine, fitbrfortran, that handles the FORTRAN call.

QuantReg.fitbrfortran — Function

fitbrfortran(n::Integer, k::Integer, X::Matrix{Number}, y::Vector{Number},
             τ::Number, nsol::Integer, ndsol::Integer, qn::Vector{Number},
             cutoff::Number, lci1::Bool)

Wraps call to the public domain Barrodale-Roberts simplex FORTRAN routine.

Arguments

n: number of observations
k: number of parameters
X: x matrix
y: y vector
τ: the desired quantile
nsol: an estimated (row) dimension of the primal solution array
ndsol1: an estimated (row) dimension of the dual solution array
qn: residuals variances from the projection of each column of X on remaining columns
cutoff: the critical point for testing
lci1: whether to calculate CI

source

Frisch-Newton Algorithm

The Frisch-Newton algorithm implemented is an interior point method that can be used to solve quantile regression problems. Unlike the other two methods in this package, this method does not produce solutions to the dual problem.

To select this method when specifying a quantile regression model, set the keyword argument fitmethod="fn" in either the rq function or the QuantRegModel constructor.

QuantReg.fitfn! — Function

fitfn!(model::QuantRegModel)

Fit model using the Frish-Newton algorithm.

Fitting with this method does not produce solutions to the dual problem.

This fitting method leverages public domain FORTRAN code written by Roger Koenker for the R quantreg package. In the quantreg package, this is equivalent to the fn and fnb methods.

source

Gurobi

The only fitting method available via this package that is not available via the R quantreg package is fitting via Gurobi. This fitting method is only available to users who already have a licensed version of Gurobi installed on their machine and the Gurobi.jl package added in Julia. Even if an active Gurobi license is installed on the user's machine, the GUROBI_HOME environmental variable must be properly set for the Gurobi fitting method to be available.

To select this method when specifying a quantile regression model, set the keyword argument fitmethod="gurobi" in either the rq function or the QuantRegModel constructor.

QuantReg.fitgurobi! — Function

fitgurobi(model::QuantRegModel)

Fit model using Gurobi via Julia's JuMP library.

This algorithm has been tailored specifically to work with Julia (hence the use of direct) mode. Some time was spent researching the use of open-source solvers such as GLPK, but they proved to be too slow. Extending this method to work with other solvers, such as CPLEX could be accomplished by switching the optimizer and creating the model in automatic mode, but this may leave some small efficiency gains on the table.

source