hierNest: Penalized Regression with Hierarchical Nested Parameterization Structure

Description

Efficient implementation of penalized regression with hierarchical nested parametrization for grouped data. The package provides penalized regression methods that decompose subgroup specific effects into shared global effects, Major subgroup specific effects, and Minor subgroup specific effects, enabling structured borrowing of information across related clinical subgroups. Both lasso and hierarchical overlapping group lasso penalties are supported to encourage sparsity while respecting the nested subgroup structure. Efficient computation is achieved through a modified design matrix representation and a custom majorization minimization algorithm for overlapping group penalties.

Author(s)

Maintainer: Ziren Jiang jian0746@umn.edu

Authors:

• Jared Huling huling@umn.edu

• Jue Hou hou00123@umn.edu

• Lingfeng Huo

Other contributors:

• Daniel J. McDonald [contributor]

• Xiaoxuan Liang [contributor]

• Anibal Solón Heinsfeld [contributor]

• Aaron Cohen [contributor]

• Yi Yang [contributor]

• Hui Zou [contributor]

• Jerome Friedman [contributor]

• Trevor Hastie [contributor]

• Rob Tibshirani [contributor]

• Balasubramanian Narasimhan [contributor]

• Kenneth Tay [contributor]

• Noah Simon [contributor]

• Junyang Qian [contributor]

• James Yang [contributor]

See Also

Useful links:

• https://github.com/ZirenJiang/hierNest

• Report bugs at https://github.com/ZirenJiang/hierNest/issues

Pipe operator

Description

See magrittr::%>% for details.

Usage

lhs %>% rhs

Arguments

Value

The result of calling 'rhs(lhs)'.

Extract coefficients from a 'cv.sparsegl' object.

Description

This function etracts coefficients from a cross-validated [sparsegl()] model, using the stored '"sparsegl.fit"' object, and the optimal value chosen for 'lambda'.

Usage

## S3 method for class 'cv.sparsegl'
coef(object, s = c("lambda.1se", "lambda.min"), ...)

Arguments

Value

The coefficients at the requested value(s) for 'lambda'.

See Also

[cv.sparsegl()] and [predict.cv.sparsegl()].

Extract model coefficients from a 'sparsegl' object.

Description

Computes the coefficients at the requested value(s) for 'lambda' from a [sparsegl()] object.

Usage

## S3 method for class 'sparsegl'
coef(object, s = NULL, ...)

Arguments

Details

's' is the new vector of 'lambda' values at which predictions are requested. If 's' is not in the lambda sequence used for fitting the model, the 'coef' function will use linear interpolation to make predictions. The new values are interpolated using a fraction of coefficients from both left and right 'lambda' indices.

Value

The coefficients at the requested values for 'lambda'.

See Also

[sparsegl()] and [predict.sparsegl()].

Cross-validated hierarchical nested regularization for subgroup models

Description

Fits regularization paths for hierarchical subgroup-specific penalized learning problems, leveraging nested group structure (such as Major Diagnostic Categories [MDC] and Diagnosis-Related Groups [DRG]) with options for lasso or overlapping group lasso penalties. Performs cross-validation to select tuning parameters for penalization and subgroup structure.

This function enables information sharing across related subgroups by reparameterizing covariate effects into overall, group-specific, and subgroup-specific components and supports structured shrinkage through hierarchical regularization. Users may select between the lasso (hierNest-Lasso) and overlapping group lasso (hierNest-OGLasso) frameworks, as described in Jiang et al. (2024, submitted, see details below).

Usage

cv.hierNest(
  x,
  y,
  group = NULL,
  family = c("gaussian", "binomial"),
  nlambda = 100,
  lambda.factor = NULL,
  pred.loss = c("default", "mse", "deviance", "mae", "misclass", "ROC"),
  lambda = NULL,
  pf_group = NULL,
  pf_sparse = NULL,
  intercept = FALSE,
  asparse1 = c(0.5, 20),
  asparse2 = c(0.01, 0.2),
  asparse1_num = 4,
  asparse2_num = 4,
  standardize = TRUE,
  lower_bnd = -Inf,
  upper_bnd = Inf,
  eps = 1e-08,
  maxit = 3e+06,
  hier_info = NULL,
  method = "overlapping",
  partition = "subgroup",
  cvmethod = "general"
)

Arguments

Details

The hierarchical nested framework decomposes covariate effects into overall, group, and subgroup-specific components, with regularization encouraging fusion or sparsity across these hierarchical levels. The function can fit both the lasso penalty (allowing arbitrary zero/non-zero patterns) and the overlapping group lasso penalty (enforcing hierarchical selection structure), as described in Jiang et al. (2024, submitted).

The argument 'hier_info' must be supplied for '"overlapping"' method, and encodes the hierarchical relationship between groups and subgroups (e.g., MDCs and DRGs).

Cross-validation is used to select tuning parameters, optionally over a grid for hierarchical penalty weights (asparse1, asparse2), and the regularization parameter lambda.

Value

An object containing the fitted hierarchical model and cross-validation results, including:

References

Jiang, Z., Huo, L., Hou, J., & Huling, J. D. "Heterogeneous readmission prediction with hierarchical effect decomposition and regularization".

Calculate information criteria.

Description

This function uses the degrees of freedom to calculate various information criteria. This function uses the "unknown variance" version of the likelihood. Only implemented for Gaussian regression. The constant is ignored (as in [stats::extractAIC()]).

Usage

estimate_risk(object, x, type = c("AIC", "BIC", "GCV"), approx_df = FALSE)

Arguments

Value

a 'data.frame' with as many rows as 'object$lambda'. It contains columns 'lambda', 'df', and the requested risk types.

References

Vaiter S, Deledalle C, Peyré G, Fadili J, Dossal C. (2012). The Degrees of Freedom of the Group Lasso for a General Design. https://arxiv.org/pdf/1212.6478.pdf.

See Also

[sparsegl()] method.

Example dataset

Description

Example dataset for the sample code of hierNest

Usage

data(example_data)

Format

A data frame with X rows and Y columns:

X

Matrix of predictors

Y

Outcome variable

hier_info

Group information passed to hierNest functions

Fit Hierarchical Nested Regularization Model (hierNest)

Description

Fits a hierarchical nested penalized regression model for subgroup-specific effects using overlapping group lasso penalties. This function encodes the hierarchical structure (e.g., MDC and DRG) via a reparameterized design matrix and enables information borrowing across related subgroups.

Usage

hierNest(
  x,
  y,
  group = NULL,
  family = c("gaussian", "binomial"),
  nlambda = 100,
  lambda.factor = NULL,
  lambda = NULL,
  pf_group = NULL,
  pf_sparse = NULL,
  intercept = FALSE,
  asparse1 = 1,
  asparse2 = 0.05,
  standardize = TRUE,
  lower_bnd = -Inf,
  upper_bnd = Inf,
  eps = 1e-08,
  maxit = 3e+06,
  hier_info = NULL,
  random_asparse = FALSE,
  method = "overlapping"
)

Arguments

Details

This function builds a hierarchical design matrix reflecting group/subgroup structure (e.g., Major Diagnostic Categories [MDCs] and Diagnosis Related Groups [DRGs]), encoding overall, group-specific, and subgroup-specific effects. It fits a penalized model using overlapping group lasso, as described in Jiang et al. (2024, submitted). The main computational engine is hierNest::overlapping_gl.

Value

Returns a model fit object as produced by hierNest::overlapping_gl, including selected coefficients, cross-validation results, and tuning parameters.

References

Jiang, Z., Huo, L., Hou, J., Vaughan-Sarrazin, M., Smith, M. A., & Huling, J. D. (2024). Heterogeneous readmission prediction with hierarchical effect decomposition and regularization.

Create starting values for iterative reweighted least squares

Description

This function may be used to create potentially valid starting values for calling [sparsegl()] with a [stats::family()] object. It is not typically necessary to call this function (as it is used internally to create some), but in some cases, especially with custom generalized linear models, it may improve performance.

Usage

make_irls_warmup(nobs, nvars, b0 = 0, beta = double(nvars), r = double(nobs))

Arguments

Details

Occasionally, the irls fitting routine may fail with an admonition to create valid starting values.

Value

List of class 'irlsspgl_warmup'

Fit an Overlapping Group Lasso Model

Description

This function fits an overlapping group lasso model with hierarchical regularization, allowing both sparsity within groups and across groups.

Usage

overlapping_gl(
  x,
  y,
  group = NULL,
  family = c("gaussian", "binomial"),
  nlambda = 100,
  lambda.factor = ifelse(nobs < nvars, 0.01, 1e-04),
  lambda = NULL,
  pf_group = sqrt(bs),
  pf_sparse = rep(1, nvars),
  intercept = TRUE,
  asparse1 = 1,
  asparse2 = 0.05,
  standardize = TRUE,
  lower_bnd = -Inf,
  upper_bnd = Inf,
  weights = NULL,
  offset = NULL,
  warm = NULL,
  trace_it = 0,
  dfmax = as.integer(max(group)) + 1L,
  pmax = min(dfmax * 1.2, as.integer(max(group))),
  eps = 1e-08,
  maxit = 3e+06,
  cn,
  drgix,
  drgiy,
  cn_s,
  cn_e,
  random_asparse = FALSE
)

Arguments

Value

An object of class 'sparsegl' containing:

Additional components relevant to model diagnostics and fitting.

Plot method for cv.hierNest objects

Description

Plot method for cv.hierNest objects

Usage

## S3 method for class 'cv.hierNest'
plot(x, type = c("coefficients", "Subgroup effects"), ...)

Arguments

Value

Invisibly returns x

Make predictions from a 'cv.sparsegl' object.

Description

This function makes predictions from a cross-validated [cv.sparsegl()] object, using the stored 'sparsegl.fit' object, and the value chosen for 'lambda'.

Usage

## S3 method for class 'cv.sparsegl'
predict(
  object,
  newx,
  s = c("lambda.1se", "lambda.min"),
  type = c("link", "response", "coefficients", "nonzero", "class"),
  ...
)

Arguments

Value

A matrix or vector of predicted values.

See Also

[cv.sparsegl()] and [coef.cv.sparsegl()].

Make predictions from a 'sparsegl' object.

Description

Similar to other predict methods, this function produces fitted values and class labels from a fitted ['sparsegl'] object.

Usage

## S3 method for class 'sparsegl'
predict(
  object,
  newx,
  s = NULL,
  type = c("link", "response", "coefficients", "nonzero", "class"),
  ...
)

Arguments

Details

's' is the new vector of 'lambda' values at which predictions are requested. If 's' is not in the lambda sequence used for fitting the model, the 'coef' function will use linear interpolation to make predictions. The new values are interpolated using a fraction of coefficients from both left and right 'lambda' indices.

Value

The object returned depends on type.

See Also

[sparsegl()], [coef.sparsegl()].

Predict Method for hierNest Objects

Description

Provides predictions from a fitted hierarchical model ('hierNest') using new data.

Usage

predict_hierNest(
  object,
  newx,
  hier_info,
  type = c("link", "response", "coefficients", "nonzero", "class"),
  ...
)

Arguments

Details

This function prepares a hierarchical design matrix based on 'hier_info', constructs the required Khatri-Rao product, and reorganizes it before generating predictions from the provided 'object'.

Value

Predictions based on the specified 'type'. Typically, returns:

• Numeric vector or matrix of predicted values (for "link" or "response").

• Model coefficients (for "coefficients").

• Nonzero coefficient indices (for "nonzero").

• Class labels for categorical outcomes (for "class").

Calculate common norms

Description

Calculate different norms of vectors with or without grouping structures.

Usage

zero_norm(x)

one_norm(x)

two_norm(x)

grouped_zero_norm(x, gr)

grouped_one_norm(x, gr)

grouped_two_norm(x, gr)

grouped_sp_norm(x, gr, asparse)

gr_one_norm(x, gr)

gr_two_norm(x, gr)

sp_group_norm(x, gr, asparse = 0.05)

Arguments

Value

A numeric scalar or vector

Functions

• zero_norm(): l0-norm (number of nonzero entries).

• one_norm(): l1-norm (Absolute-value norm).

• two_norm(): l2-norm (Euclidean norm).

• grouped_zero_norm(): A vector of group-wise l0-norms.

• grouped_one_norm(): A vector of group-wise l1-norms.

• grouped_two_norm(): A vector of group-wise l2-norms.

• grouped_sp_norm(): A vector of length 'unique(gr)' consisting of the 'asparse' convex combination of the l1 and l2-norm for each group.

• gr_one_norm(): The l1-norm norm of a vector (a scalar).

• gr_two_norm(): The sum of the group-wise l2-norms of a vector (a scalar).

• sp_group_norm(): The sum of the 'asparse' convex combination of group l1 and l2-norms vectors (a scalar).