# Robust Estimation of Standard Errors, Confidence Intervals and p-values

The `tab_model()` function also allows the computation of standard errors, confidence intervals and p-values based on robust covariance matrix estimation from model parameters. Robust estimation is based on the packages sandwich and clubSandwich, so all models supported by either of these packages work with `tab_model()`.

## Classical Regression Models

### Robust Covariance Matrix Estimation from Model Parameters

There are two arguments that allow for choosing different methods and options of robust estimation: `vcov.fun` and `vcov.args`. Let us start with a simple example, which uses a heteroskedasticity-consistent covariance matrix estimation with estimation-type “HC3” (i.e. `sandwich::vcovHC(type = "HC3")` is called):

    data(iris)
    model <- lm(Petal.Length ~ Sepal.Length * Species + Sepal.Width, data = iris)

    # model parameters, where SE, CI and p-values are based on robust estimation
    tab_model(model, vcov.fun = "HC3", show.se = TRUE)

| Petal Length
--- | ---
Predictors | Estimates | std. Error | CI | p
(Intercept) | 0.87 | 0.45 | -0.03 – 1.76 | 0.059
Sepal Length | 0.04 | 0.12 | -0.19 – 0.28 | 0.711
Species [versicolor] | -0.78 | 0.69 | -2.15 – 0.59 | 0.265
Species [virginica] | -0.41 | 0.63 | -1.66 – 0.83 | 0.513
Sepal Width | 0.11 | 0.08 | -0.05 – 0.27 | 0.190
Sepal Length × Species [versicolor] | 0.61 | 0.13 | 0.35 – 0.87 | <0.001
Sepal Length × Species [virginica] | 0.68 | 0.12 | 0.45 – 0.91 | <0.001
Observations | 150
R2 / R2 adjusted | 0.979 / 0.978

    # compare standard errors to result from sandwich-package
    unname(sqrt(diag(sandwich::vcovHC(model))))
    #> [1] 0.45382603 0.11884474 0.69296611 0.63031982 0.08318559 0.13045539 0.11841325

### Cluster-Robust Covariance Matrix Estimation (sandwich)

If another covariance matrix estimation is required, use the `vcov.fun`-argument. This argument needs the suffix for the related `vcov*()`-functions as value, i.e. `vcov.fun = "CL"` would call `sandwich::vcovCL()`, or `vcov.fun = "HAC"` would call `sandwich::vcovHAC()`.

The specific estimation type can be changed with `vcov.args`. E.g., `sandwich::vcovCL()` accepts estimation types HC0 to HC3. In the next example, we use a clustered covariance matrix estimation with HC1-estimation type.

    # change estimation-type
    tab_model(model, vcov.fun = "CL", vcov.args = list(type = "HC1"), show.se = TRUE)

| Petal Length
--- | ---
Predictors | Estimates | std. Error | CI | p
(Intercept) | 0.87 | 0.42 | 0.03 – 1.70 | 0.042
Sepal Length | 0.04 | 0.11 | -0.18 – 0.26 | 0.692
Species [versicolor] | -0.78 | 0.65 | -2.07 – 0.51 | 0.237
Species [virginica] | -0.41 | 0.59 | -1.57 – 0.75 | 0.483
Sepal Width | 0.11 | 0.08 | -0.05 – 0.27 | 0.170
Sepal Length × Species [versicolor] | 0.61 | 0.12 | 0.37 – 0.85 | <0.001
Sepal Length × Species [virginica] | 0.68 | 0.11 | 0.46 – 0.90 | <0.001
Observations | 150
R2 / R2 adjusted | 0.979 / 0.978

    # compare standard errors to result from sandwich-package
    unname(sqrt(diag(sandwich::vcovCL(model))))
    #> [1] 0.42197635 0.11148130 0.65274212 0.58720711 0.07934029 0.12251570 0.11058144

Usually, clustered covariance matrix estimation is used when there is a cluster-structure in the data. The variable indicating the cluster-structure can be defined in `sandwich::vcovCL()` with the `cluster`-argument. In `tab_model()`, additional arguments that should be passed down to functions from the sandwich package can be specified in `vcov.args`:

    iris$cluster <- factor(rep(LETTERS[1:8], length.out = nrow(iris)))
    # change estimation-type, defining additional arguments
    tab_model(
      model,
      vcov.fun = "CL",
      vcov.args = list(type = "HC1", cluster = iris$cluster),
      show.se = TRUE
    )

| Petal Length
--- | ---
Predictors | Estimates | std. Error | CI | p
(Intercept) | 0.87 | 0.34 | 0.20 – 1.53 | 0.011
Sepal Length | 0.04 | 0.07 | -0.10 – 0.19 | 0.540
Species [versicolor] | -0.78 | 0.52 | -1.80 – 0.25 | 0.137
Species [virginica] | -0.41 | 0.26 | -0.94 – 0.11 | 0.120
Sepal Width | 0.11 | 0.07 | -0.03 – 0.25 | 0.131
Sepal Length × Species [versicolor] | 0.61 | 0.10 | 0.42 – 0.80 | <0.001
Sepal Length × Species [virginica] | 0.68 | 0.05 | 0.58 – 0.78 | <0.001
Observations | 150
R2 / R2 adjusted | 0.979 / 0.978

    # compare standard errors to result from sandwich-package
    unname(sqrt(diag(sandwich::vcovCL(model, cluster = iris$cluster))))
    #> [1] 0.33714287 0.07192334 0.51893777 0.26415406 0.07201145 0.09661348 0.05123446

### Cluster-Robust Covariance Matrix Estimation (clubSandwich)

Cluster-robust estimation of the variance-covariance matrix can also be achieved using `clubSandwich::vcovCR()`. Thus, when `vcov.fun = "CR"`, the related function from the clubSandwich package is called. Note that this function requires the specification of the `cluster`-argument.

    # create fake-cluster-variable, to demonstrate cluster robust standard errors
    iris$cluster <- factor(rep(LETTERS[1:8], length.out = nrow(iris)))

    # cluster-robust estimation
    tab_model(
      model,
      vcov.fun = "CR1",
      vcov.args = list(cluster = iris$cluster),
      show.se = TRUE
    )

| Petal Length
--- | ---
Predictors | Estimates | std. Error | CI | p
(Intercept) | 0.87 | 0.33 | 0.21 – 1.52 | 0.010
Sepal Length | 0.04 | 0.07 | -0.10 – 0.18 | 0.531
Species [versicolor] | -0.78 | 0.51 | -1.78 – 0.23 | 0.129
Species [virginica] | -0.41 | 0.26 | -0.92 – 0.10 | 0.112
Sepal Width | 0.11 | 0.07 | -0.03 – 0.25 | 0.123
Sepal Length × Species [versicolor] | 0.61 | 0.09 | 0.42 – 0.79 | <0.001
Sepal Length × Species [virginica] | 0.68 | 0.05 | 0.58 – 0.78 | <0.001
Observations | 150
R2 / R2 adjusted | 0.979 / 0.978

    # compare standard errors to result from clubSsandwich-package
    unname(sqrt(diag(clubSandwich::vcovCR(model, type = "CR1", cluster = iris$cluster))))
    #> [1] 0.33028501 0.07046034 0.50838200 0.25878087 0.07054666 0.09464825 0.05019229

### Robust Covariance Matrix Estimation on Standardized Model Parameters

Finally, robust estimation can be combined with standardization. However, robust covariance matrix estimation only works for `show.std = "std"`.

    # model parameters, robust estimation on standardized model
    tab_model(
      model,
      show.std = "std",
      vcov.fun = "HC"
    )

| Petal Length
--- | ---
Predictors | Estimates | std. Beta | CI | standardized CI | p | std. p
(Intercept) | 0.87 | -1.30 | -0.03 – 1.76 | -1.44 – -1.16 | 0.059 | <0.001
Sepal Length | 0.04 | 0.02 | -0.19 – 0.28 | -0.09 – 0.13 | 0.711 | 0.711
Species [versicolor] | -0.78 | 1.57 | -2.15 – 0.59 | 1.40 – 1.74 | 0.265 | <0.001
Species [virginica] | -0.41 | 2.02 | -1.66 – 0.83 | 1.84 – 2.20 | 0.513 | <0.001
Sepal Width | 0.11 | 0.03 | -0.05 – 0.27 | -0.01 – 0.07 | 0.190 | 0.190
Sepal Length × Species [versicolor] | 0.61 | 0.28 | 0.35 – 0.87 | 0.16 – 0.41 | <0.001 | <0.001
Sepal Length × Species [virginica] | 0.68 | 0.32 | 0.45 – 0.91 | 0.21 – 0.43 | <0.001 | <0.001
Observations | 150
R2 / R2 adjusted | 0.979 / 0.978

## Mixed Models

### Robust Covariance Matrix Estimation for Mixed Models

For linear mixed models, that by definition have a clustered (“hierarchical” or multilevel) structure in the data, it is also possible to estimate a cluster-robust covariance matrix. This is possible due to the clubSandwich package, thus we need to define the same arguments as in the above example.

    library(lme4)
    data(iris)
    set.seed(1234)
    iris$grp <- as.factor(sample(1:3, nrow(iris), replace = TRUE))

    # fit example model
    model <- lme4::lmer(
      Sepal.Length ~ Species * Sepal.Width + Petal.Length + (1 | grp),
      data = iris
    )

    # normal model parameters, like from 'summary()'
    tab_model(model)

| Sepal Length
--- | ---
Predictors | Estimates | CI | p
(Intercept) | 1.55 | 0.76 – 2.35 | <0.001
Species [versicolor] | 0.41 | -0.67 – 1.50 | 0.454
Species [virginica] | -0.41 | -1.56 – 0.74 | 0.483
Sepal Width | 0.66 | 0.44 – 0.89 | <0.001
Petal Length | 0.82 | 0.69 – 0.95 | <0.001
Species [versicolor] × Sepal Width | -0.48 | -0.85 – -0.12 | 0.010
Species [virginica] × Sepal Width | -0.36 | -0.71 – -0.00 | 0.048
Random Effects
σ2 | 0.09
τ00 grp | 0.01
ICC | 0.07
N grp | 3
Observations | 150
Marginal R2 / Conditional R2 | 0.860 / 0.870

    # model parameters, cluster robust estimation for mixed models
    tab_model(
      model,
      vcov.fun = "CR1",
      vcov.args = list(cluster = iris$grp)
    )

| Sepal Length
--- | ---
Predictors | Estimates | CI | p
(Intercept) | 1.55 | 0.76 – 2.35 | <0.001
Species [versicolor] | 0.41 | -1.17 – 1.99 | 0.608
Species [virginica] | -0.41 | -0.78 – -0.03 | 0.033
Sepal Width | 0.66 | 0.46 – 0.86 | <0.001
Petal Length | 0.82 | 0.72 – 0.91 | <0.001
Species [versicolor] × Sepal Width | -0.48 | -1.18 – 0.21 | 0.172
Species [virginica] × Sepal Width | -0.36 | -0.57 – -0.15 | 0.001
Random Effects
σ2 | 0.09
τ00 grp | 0.01
ICC | 0.07
N grp | 3
Observations | 150
Marginal R2 / Conditional R2 | 0.860 / 0.870

### Robust Covariance Matrix Estimation on Standardized Mixed Model Parameters

Again, robust estimation can be combined with standardization for linear mixed models as well, which in such cases also only works for `show.std = "std"`.

    # model parameters, cluster robust estimation on standardized mixed model
    tab_model(
      model,
      show.std = "std",
      vcov.fun = "CR1",
      vcov.args = list(cluster = iris$grp)
    )

| Sepal Length
--- | ---
Predictors | Estimates | std. Beta | CI | standardized CI | p | std. p
(Intercept) | 1.55 | 0.97 | 0.76 – 2.35 | 0.82 – 1.12 | <0.001 | <0.001
Species [versicolor] | 0.41 | -1.29 | -1.17 – 1.99 | -1.95 – -0.63 | 0.608 | <0.001
Species [virginica] | -0.41 | -1.81 | -0.78 – -0.03 | -2.26 – -1.37 | 0.033 | <0.001
Sepal Width | 0.66 | 0.35 | 0.46 – 0.86 | 0.24 – 0.45 | <0.001 | <0.001
Petal Length | 0.82 | 1.74 | 0.72 – 0.91 | 1.54 – 1.94 | <0.001 | <0.001
Species [versicolor] × Sepal Width | -0.48 | -0.25 | -1.18 – 0.21 | -0.62 – 0.11 | 0.172 | 0.172
Species [virginica] × Sepal Width | -0.36 | -0.19 | -0.57 – -0.15 | -0.30 – -0.08 | 0.001 | 0.001
Random Effects
σ2 | 0.09
τ00 grp | 0.01
ICC | 0.07
N grp | 3
Observations | 150
Marginal R2 / Conditional R2 | 0.860 / 0.870