# Additive Bayesian Networks

#### Matteo Delucchi, Jonas I. Liechti, Reinhard Furrer

#### 17. March 2024

# Summary

The `abn` package is a comprehensive tool for Bayesian network (BN) analysis, a form of probabilistic graphical model. It derives a directed acyclic graph from empirical data, describing the dependency structure between random variables. This package provides routines for structure learning and parameter estimation of additive Bayesian network (ABN) models.

BNs are a type of statistical model that leverages the principles of Bayesian statistics and graph theory to provide a framework for representing complex multivariate data. ABN models extend the concept of generalized linear models, typically used for predicting a single outcome, to scenarios with multiple dependent variables (e.g. Kratzer et al. (2023)). This makes them a powerful tool for understanding complex, multivariate datasets.

# Statment of need

The need for a tool like `abn` arises from the increasing complexity of data in various fields. Researchers often face multivariate, tabular data where the relationships between variables are not straightforward. BN analysis becomes essential when traditional statistical methods fall short in analyzing multivariate data with intricate relationships, as it models these relationships graphically for more straightforward data interpretation.

However, most existing implementations of BN models limit variable types, often allowing discrete variables to have only discrete parent variables, where a parent starts a directed edge in the graph. This limitation can pose challenges when dealing with continuous or mixed-type data (i.e., data that includes both continuous and discrete variables) or when attempting to model complex relationships that do not fit these restricted categories. Further details have been discussed in the context of patient data in the study from Delucchi et al. (2022), particularly focusing on the widely used `bnlearn` package (Scutari 2010) and the `abn` package.

The `abn` package overcomes these limitations through its additive model formulation, which generalizes the usual (Bayesian) multivariable regression to accommodate multiple dependent variables. Additionally, the `abn` package offers a comprehensive suite of features for model selection, structure learning, and parameter estimation. It includes exact and greedy search algorithms for structure learning and allows for integrating prior expert knowledge into the model selection process by specifying structural constraints. Unlike other software, `abn` offers a Bayesian and information-theoretic model scoring approach. Furthermore, it supports mixed-effect models to control one-layer clustering, making it suitable, e.g., for handling data from different sources.

Previous versions of the `abn` package have been successfully used in various fields, including epidemiology Kratzer and Furrer (2018) and health Delucchi et al. (2022). Despite its promise, the `abn` package encountered historical obstacles. Sporadic maintenance and an incomplete codebase hindered its full potential. Recognizing the need for enhancement, we undertook a substantial upgrade and meticulously addressed legacy issues, revamped the codebase, and introduced significant improvements. The latest version 3 of `abn` is now a robust and reliable tool for BN analysis. Applying the latest standards for open-source software, we guarantee active maintenance of `abn`. Future updates are planned to enhance its functionality and user experience further. We highly value feedback from the user community, which will guide our ongoing developments.

In summary, `abn` sets itself apart by emphasizing ABNs and its exhaustive features for model selection and structure learning. Its unique contribution is the implementation of mixed-effect BN models, thereby extending its applicability to a broader range of complex, multivariate datasets of mixed, continuous and discrete data.

# Implementation

As outlined in Kratzer et al. (2023), the package’s comprehensive framework integrates the mixed-effects model for clustered data, considering data heterogeneity and grouping effects. However, this was confined to a Bayesian context. The implementation under the information-theoretic (“mle”) setting was notably missing in previous versions, an omission that has been rectified in the current version 3 onwards.

Analyzing hierarchical or grouped data, i.e., observations nested within higher-level units, requires statistical models with group-varying parameters (e.g., mixed-effect models). The `abn` package facilitates single-layer clustering, where observations are grouped into a single layer of clusters. These clusters are assumed to be independent, but intra-cluster observations may exhibit correlation (e.g., students within schools, patient-specific measurements over time, etc.). The ABN model is fitted independently as a varying intercept model, where the intercept can vary while the slope is assumed constant across all group levels.

Under the frequentist paradigm (`method = "mle"`), `abn` employs the `lme4` package (Bates et al. 2015) to fit generalized linear mixed models for each of the Binomial, Poisson, and Gaussian distributed variables. For multinomial distributed variables, `abn` fits a multinomial baseline category logit model with random effects using the `mclogit` package (Elff 2022). Currently, only single-layer clustering is supported (e.g., for `method = "mle"`, this corresponds to a random intercept model).

With a Bayesian approach (`method = "bayes"`), `abn` utilizes its own implementation of the Laplace approximation as well as the `INLA` package (Martins et al. 2013) to fit a single-level hierarchical model for Binomial, Poisson, and Gaussian distributed variables. Independent of the type of data, multinomial distributed variables are not yet implemented with `method ="bayes"` (details in the online manual).

Furthermore, the code base has been enhanced to be more efficient, reliable, and user-friendly through code optimization, regular reviews and continuous integration practices. We have adhered to the latest open-source software standards, including active maintenance of `abn`. Future updates to augment its functionality are planned via a flexible roadmap. User feedback is valued through open communication channels, which will steer our ongoing developments. Consequently, the latest version of `abn` is now a robust and reliable tool for BN analysis.

# Validation and Testing

A comprehensive set of documented case studies has been published to validate the `abn` package (see the `abn` website). The numerical accuracy and quality assurance exercises were demonstrated in Kratzer et al. (2023). A rigorous testing framework is implemented using the `testthat` package (Wickham 2011), which is executed as part of an extensive continuous integration pipeline designed explicitly for non-standard R packages that rely on `Rcpp` (Eddelbuettel et al. 2023) and `JAGS` (Plummer 2003). Additional documentation and resources are available on the `abn` website for further reference and guidance.

# Availability

The `abn` package is available on CRAN and can be installed with the R command:

    install.packages("abn")

The development version of the `abn` package is hosted on GitHub and can be installed using the `devtools` package:

    devtools::install_github("furrer-lab/abn")

# Acknowledgments

The development of the `abn` package would not have been possible without the significant contributions of the former developers whose efforts have been instrumental in shaping this project. We acknowledge the contributions of Fraser Iain Lewis, Marta Pittavino, Gilles Kratzer, and Kalina Cherneva, in particular. We want to extend our gratitude to the faculty staff at the Department of Mathematical Modeling and Machine Learning from the University of Zurich and the Institute of Mathematics who maintain the research and teaching infrastructure. Our appreciation also goes to the UZH and the ZHAW for their financial support. We would like to highlight the funding from the Digitalization Initiative of the Zurich Higher Education Institutions (DIZH), which was instrumental in the realization of this project, particularly within the context of the “Modeling of multicentric and dynamic stroke health data” and “Stroke DynamiX” projects. This work has been conducted as part of M.D.’s PhD project, which is supervised by R.F.

# References

Bates, Douglas, Martin Mächler, Ben Bolker, and Steve Walker. 2015. “Fitting Linear Mixed-Effects Models Using lme4.” Journal of Statistical Software 67 (1): 1–48. https://doi.org/10.18637/jss.v067.i01.
Delucchi, Matteo, Georg R. Spinner, Marco Scutari, Philippe Bijlenga, Sandrine Morel, Christoph M. Friedrich, Reinhard Furrer, and Sven Hirsch. 2022. “Bayesian Network Analysis Reveals the Interplay of Intracranial Aneurysm Rupture Risk Factors.” Computers in Biology and Medicine 147: 105740. https://doi.org/10.1016/j.compbiomed.2022.105740.
Eddelbuettel, Dirk, Romain Francois, JJ Allaire, Kevin Ushey, Qiang Kou, Nathan Russell, Inaki Ucar, Douglas Bates, and John Chambers. 2023. Rcpp: Seamless r and c++ Integration. https://CRAN.R-project.org/package=Rcpp.
Elff, Martin. 2022. Mclogit: Multinomial Logit Models, with or Without Random Effects or Overdispersion. https://CRAN.R-project.org/package=mclogit.
Hartnack, Sonja, Terence Odoch, Gilles Kratzer, Reinhard Furrer, Yngvild Wasteson, Trine M. L’Abée-Lund, and Eystein Skjerve. 2019. “Additive Bayesian Networks for Antimicrobial Resistance and Potential Risk Factors in Non-Typhoidal Salmonella Isolates from Layer Hens in Uganda.” BMC Veterinary Research 15: 212. https://doi.org/10.1186/s12917-019-1965-y.
Kratzer, Gilles, and Reinhard Furrer. 2018. “Information-Theoretic Scoring Rules to Learn Additive Bayesian Network Applied to Epidemiology.” arXiv:1808.01126 [Cs, Stat]. http://arxiv.org/abs/1808.01126.
Kratzer, Gilles, Fraser I. Lewis, Arianna Comin, Marta Pittavino, and Reinhard Furrer. 2023. “Additive Bayesian Network Modeling with the r Package Abn.” Journal of Statistical Software 105: 1–41. https://doi.org/10.18637/jss.v105.i08.
Kratzer, Gilles, Fraser I. Lewis, Barbara Willi, Marina L. Meli, Felicitas S. Boretti, Regina Hofmann-Lehmann, Paul Torgerson, Reinhard Furrer, and Sonja Hartnack. 2020. “Bayesian Network Modeling Applied to Feline Calicivirus Infection Among Cats in Switzerland.” Frontiers in Veterinary Science 7. https://doi.org/10.3389/fvets.2020.00073.
Martins, Thiago G., Daniel Simpson, Finn Lindgren, and Håvard Rue. 2013. “Bayesian Computing with INLA: New Features.” Computational Statistics and Data Analysis 67: 68–83.
Pittavino, Marta, Anou Dreyfus, Cord Heuer, Jackie Benschop, Peter Wilson, Julie Collins-Emerson, Paul R. Torgerson, and Reinhard Furrer. 2017. “Comparison Between Generalized Linear Modelling and Additive Bayesian Network; Identification of Factors Associated with the Incidence of Antibodies Against Leptospira Interrogans Sv Pomona in Meat Workers in New Zealand.” Acta Tropica 173: 191–99. https://doi.org/10.1016/j.actatropica.2017.04.034.
Plummer, Martyn. 2003. “JAGS: A Program for Analysis of Bayesian Graphical Models Using Gibbs Sampling.” In Proceedings of the 3rd International Workshop on Distributed Statistical Computing, 1–10. Vienna, Austria.
Scutari, Marco. 2010. “Learning Bayesian Networks with the bnlearn R Package.” Journal of Statistical Software 35: 1–22. https://doi.org/10.18637/jss.v035.i03.
Wickham, Hadley. 2011. “Testthat: Get Started with Testing.” The R Journal 3: 5–10. https://journal.r-project.org/archive/2011-1/RJournal_2011-1_Wickham.pdf.