Citation:

Gabriel Terna Ayem, Ozcan Asilkan, Aamo Iorliam, Rabiu Ibrahim, Salu George Thandekkattu, 2023. Causal Inference Estimates with Backdoor Adjustment Condition vs. the Unconfoundedness Assumption: A Comparative Analysis Study of the Structural Causal Model and the Potential Outcome Frameworks, ESP Journal of Engineering & Technology Advancements  3(4): 1-22.

Abstract:

We present an empirical and theoretical comparative analysis of the structural causal model (SCM) and the potential outcome (PO) frameworks under the presence of biases of confounding and selection in the dataset. We used the Early Grade Reading Assessment (EGRA) dataset tagged “Strengthening Education in Northeast (SENSE) Nigeria, - an educational intervention program of the American University of Nigeria under the sponsorship of the United States Agency for International Development (USAID). The SCM backdoor adjustment criteria and the unconfoundedness assumption of the PO framework and other assumptions are employed to overcome confounding bias in the dataset to estimate the causal inference of the SENSE-EGRA intervention program. Further, we employed 4 statistical estimators for the adjustments of covariates and the estimation of the causal inference for the intervention program. These estimators are structured to handle selection bias during covariates adjustment, by carefully matching up the treated and control units in the dataset. The 4 estimators are. Viz. The ordinary least square regression adjustment (OLS), the propensity score weighting (PSW), the propensity score stratification (PSS), and the propensity score matching (PSM) adjustments. The results evinced higher average treatment effects (ATE) estimates for the SCM framework than that of the PO’s ATE estimates, which are (SCM’). Viz. OLS = 0.661, the PSS = 1.592, the PSM = 2.173, and the PSW = 4.931, while the PO is. Viz. The PSW = 0.035, the PSS = 0.036, the OLS = 0.079, and the PSM = 0.066. These disparities in the ATE results for the two frameworks are due to the fact, that the covariates adjustment was applied to all the covariates in the dataset including the mediator variable under the PO framework, which is considered a forbidden act under the SCM framework. Additionally, the explicit representation of the dataset generation process in a direct acyclic graph (DAG), and the concomitant proving of the same, clarifies and validates the inference estimates from the SCM framework - a key component that is lacking in the PO framework.

References:

[1] J. Tian, and J. Pearl, A general identification condition for causal effects: eScholarship, University of California, 2002.
[2] J. Pearl, and T. S. Verma, "A theory of inferred causation," Studies in Logic and the Foundations of Mathematics, pp. 789-811: Elsevier, 1995.
[3] J. Pearl, and D. Mackenzie, The book of why: the new science of cause and effect: Basic books, 2018.
[4] J. Pearl, Causality: Cambridge university press, 2009.
[5] C. Cinelli, D. Kumor, B. Chen, J. Pearl, and E. Bareinboim, "Sensitivity analysis of linear structural causal models." pp. 1252-1261.
[6] J. Y. Halpern, "The Book of Why, Judea Pearl, Basic Books (2018)," Elsevier, 2019.
[7] D. B. Rubin, “Causal inference using potential outcomes: Design, modeling, decisions,” Journal of the American Statistical Association, vol. 100, no. 469, pp. 322-331, 2005.
[8] G. W. Imbens, and D. B. Rubin, "Rubin causal model," Microeconometrics, pp. 229-241: Springer, 2010.
[9] D. Aliprantis, “A distinction between causal effects in Structural and Rubin Causal Models,” 2015.
[10] D. B. Rubin, E. A. Stuart, and E. L. Zanutto, “A potential outcomes view of value-added assessment in education,” Journal of educational and behavioral statistics, vol. 29, no. 1, pp. 103-116, 2004.
[11] D. B. Rubin, “Estimating causal effects of treatments in randomized and nonrandomized studies,” Journal of educational Psychology, vol. 66, no. 5, pp. 688, 1974.
[12] T. J. VanderWeele, “Ignorability and stability assumptions in neighborhood effects research,” Statistics in medicine, vol. 27, no. 11, pp. 1934-1943, 2008.
[13] D. F. Heitjan, and D. B. Rubin, “Ignorability and coarse data,” The annals of statistics, pp. 2244-2253, 1991.
[14] C. A. Flores, and A. Flores-Lagunes, “Identification and estimation of causal mechanisms and net effects of a treatment under unconfoundedness,” 2009.
[15] J. M. Wooldridge, “Violating ignorability of treatment by controlling for too many factors,” Econometric Theory, vol. 21, no. 5, pp. 1026-1028, 2005.
[16] Y. Zhang, A. Bellot, and M. Schaar, "Learning overlapping representations for the estimation of individualized treatment effects." pp. 1005-1014.
[17] V. Karwa, A. B. Slavković, and E. T. Donnell, “Causal inference in transportation safety studies: Comparison of potential outcomes and causal diagrams,” 2011.
[18] K. A. Markus, “Causal effects and counterfactual conditionals: contrasting Rubin, Lewis and Pearl,” Economics & Philosophy, vol. 37, no. 3, pp. 441-461, 2021.
[19] G. W. Imbens, “Potential outcome and directed acyclic graph approaches to causality: Relevance for empirical practice in economics,” Journal of Economic Literature, vol. 58, no. 4, pp. 1129-1179, 2020.
[20] A. U. o. N. (AUN), "SENSE PROJECT EVALUATION: READING PROFICIENCY ASSESSMENT, RESULTS OF THE HAUSA EARLY GRADE READING ASSESSMENT (EGRA) IN ADAMAWA AND GOMBE," 2022.
[21] S. Bhaskaran, and R. Marappan, “Enhanced personalized recommendation system for machine learning public datasets: generalized modeling, simulation, significant results and analysis,” International Journal of Information Technology, vol. 15, no. 3, pp. 1583-1595, 2023.
[22] B. Piper, S. Dryden-Peterson, V. Chopra, C. Reddick, and A. Oyanga, “Are refugee children learning? Early grade literacy in a refugee camp in Kenya,” 2020.
[23] S. Taylor, J. Cilliers, C. Prinsloo, B. Fleisch, and V. Reddy, “The Early Grade Reading Study: Impact evaluation after two years of interventions,” EGRS Evaluafion Report, 2017.
[24] B. Fleisch, V. Schöer, G. Roberts, and A. Thornton, “System-wide improvement of early-grade mathematics: New evidence from the Gauteng Primary Language and Mathematics Strategy,” International Journal of Educational Development, vol. 49, pp. 157-174, 2016.
[25] L. O. Costa, and M. Carnoy, “The effectiveness of an early-grade literacy intervention on the cognitive achievement of Brazilian students,” Educational Evaluation and Policy Analysis, vol. 37, no. 4, pp. 567-590, 2015.
[26] B. Piper, and M. Korda, “EGRA Plus: Liberia. Program Evaluation Report,” RTI International, 2011.
[27] M. Davidson, M. Korda, and O. W. Collins, “Teachers’ use of EGRA for continuous assessment: the case of EGRA Plus: Liberia,” The Early Grade Reading Assessment, pp. 113, 2011.
[28] M. Davidson, and J. Hobbs, “Delivering reading intervention to the poorest children: The case of Liberia and EGRA-Plus, a primary grade reading assessment and intervention,” International Journal of Educational Development, vol. 33, no. 3, pp. 283-293, 2013.
[29] M. M. de Oca, J. Hill, L. Aber, C. T. Dolan, and K. Gjicali, “The Impact of Attending a Remedial Support Program on Syrian Children's Reading Skills: Using BART for Causal Inference,” arXiv preprint arXiv:2208.13906, 2022.
[30] A. Ankan, I. M. Wortel, and J. Textor, “Testing graphical causal models using the R package “dagitty”,” Current Protocols, vol. 1, no. 2, pp. e45, 2021.
[31] I. Shpitser, T. VanderWeele, and J. M. Robins, “On the validity of covariate adjustment for estimating causal effects,” arXiv preprint arXiv:1203.3515, 2012.
[32] B. Van der Zander, M. Liskiewicz, and J. Textor, "Constructing Separators and Adjustment Sets in Ancestral Graphs." pp. 11-24.
[33] E. Perkovic, J. Textor, M. Kalisch, and M. H. Maathuis, “Complete graphical characterization and construction of adjustment sets in Markov equivalence classes of ancestral graphs,” 2018.
[34] P. C. Austin, and E. A. Stuart, “Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies,” Statistics in medicine, vol. 34, no. 28, pp. 3661-3679, 2015.
[35] P. R. Rosenbaum, and D. B. Rubin, “The central role of the propensity score in observational studies for causal effects,” Biometrika, vol. 70, no. 1, pp. 41-55, 1983.
[36] T. J. VanderWeele, and I. Shpitser, “A new criterion for confounder selection,” Biometrics, vol. 67, no. 4, pp. 1406-1413, 2011.
[37] J. L. Hill, “Bayesian nonparametric modeling for causal inference,” Journal of Computational and Graphical Statistics, vol. 20, no. 1, pp. 217-240, 2011.
[38] F. Johansson, U. Shalit, and D. Sontag, "Learning representations for counterfactual inference." pp. 3020-3029.
[39] C. Louizos, U. Shalit, J. M. Mooij, D. Sontag, R. Zemel, and M. Welling, “Causal effect inference with deep latent-variable models,” Advances in neural information processing systems, vol. 30, 2017.
[40] D. Sontag, U. Shalit, and F. D. Johansson, “Estimating individual treatment effect: Generalization bounds and algorithms,” 2017.
[41] S. Weichenthal, S. Ripley, and J. Korsiak, “Fine particulate air pollution and the “no-multiple-versions-of-treatment” assumption: does particle composition matter for causal inference?,” American Journal of Epidemiology, vol. 192, no. 2, pp. 147-153, 2023.
[42] M. Lechner, “Earnings and employment effects of continuous gff-the-job training in east germany after unification,” Journal of Business & Economic Statistics, vol. 17, no. 1, pp. 74-90, 1999.
[43] J. Pearl, “Causal inference in statistics: An overview,” Statistics surveys, vol. 3, pp. 96-146, 2009.
[44] T. Emura, J. Wang, and H. Katsuyama, Assessing the assumption of strongly ignorable treatment assignment under assumed causal models, Technical Report of Mathematical Sciences, 2008.
[45] A. Chesher, and E. Battistin, "The impact of measurement error on evaluation methods based on strong ignorability."
[46] R. H. Dehejia, and S. Wahba, “Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs,” Journal of the American statistical Association, vol. 94, no. 448, pp. 1053-1062, 1999.
[47] R. H. Dehejia, and S. Wahba, “Propensity score-matching methods for nonexperimental causal studies,” Review of Economics and statistics, vol. 84, no. 1, pp. 151-161, 2002.
[48] T. J. VanderWeele, and O. A. Arah, “Unmeasured confounding for general outcomes, treatments, and confounders: bias formulas for sensitivity analysis,” Epidemiology (Cambridge, Mass.), vol. 22, no. 1, pp. 42, 2011.
[49] R. Guo, L. Cheng, J. Li, P. R. Hahn, and H. Liu, “A survey of learning causality with data: Problems and methods,” ACM Computing Surveys (CSUR), vol. 53, no. 4, pp. 1-37, 2020.
[50] P. C. Austin, “An introduction to propensity score methods for reducing the effects of confounding in observational studies,” Multivariate behavioral research, vol. 46, no. 3, pp. 399-424, 2011.
[51] K. Imai, and M. Ratkovic, “Estimating treatment effect heterogeneity in randomized program evaluation,” The Annals of Applied Statistics, vol. 7, no. 1, pp. 443-470, 2013.
[52] J. K. Lunceford, and M. Davidian, “Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study,” Statistics in medicine, vol. 23, no. 19, pp. 2937-2960, 2004.
[53] S. L. Morgan, and C. Winship, Counterfactuals and causal inference: Cambridge University Press, 2015.
[54] K. Hirano, G. W. Imbens, and G. Ridder, “Efficient estimation of average treatment effects using the estimated propensity score,” Econometrica, vol. 71, no. 4, pp. 1161-1189, 2003.
[55] M. J. Funk, D. Westreich, C. Wiesen, T. Stürmer, M. A. Brookhart, and M. Davidian, “Doubly robust estimation of causal effects,” American journal of epidemiology, vol. 173, no. 7, pp. 761-767, 2011.
[56] M. M. Joffe, T. R. Ten Have, H. I. Feldman, and S. E. Kimmel, “Model selection, confounder control, and marginal structural models: review and new applications,” The American Statistician, vol. 58, no. 4, pp. 272-279, 2004.
[57] M. Á. Hernán, B. Brumback, and J. M. Robins, “Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men,” Epidemiology, pp. 561-570, 2000.
[58] X. S. Gu, and P. R. Rosenbaum, “Comparison of multivariate matching methods: Structures, distances, and algorithms,” Journal of Computational and Graphical Statistics, vol. 2, no. 4, pp. 405-420, 1993.
[59] O. Tinubu, A. Sodiya, and O. Ojesanmi, “A behavioral model for characterizing flooding distributed denial of service attacks,” International Journal of Information Technology, vol. 15, no. 2, pp. 955-964, 2023.
[60] A. Thukral, S. Dhiman, R. Meher, and P. Bedi, “Knowledge graph enrichment from clinical narratives using NLP, NER, and biomedical ontologies for healthcare applications,” International Journal of Information Technology, vol. 15, no. 1, pp. 53-65, 2023.
[61] J. Pearl, “Models, reasoning and inference,” Cambridge, UK: CambridgeUniversityPress, vol. 19, no. 2, 2000.
[62] E. Bareinboim, J. Tian, and J. Pearl, “Recovering from Selection Bias in Causal and Statistical Inference (Supplemental Material),” 2014.
[63] E. Bareinboim, and J. Pearl, "Controlling selection bias in causal inference." pp. 100-108.
[64] P. W. Tennant, E. J. Murray, K. F. Arnold, L. Berrie, M. P. Fox, S. C. Gadd, W. J. Harrison, C. Keeble, L. R. Ranker, and J. Textor, “Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: review and recommendations,” International journal of epidemiology, vol. 50, no. 2, pp. 620-632, 2021.
[65] L. Wong, Three essays in causal inference: Stanford University, 2015.
[66] L. Wong, "Causalinference in Python Documentation," 2016.
[67] L. Wong, "Causal Inference in Python," Causal Inference in Python Blog, 3rd, Nov, 2022, L. Wong, ed., 2016.
[68] A. Sharma, and E. Kiciman, “DoWhy: An end-to-end library for causal inference,” arXiv preprint arXiv:2011.04216, 2020.
[69] S. Schwartz, N. M. Gatto, and U. B. Campbell, “Extending the sufficient component cause model to describe the Stable Unit Treatment Value Assumption (SUTVA),” Epidemiologic Perspectives & Innovations, vol. 9, no. 1, pp. 1-11, 2012.
[70] A.-G. Dosne, M. Bergstrand, K. Harling, and M. O. Karlsson, “Improving the estimation of parameter uncertainty distributions in nonlinear mixed effects models using sampling importance resampling,” Journal of pharmacokinetics and pharmacodynamics, vol. 43, pp. 583-596, 2016.
[71] L. A. Aaron, and D. Buchwald, “A review of the evidence for overlap among unexplained clinical conditions,” Annals of internal medicine, vol. 134, no. 9_Part_2, pp. 868-881, 2001.

Keywords:

Backdoor Adjustment, Comparative Research, Potential Outcome, Structural Causal Model, Unconfoundedness Assumption.