Thanks so much for your comment. This is exactly the kind of tone I was hoping for.
Without having some causal explanation in mind, you don't know what to put in the model. This was Simpson's point (in "Simpson's Paradox," https://wildetruth.substack.com/p/simpsons-paradox-and-existential). So we are really doing story-telling with statistics, whether we're aware of it, or not. And must (ethically) make it scientific in the best way we know how -- which means using DAGs. Otherwise you wind up, like consensus abortion science, relying on models that "correct for" confounds that are really colliders, risking introducing more bias than they correct for because structural thinking about causality was missing.
What I've understood from Sander Greenland and Richard McElreath is that there's no need to pick a dog in the frequentist-Bayesian fight. They're just different tools best-suited to different jobs (SG), and that's a Boomer battle (RM).
We definitely need to be doing the most ethical science possible by using the best methods from the start and iterating in an open, honest way.
Regarding statistical significance, you might be interested in Norbert Hirschauer's “Some Thoughts About Statistical Inference in the 21st Century" (https://osf.io/preprints/socarxiv/exdfg/). His emphasis is on good inferential practices.
In Richard McElreath's amazing "Statistical Rethinking" course, he was just talking about how selecting for the best AIC as a stand-in for real causal inference is bad news. https://www.youtube.com/watch?v=Y2ZLt4iOrXU&list=RDCMUCNJK6_DZvcMqNSzQdEkzvzA&start_radio=1 The lectures are all free online, and I can't recommend them enough. One of his points throughout is that you (1) define some theoretical estimand, (2) build scientific (causal) models, (3) build statistical models from those, (4) simulate from (2) to validate (3) yields (1), and then (5) analyze real data ("drawing the Bayesian owl"). Again the emphasis here is on good inferential practices. Echoes of Simpson, who warned, essentially -- "you want a rule of thumb for doing good statistics, but you can't have one because this is story-telling and you have to actually think about the story."
Regarding expecting data to be used in adversarial ways, you might be interested in Sander Greenland's "Critical Appraisal of Expert Witnesses" piece I referenced at the end of the second part of this series: https://wildetruth.substack.com/p/abortion-myths-part-2.
I think the main point you raise in closing, about non-neutrality in science, is really important and in some ways irresolveable. It got its own post in this series here: https://wildetruth.substack.com/p/abortion-myths-part-3.
Thanks so much for your comment. This is exactly the kind of tone I was hoping for.
Without having some causal explanation in mind, you don't know what to put in the model. This was Simpson's point (in "Simpson's Paradox," https://wildetruth.substack.com/p/simpsons-paradox-and-existential). So we are really doing story-telling with statistics, whether we're aware of it, or not. And must (ethically) make it scientific in the best way we know how -- which means using DAGs. Otherwise you wind up, like consensus abortion science, relying on models that "correct for" confounds that are really colliders, risking introducing more bias than they correct for because structural thinking about causality was missing.
It's an iterative process, and that's not new. https://statmodeling.stat.columbia.edu/2014/07/03/great-advantage-model-based-ad-hoc-ap-proach-seems-given-time-know/
What I've understood from Sander Greenland and Richard McElreath is that there's no need to pick a dog in the frequentist-Bayesian fight. They're just different tools best-suited to different jobs (SG), and that's a Boomer battle (RM).
We definitely need to be doing the most ethical science possible by using the best methods from the start and iterating in an open, honest way.
Regarding statistical significance, you might be interested in Norbert Hirschauer's “Some Thoughts About Statistical Inference in the 21st Century" (https://osf.io/preprints/socarxiv/exdfg/). His emphasis is on good inferential practices.
In Richard McElreath's amazing "Statistical Rethinking" course, he was just talking about how selecting for the best AIC as a stand-in for real causal inference is bad news. https://www.youtube.com/watch?v=Y2ZLt4iOrXU&list=RDCMUCNJK6_DZvcMqNSzQdEkzvzA&start_radio=1 The lectures are all free online, and I can't recommend them enough. One of his points throughout is that you (1) define some theoretical estimand, (2) build scientific (causal) models, (3) build statistical models from those, (4) simulate from (2) to validate (3) yields (1), and then (5) analyze real data ("drawing the Bayesian owl"). Again the emphasis here is on good inferential practices. Echoes of Simpson, who warned, essentially -- "you want a rule of thumb for doing good statistics, but you can't have one because this is story-telling and you have to actually think about the story."
Regarding expecting data to be used in adversarial ways, you might be interested in Sander Greenland's "Critical Appraisal of Expert Witnesses" piece I referenced at the end of the second part of this series: https://wildetruth.substack.com/p/abortion-myths-part-2.
I think the main point you raise in closing, about non-neutrality in science, is really important and in some ways irresolveable. It got its own post in this series here: https://wildetruth.substack.com/p/abortion-myths-part-3.