tag:blogger.com,1999:blog-26951738.post3794147198774829515..comments2024-03-25T11:49:21.281-07:00Comments on The Splintered Mind: What a Non-Effect Looks LikeEric Schwitzgebelhttp://www.blogger.com/profile/11541402189204286449noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-26951738.post-1219120171801089162013-08-08T13:23:15.579-07:002013-08-08T13:23:15.579-07:00The first point is just an observation about typic...The first point is just an observation about typical meta-analyses that I have seen in the literature. The second point is an explanation of why the first might be so.Eric Schwitzgebelhttps://www.blogger.com/profile/11541402189204286449noreply@blogger.comtag:blogger.com,1999:blog-26951738.post-80940644186494623692013-08-08T11:54:16.141-07:002013-08-08T11:54:16.141-07:00When you say "Mathematically, if you combine ...When you say "Mathematically, if you combine one-third null-effect studies with two-thirds positive-effect studies, you'll typically find a statistically significant effect (even with the typical "file-drawer" corrections)." <br /><br />Are you referring to "This correction works if you assume that there isn't a systematic bias in the data collection *other* than the overrepresentation of statistically significant findings, e.g., it does nothing to correct for an uncontrolled positive confound throughout the literature." or do you also have some other issue in mind? jsalvatihttps://www.blogger.com/profile/16509764680257537430noreply@blogger.comtag:blogger.com,1999:blog-26951738.post-4774732938315768152013-08-07T14:19:38.333-07:002013-08-07T14:19:38.333-07:00Cool, thanks. I was not familiar with that term.
...Cool, thanks. I was not familiar with that term.<br />G. Randolph Mayeshttps://www.blogger.com/profile/18285281186698499962noreply@blogger.comtag:blogger.com,1999:blog-26951738.post-17672716468322584252013-08-07T11:24:21.334-07:002013-08-07T11:24:21.334-07:00Thanks for the kind comments, Randy and Jonathan!
...Thanks for the kind comments, Randy and Jonathan!<br /><br />Randy: Yes, this is the "file drawer" phenomenon. It is not uncommon for meta-analyses, following Rosenthal, to use a file-drawer correction assuming that null-effect studies are less likely to be published. This correction works if you assume that there isn't a systematic bias in the data collection *other* than the overrepresentation of statistically significant findings, e.g., it does nothing to correct for an uncontrolled positive confound throughout the literature.Eric Schwitzgebelhttps://www.blogger.com/profile/11541402189204286449noreply@blogger.comtag:blogger.com,1999:blog-26951738.post-1901173658620405352013-08-07T10:38:38.827-07:002013-08-07T10:38:38.827-07:00Great post, Eric!
Great post, Eric!<br />jonathan weinbergnoreply@blogger.comtag:blogger.com,1999:blog-26951738.post-55507081420496822002013-08-07T10:06:16.138-07:002013-08-07T10:06:16.138-07:00Eric, nice piece. Are studies concluding that ther...Eric, nice piece. Are studies concluding that there is no effect substantially more likely to be published in this field than in other scientific fields? If not, then shouldn't part of our picture of a non effect consider the systematic bias toward publishing the 5% of findings likely to be due to chance?G. Randolph Mayeshttps://www.blogger.com/profile/18285281186698499962noreply@blogger.com