The quality of scientific publications suffered during the Covid-19 pandemic. From dubious prepublication studies to bombshell papers quietly retracted months later, it has been a busy but difficult time for major medical and epidemiology journals. Peer-review and clinical-trial processes have sped up, and few would claim this generally improves the quality of the research or its conclusions.
The same has happened in the less rigorous world of policy analysis. Working groups and research collaboratives have issued recommendations to government, businesses, universities and other institutions without even a cursory peer review. Often these groups offer more encouragement and opinion than rigorous findings.
When North Carolina reopened its schools to in-person learning in March, the state asked the ABC Science Collaborative, led by Duke University researchers, to study transmission of Covid in classrooms. On June 30 the collaborative shared findings in a report and news conference touting the effectiveness of masks. The impression was that the data from schools led them to this conclusion, though the way North Carolina chose to reopen did not allow for a control group. The report nonetheless declared wearing masks “an effective strategy to prevent in-school COVID-19 transmission.”
The collaborative also determined that social distancing, the only intervention that did have something close to a control group, had no measurable impact on transmission as it changed from minimal to very onerous distancing requirements. That was true not only in classrooms but on the way to school: They concluded that having one, two or three youngsters in a bus seat made no difference in transmission. This is a major reversal given public health officials’ focus on social distancing over the past year. The report also recommended loosening the policy of quarantining students who come into contact with an infected person, another pillar of the state’s strategy.
Despite all this, the report describes the state’s Covid strategy as “outstanding.” All schools showed low degrees of transmission, for which the report credited the only unstudied pillar, the masking policy. Because it applied everywhere in the state, there was no control group. But they could have compared transmission rates with school districts in other states and in Europe that didn’t mandate masks.