Benjamin Kirkup writes:
As you sometimes comment on such things; I’m forwarding you a journal editorial (in a society journal) that presents “lessons learned” from an associated research study.
What caught my attention was the comment on the “notorious” design, the lack of “significant” results, and the “interesting data on nonsignificant associations.” Apparently, the work “does not serve to inform the regulatory decision-making process with respect to antimicrobial compounds” but is “still valuable and can be informative.”
Given the commissioning of a lessons-learned, how do you think the scientific publishing community should handle manuscripts presenting work with problematic designs and naturally uninformative outcomes?
The editorial in question is called Lessons Learned from Probing for Impacts of Triclosan and Triclocarban on Human Microbiomes, it is by Rolf Halden, and it appeared in a journal of the American Society for Microbiology.
I do find the whole story puzzling, that Halden describes the study as small and underpowered, while also “presenting a treasure trove of information.” The editorial almost like a political effort, designed to make everyone happy. That said, I don’t know jack about the effects of triclosan and triclocarban on human biology, so maybe this all makes sense in context.
The “underpowered treasure trove” thing reminds me a bit of when food researcher and business school professor Brian Wansink told the story of a “failed study which had null results” (in his words) which at the same time was “a cool (rich & unique) data set” that resulted in four completely independent published papers. Failed yet wonderful.