Eight Tips to Publish a Negative Study

It is often said that most scientific hypotheses are false. However, publishing an article when the results are negative can be a challenge for even the best researchers. The simple existence of a non-significant p-value, or a wide confidence interval is often the death of a paper. And, there is a well-known publication bias against negative papers. Although this is a frustration for all of us as researchers, negative papers are often publishable if the researcher is persistent.

`

Professor Brian Rowe has been my unofficial mentor since I was a resident in Emergency Medicine. A professor of Emergency Medicine at the University of Alberta with 548 peer-reviewed articles and 38 text book chapters, he certainly has experience in the publication and peer-review process. I was fortunate to get from him his top eight tips on how to publish a negative article.

1. Target the Appropriate Journal

Always target the appropriate journal. It's of critical importance that your paper is targeted to the right audience. "Don't waste your time on the wrong audience," he says, stating that researchers often do not spend enough time considering who the paper's audience really is.

What's a good way to know if you are picking the right journal? Professor Rowe suggests looking at your own references in the paper: "if all your references are from palliative care journals, then you should probably target a palliative care audience."

Once you have selected a target journal, explain in your cover letter why you have selected that journal. "It helps to compliment the journal in your cover letter," Professor Rowe states. Also, making sure the cover letter is on letterhead, is signed, and contains no spelling errors can go a long way in showing the editors that you are serious about publishing your paper in their prestigious journal.

2. Make a Smart Title

Professor Rowe suggests using a pithy, smart, or even provocative title. "It's the first thing that hits them," he notes, explaining that many authors do not spend enough time thinking about the title. My favourite titles from Professor Rowe are "Characteristics of Patients Who Leave Emergency Departments without Being Seen", "Performance of an Asthma Quality of Life Questionnaire in an Outpatient Setting", and "Impact of a Triage Liaison Physician on Emergency Department Overcrowding and Throughput: a Randomized Controlled Trial." Does he like to use catchy titles? He does, including titles such as "...is the Juice Worth the Squeeze" and "...Stop Blaming the Patient" although he admits the editors may ask you to revise or even omit them.

Making sure your title contains appropriate keywords is also advisable. For instance, if the paper is a systematic review, or randomized study then those words should be contained in the title.

3. Avoid Concluding More Research is Needed in the Abstract

Don't conclude the obvious - more research is necessary - in the abstract. Professor Rowe states: "clearly these are self-serving and obvious. Few studies conclude 'stop all further research, my study has provided the cure.'" The abstract should emphasize what you did find, not what needs to be found in the future. Also, "don't stretch," Professor Rowe advises. He suggests the conclusion should answer the research question but should not speculate and should not contain any new ideas.

4. Focus on the Methods

When the study result is not what you expected, focus carefully on the methods. If a study has excellent methodology, was properly implemented, and had an appropriate sample size a lack of effect may be seen as supportive evidence that the true effect is either small or non-existent. On the other hand, if the study is poorly documented readers may assume that the lack of effect found is just due to poor study design.

Methods should be succinct, and contain adequate detail. In addition, always make sure that no results creep into the methods section. As your methods may pave the way for further studies to replicate your findings, the methods should contain enough detail so that future researchers can duplicate your study.

Professor Rowe notes that one of the most frequently forgotten parts of the methods is the setting. He feels that this information is critical for readers to understand the generalizability of the findings. For instance, specifying such details are adult or mixed emergency department, annual emergency department visits, staffing, and use of other professionals can all be useful for the reader to understand the setting.

5. Follow the Instructions for Authors

This is a recurring theme, that was mentioned by Sam Stratton in a previous post on how to publish COVID-19 related research. I must admit that in my early years as a researcher I often ignored this advice, assuming that reviewers and editors would still read my paper and accept it with revisions. Not so unfortunately. Following the instructions for authors and writing a clear succinct paper are both ways that authors show respect for the journal and for the reviewers time. Not following the formatting is often a cause for rapid rejection.

Common areas where authors fail to follow the instructions for authors include exceeding the word count and not following the journal's instruction for headings.

6. Provide a Thoughtful Discussion

Think carefully about what your article contributes to the existing literature and detail that thoroughly in the discussion. Don't forget in the discussion to address the overall goals of the study explicitly.

Professor Rowe prefers to use the following five steps in the discussion in this order:

1. Explain what was found in the study
2. Detail the implications that the findings have
3. Compare the findings to other findings in the literature
4. Discuss limitations of the study
5. Sum it all up

Professor Rowe echoed that the discussion should never contain any new findings from the study. All findings should be presented in the results first. In addition, the discussion should be focused on the findings of the study and never reach to speculation.

7. Always Provide a Limitations Section

Ensure that the limitations sections is explicit in the article. Reviewers want to know that you understand your topic well enough to describe the limitations. If you don't describe them, it signals to the reviewers that you do not really understand the implications of your research. And don't try to hide the limitations from the reviewers: if you know they are there the reviewers will too.

Professor Rowe suggests thinking of the limitations section as a time to "anticipate" what the reviewers will think about the paper. Thinking carefully about your study design can also help to anticipate these limitations. For instance, systematic reviews are always subject to publication bias. Authors of systematic reviews should mention potential publication bias as a limitation and then detail the steps that they have taken to mitigate that limitation.

Furthermore, limit your limitation to true limitations. "Don't talk about small sample size if your results are significant," Professor Rowe advises. He also suggests to not pad the limitation section with trivial limitations such as concluding that the survey response rate was a limitation if 95% of participants answered the survey.

8. Respond Respectfully to Reviewers

The peer-review process can be long and painful. However, you should always respond to each concern from the reviewers in a timely manner and with respect. When responding to concerns from reviewers, your goal as an author should be to make the job of the reviewer as easy as possible.

"I respond to every single point," Professor Rowe explains. He also feels that it is important to be complimentary to the reviewer by validating his or her comments. "Remember, don't take it personally," he suggests.

Professor Rowe also feels that articles that come back to the author with suggestions for revisions are like "a fish on a hook." It pays to be persistent.

Every Article has a Home

One of the best pieces of advice I ever received from Professor Rowe was: "every article has a home." He advises to be persistent and flexible and to learn from your mentors. By following the eight steps above, you too can find a home for your article, even with a p-value of .051.

To get further information and hints on how to increase your publication success for negative studies please download our infographic.