This is not about the huge scandals surrounding scientific research. This is about how more than 50 thousands bad “scientific” articles are introduced into the international debate every year without passing the minimum standards. Let me show you how this can happen.

Beall’s List

Jeffray Beall is a famous librarian and professor at the University of Colorado. He is also the creator of the Beall’s list, a collection of  “predatory publishers”. What are predatory publishers? Potential predatory scholarly open‑access publishers are companies hell-bent on passing off as relevant knowledge what is nothing more than shoddy science. All for the profit. Calling it science might even be too much.

Review ‘peer review’

Predatory magazines actively research potential authors that want (or need) to publish their work without passing peer-review. Peer review is a process meant to guarantee highest scientific standards by scrutinising the scientific methods and conclusion of the author. Highly qualified scientists of the same field as the author that wants to be published read the paper, analyse it and write a report. They then can either recommend the article for publication or refuse it, if it doesn’t keep up to the scientific standard of form and method.

Authors are motivated to pay to have their work published for the sake of career progression or research evaluation […]. In return, predatory publishers turn a blind eye to any limitations of papers during peer-review in favour of generating income from authors’ fees; the worst of them fake the peer-review process and print almost anything for money, without scruples.

In their analysis ^[1]Predatory publishing in Scopus: evidence on cross-country differences Link from February 2021 the Czech scientists Vít Macháček and Martin Srholec found that out of 3,293 journals listed in Ulrichsweb ^[2] a directory and database providing information about popular and academic magazines, scientific journals, newspapers and other serial publications, published since 1932 324 also appeared on the Beall’s list, indicating that they were fraudulent.

Between 2015 and 2017 these deceitful scientific magazines were able to introduce over 160 thousand “scientific” articles in the research database Scopus ^[3]by Elsevier link amounting to a total of nearly 3 percent of the articles published in this period. The countries most effected by this practice were developing countries in Asia and North Africa, and oil-rich nations like Saudi Arabia.

How to identify bad scientific papers

Not everything that is published automatically is good, sound science. Now that we know that we have to be aware of bad scientific publications, how can we winnow the good from the bad?

Peer review

Is the article peer-reviewed? Referees, the colleague scientists that recommend the publication or refusal of a scientific work, put their own reputation forth. Only peer-reviewed articles should even be considered by laymen that don’t have direct intimate knowledge of the subject of articles. A pre-print is an article that has not been published yet and therefor has not undergone peer-review. If you’re no expert in the field stay away from unpublished articles and pre-prints.

Cross-referencing

Every sound scientific paper will mention reputable sources for pretty much every claim. If you are not sure if a scientific paper is sound science, check related and cited articles. On research databases like pubmed or researchgate you can see how often an article has been read and cited by other scientists, which is a good indicator if that article is sound science.

Who is the publisher

Check the Beall’s list for the publishing magazine. If it appears there, chances are that the article is not up to the highest scientific standards. Predatory publishers let the authors pay the fee for getting published. They then offer the articles free of charge to the public. They often forgo any scrutiny as for the quality of the work submitted.

Follow the money

Serious publications expect their authors to make a declaration of conflicts of interests as well as the provenance of the funding. If an article was financed by a stakeholder, or if the author her/himself has any personal interest chances are that the result might be biased in favour of the money-giver.

Bad science published in reputable magazines

Of course it’s not only the underdogs of scientific publication that may be subject to bad science. Even some of the magazines with the highest reputation sometimes publish poorly conducted studies or articles with unsound conclusions. And journalists and bloggers sometimes take scientific studies and twist the outcome to their liking. If you read articles about scientific studies that make bold claims in absolute terms, like “scientists found cure for cancer”, be alarmed!

The only science that can speak in absolute terms might be mathematics. If you read about anything BUT mathematics, scientist will almost always used cautionary terms like ‘may be’, ‘it seems’, ‘it indicates’, and ‘more research is needed’.

Date of publication

Sometimes articles are cited over and over, even though the findings might be outdated or even proved false in later research. Try to find the latest research on the subject you are interested in. An article from the 1970’s might have been up to the highest standards of that time, but science moved on and many once up-to-date outcomes are no longer relevant or even true.

Correlation and Causation

Especially in nutrition and health outcomes the variables are impressively complex. It might be near to impossible to uniquely define the causation of an outcome. Keep in mind that

• Correlation, meaning that something happens at the same moment in time, does not automatically translate into causation, meaning that one thing has caused the other thing to happen
• Results of Animal studies translate into the same outcome in humans only 10 percent of the time
• Small cohorts are not predictive for health outcomes in bigger more diversified groups

So, if you are not a rat the findings of a study that used rat as study objects are probably not significant. And if a study was carried out on a cohort of 30 Indian women in their 50s the results can probably not be translated into general recommendations for a broader part of the population.

Authority

Last but not least it is useful to consider the authority of the author or group of scientists behind a paper. If the author has published many papers in that specific field of research and if he holds a position in an internationally recognised university, research facility or clinic chances are that his work is carried out by the highest standards. Young research fellows will start their career co-authoring papers with more experienced scientists in their fields before they start doing their own research. If an author has no co-authors, no scientific graduation, and is not affiliated to a scientific research facility chances are that the findings might not be up to the highest standards.

Take away

The scientific community has an inherent interest in keeping their stall clean. Unfortunately, the mechanisms of scientific publication in combination with the technical possibilities of the modern world make it difficult to sort through the vast amount of information that is published on an everyday basis. For laymen it is best to find a couple of trusted sources that apply Sanity and Reason, integrity and selflessness in order to read out the relevant results and steer clear from the predatory publications and claims.