online study
not peer reviewed
“published” on arxiv (which is a public document server, not a journal)
study and authors not named or linked in the article
tl/dr: “Someone uploaded a pdf and we’re writing about it.”
You are overrating peer reviewing. It’s basically a tool to help editors to understand if a paper “sells”, to improve readability and to discard clear garbage.
If methodologies are not extremely flawed, peer reviewing almost never impact quality of the results, as reviewers do not redo the work. From the “trustworthy” point of view, peer reviewing is comparable to a biased rng. Google for actual reproducibility of published experiments and peer-reviewing biases for more details
Most peer reviewed papers are non reproducible. Peer review has the primary purpose of telling the editor how sellable is a paper in a small community he only superficially knows, and to make it more attractive to that community by suggesting rephrasing of paragraphs, additional references, additional supporting experiment to clarify unclear points.
But it doesn’t guarantees methodology is not flawed. Editor chooses reviewer very superficially, and reviews are mainly driven by biases, and reviewers cannot judge the quality of a research because they do not reproduce it.
Honesty of researchers is what guarantees quality of a paper
There is some variation across disciplines; I do think that in general the process does catch a lot of frank rubbish (and discourages submission of obvious rubbish), but from time to time I do come across inherently flawed work in so-called “high impact factor” and allegedly “prestigious” journals.
In the end, even after peer review, you need to have a good understanding of the field and to have developed and applied your critical appraisal skills.
Absolutely. One needs to know what is reading. That’s why pre prints are fine.
High impact factor journals are full of works purposely wrong, made because author wants the results that readers are looking for (that is the easiest way to be published in high impact factor journal).
tl/dr: “Someone uploaded a pdf and we’re writing about it.”
You are overrating peer reviewing. It’s basically a tool to help editors to understand if a paper “sells”, to improve readability and to discard clear garbage.
If methodologies are not extremely flawed, peer reviewing almost never impact quality of the results, as reviewers do not redo the work. From the “trustworthy” point of view, peer reviewing is comparable to a biased rng. Google for actual reproducibility of published experiments and peer-reviewing biases for more details
Preprints are fine, just less polished
Peer reviewing is how you know the methodology is not flawed…
Unfortunately not. https://www.nature.com/articles/533452a
Most peer reviewed papers are non reproducible. Peer review has the primary purpose of telling the editor how sellable is a paper in a small community he only superficially knows, and to make it more attractive to that community by suggesting rephrasing of paragraphs, additional references, additional supporting experiment to clarify unclear points.
But it doesn’t guarantees methodology is not flawed. Editor chooses reviewer very superficially, and reviews are mainly driven by biases, and reviewers cannot judge the quality of a research because they do not reproduce it.
Honesty of researchers is what guarantees quality of a paper
Yes. A senior colleague sometimes tongue-in-cheek referred to it as Pee Review.
The downvotes to my comments shows that no many people here has ever done research or knows the editorial system of scientific journals :D
There is some variation across disciplines; I do think that in general the process does catch a lot of frank rubbish (and discourages submission of obvious rubbish), but from time to time I do come across inherently flawed work in so-called “high impact factor” and allegedly “prestigious” journals.
In the end, even after peer review, you need to have a good understanding of the field and to have developed and applied your critical appraisal skills.
Absolutely. One needs to know what is reading. That’s why pre prints are fine.
High impact factor journals are full of works purposely wrong, made because author wants the results that readers are looking for (that is the easiest way to be published in high impact factor journal).
https://www.timeshighereducation.com/news/papers-high-impact-journals-have-more-statistical-errors
It’s the game. Reader must know how to navigate the game. Both for peer reviewed papers and pre prints