Georgia Firearm Forums - Georgia Packing banner
1 - 20 of 26 Posts

·
GeePeeDoHolic
Joined
·
6,413 Posts
Discussion Starter · #1 ·
I'd always thought an incorrect scientific paper was an anomaly and that, except for maybe certain disciplines being more prone to error, science was generally reliable.

This article makes a convincing case that scientific research is systemically flawed. Researchers may be going through the motions, but the fundamentals aren't getting the attention they need.

http://www.economist.com/news/brief...elf-correcting-alarming-degree-it-not-trouble

Various factors contribute to the problem. Statistical mistakes are widespread. The peer reviewers who evaluate papers before journals commit to publishing them are much worse at spotting mistakes than they or others appreciate. Professional pressure, competition and ambition push scientists to publish more quickly than would be wise. A career structure which lays great stress on publishing copious papers exacerbates all these problems. “There is no cost to getting things wrong,†says Brian Nosek, a psychologist at the University of Virginia who has taken an interest in his discipline’s persistent errors. “The cost is not getting them published.â€
 

·
Deplorable bitter clinger.
Joined
·
5,695 Posts
Interesting article. Thanks for posting this.
 

·
GeePeeDoHolic
Joined
·
6,413 Posts
Discussion Starter · #3 ·
Sultan Knish publishes tl;dr posts sometimes, but usually worth the read.

"The End of Science"
Science has been thoroughly Saganized. The vast majority of research papers are wrong, their results cannot be replicated. The researchers writing them often don't even understand what they're doing wrong and don't care. Research is increasingly indistinguishable from politics. Studies are framed in ways that prove a political premise, whether it's that the world will end without a carbon tax or that racism causes obesity. If they prove the premise, the research is useful to the progressive non-profits and politicians who always claim to have science in their corner. If it doesn't, then it isn't funded.
http://sultanknish.blogspot.com/2014/03/the-end-of-science.html
 

·
Registered
Joined
·
9,458 Posts
Expert witnesses are bought and paid for.
 

·
GeePeeDoHolic
Joined
·
6,413 Posts
Discussion Starter · #6 ·
More on state of science

"omething has gone fundamentally wrong with one of our greatest human creations," Dr. Richard Horton, editor-in-chief of prestigious medical journal Lancet, lamented this year. "Much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness."

http://www.washingtonexaminer.com/junk-science-garbage-policy/article/2567516
 

·
Lawyer and Gun Activist
Joined
·
28,519 Posts

·
GeePeeDoHolic
Joined
·
6,413 Posts
Discussion Starter · #8 ·
http://gizmodo.com/over-120-science-journal-papers-pulled-for-being-total-1534110496

120 science journal papers (Springer and IEEE) are being pulled because they were randomly generated by a computer program. The, yes, peer-reviewed, journal can't explain how they got published.

http://www.nature.com/news/publishe...al&utm_source=twitter.com&utm_campaign=buffer

Labbé says that the latest discovery is merely one symptom of a “spamming war started at the heart of science†in which researchers feel pressured to rush out papers to publish as much as possible.
 

·
Moderator
Joined
·
69,784 Posts
Rugerer, total gibberish! LOL! That's hilarious. Apparently the peers doing the reviewing did not want to admit that they could not understand what they were reading. LOL!
 

·
Registered
Joined
·
2,051 Posts
Rugerer, total gibberish! LOL! That's hilarious. Apparently the peers doing the reviewing did not want to admit that they could not understand what they were reading. LOL!
That's actually a big problem with the "peer reviewed" system. The system makes people think it means that "experts have verified this information to be correct" but that isn't what it means at all. All it really means is that people who have studied similar things looked over the paper for obvious errors with the things stated, is the stated methodology acceptable? (they can't verify you followed it), does the data look to have obvious problems? (they aren't going to source new data to compare it to), does the work track with similar stuff previously done (do your citations make sense) or explain why not? Did you include relevant similar study information (citations included and reference other articles "rigorously" reviewed)?

They aren't "peer reviewing" an article to be correct, right, or anything of the sort as for the most part that would mean recreating the entirety of the work. They're perusing for obvious problems. Like an assignment partner glancing at your group paper to make sure you've removed the links to wikipedia references and aren't keeping words that the professor would have to look up, despite how impressive they looked in the paper you were copying from.....

That the general public thinks "peer reviewed" means that the study/findings/etc have been retested, verified correct, etc is a problem with their understanding, not the system. It's just not feasible to have every study duplicated or verified to that extent, thus replication is the least common form of testing and is generally reserved for really important or critical things.
 

·
Premium Member
Joined
·
8,460 Posts
That's actually a big problem with the "peer reviewed" system. The system makes people think it means that "experts have verified this information to be correct" but that isn't what it means at all. All it really means is that people who have studied similar things looked over the paper for obvious errors with the things stated, is the stated methodology acceptable? (they can't verify you followed it), does the data look to have obvious problems? (they aren't going to source new data to compare it to), does the work track with similar stuff previously done (do your citations make sense) or explain why not? Did you include relevant similar study information (citations included and reference other articles "rigorously" reviewed)?

They aren't "peer reviewing" an article to be correct, right, or anything of the sort as for the most part that would mean recreating the entirety of the work. They're perusing for obvious problems. Like an assignment partner glancing at your group paper to make sure you've removed the links to wikipedia references and aren't keeping words that the professor would have to look up, despite how impressive they looked in the paper you were copying from.....

That the general public thinks "peer reviewed" means that the study/findings/etc have been retested, verified correct, etc is a problem with their understanding, not the system. It's just not feasible to have every study duplicated or verified to that extent, thus replication is the least common form of testing and is generally reserved for really important or critical things.
If it is even possible to replicate.

Some research is nothing more than taking data other research teams have compiled and then running a program that the scientist wrote on the data, and come up with a result. The journals typically do not mandate the inclusion of the original data nor the computer code that was used. While that might be OK if the research paper gives all the details of the data used and the statistical methods the programs used, that usually isn't the case. The raw data has to be requested from the original researcher, who doesn't have to provide it if they don't want to as this wasn't their paper, and the articles are usually vague on what statistical methods were actually used.

So an unpaid peer reviewer gets a paper, with no underlying data, no code to review, just the paper based on it. Was the code faulty? Is the description of the statistical methods used in the paper the actual ones used in the code? Who knows, the peer reviewer sure doesn't.

As for it being a problem with the system, I have to say it is an equal part the system and the people. You could educate the people, however in school I was taught that a paper should be written so that someone reading it could follow what you did and get a similar result. This is not the case with modern peer review. This is something Journals should fix but they do not.

That doesn't even go into pal review... peer review by your own friends.

I could keep going with all the problems. You would think that it is just a low bar for keeping crappy articles out and that all good ones can get published, but that isn't so. If you have a competing theory from the reviewer's theory, then even if your research is perfect, it can be denied being published based on biased peer review.
 

·
Registered
Joined
·
1,254 Posts
If you have a competing theory from the reviewer's theory, then even if your research is perfect, it can be denied being published based on biased peer review.
This! x100000000000000000000000
 

·
Registered
Joined
·
2,051 Posts
If it is even possible to replicate.

Some research is nothing more than taking data other research teams have compiled and then running a program that the scientist wrote on the data, and come up with a result. The journals typically do not mandate the inclusion of the original data nor the computer code that was used. While that might be OK if the research paper gives all the details of the data used and the statistical methods the programs used, that usually isn't the case. The raw data has to be requested from the original researcher, who doesn't have to provide it if they don't want to as this wasn't their paper, and the articles are usually vague on what statistical methods were actually used.

So an unpaid peer reviewer gets a paper, with no underlying data, no code to review, just the paper based on it. Was the code faulty? Is the description of the statistical methods used in the paper the actual ones used in the code? Who knows, the peer reviewer sure doesn't.

As for it being a problem with the system, I have to say it is an equal part the system and the people. You could educate the people, however in school I was taught that a paper should be written so that someone reading it could follow what you did and get a similar result. This is not the case with modern peer review. This is something Journals should fix but they do not.

That doesn't even go into pal review... peer review by your own friends.

I could keep going with all the problems. You would think that it is just a low bar for keeping crappy articles out and that all good ones can get published, but that isn't so. If you have a competing theory from the reviewer's theory, then even if your research is perfect, it can be denied being published based on biased peer review.
Also, most research isn't published. A common (almost universal) qualifier to make it into the paper is if it's something the editor and peers think people who read the paper would want to read about.. so that groundbreaking (but boring) research may never make a paper but that clarification to an addendum to a popular theory might.
 

·
GeePeeDoHolic
Joined
·
6,413 Posts
Discussion Starter · #14 ·
Having brought mankind so far, has traditional science finally outlived its usefulness? Many seem to think so, finding the rules of the scientific method-the strict guidelines a researcher must follow to actually practice science-far too restrictive and cumbersome.
It seems that the scientific method is being wished away.

http://theresilientearth.com/?q=content/science-death-spiral
 

·
Junior Butt Warmer
Joined
·
46,427 Posts
Gunstar1 said:
So an unpaid peer reviewer gets a paper, with no underlying data, no code to review, just the paper based on it. Was the code faulty? Is the description of the statistical methods used in the paper the actual ones used in the code? Who knows, the peer reviewer sure doesn't.
It must be right, 'cause he used a computer.

[/sarcasm]

:lol:
 

·
Junior Butt Warmer
Joined
·
46,427 Posts
Rugerer said:
Having brought mankind so far, has traditional science finally outlived its usefulness? Many seem to think so, finding the rules of the scientific methodâ€"the strict guidelines a researcher must follow to actually practice scienceâ€"far too restrictive and cumbersome.
Sounds like a millennial.
 

·
GeePeeDoHolic
Joined
·
6,413 Posts
Discussion Starter · #18 · (Edited)

·
Registered
Joined
·
1,992 Posts
I read all manner of scientific studies in the NYT. A lot of the time, they just don't meet the common sense mark, but if they back some NYT point they get an article. A lot of research out there is what I'd describe as analyzing a thunderstorm based on one rain drop worth of data.
 

·
American
Joined
·
3,289 Posts
1 - 20 of 26 Posts
Top