Scientists can draw very different meanings from the same data, study shows

Go down

Scientists can draw very different meanings from the same data, study shows Empty Scientists can draw very different meanings from the same data, study shows

Post by eddie on Sun Jan 31, 2016 4:26 pm

Scientists can draw very different meanings from the same data, study shows


Giving the same information to multiple scientific teams can lead to very different conclusions, a report published today in Nature shows.

And that’s exactly why two researchers think scientists should share their data with others — well before they publish.

In this experiment, 29 scientific teams were given the same information about soccer games. They were asked to answer the question “Are dark-skinned players more likely to be given red cards than light-skinned ones?” Some scientists found that there was no significant difference between light-skinned and dark-skinned players, whereas others found a very strong trend toward giving more red cards to dark-skinned players. So, even though a pooled result showed that dark-skinned players were 30 percent more likely than light-skinned players to receive red cards, the final conclusion drawn from this exercise — that a bias exists — was a lot more nuanced than it likely would have been if only one team had conducted the analysis.

http://www.thinkgasms.com/2015/10/09/scientists-can-draw-very-different-meanings-from-the-same-data-study-shows/


Science: a "truth" that is ever-changing and never seen the same by everyone.
Some truth! Rolling Eyes


_________________
Please stop saying "You can't joke about anything anymore". You can. You can joke about whatever the fuck you like. And some people won't like it and they will tell you they don't like it. And then it's up to you whether you give a fuck or not. And so on. It's a good system. ~ Ricky Gervais
eddie
eddie
King of Beards. Keeper of the Whip. Top Chef. BEES!!!!!! Mushroom muncher. Spider aficionado!

Posts : 39308
Join date : 2013-07-28
Age : 49
Location : England

View user profile

Back to top Go down

Scientists can draw very different meanings from the same data, study shows Empty Re: Scientists can draw very different meanings from the same data, study shows

Post by Raggamuffin on Sun Jan 31, 2016 5:06 pm

That's fairly meaningless without the actual data and the averages or medians. If some teams said that there was no significant difference, there needed to be teams which said there was a large difference. If one team had been asked, they may well have said there was an even larger difference than the pooled result showed.

One would also need to define what "significant" means in this context, and also one would need to know the definition of "dark" versus "light" re skin colour, and whether all the teams applied the same definition or not because it's open to interpretation otherwise.

In medical trials they do share data, which results in meta-analyses - pooled results. However, this is not really about pooled data, it's about how a number of people interpret the same data.

_________________

"There never was an Aaron, counselor".
Raggamuffin
Raggamuffin

Posts : 33746
Join date : 2014-02-10

View user profile

Back to top Go down

Scientists can draw very different meanings from the same data, study shows Empty Re: Scientists can draw very different meanings from the same data, study shows

Post by Original Quill on Sun Jan 31, 2016 5:14 pm

There's nothing magical about the term 'science'. You just have to pay attention to what is being said. People often say 'statistics are another way to lie', but it's all in the framing of the statement.

Let's say in a tournament, overall 10 cards are pulled on blacks, and 8 cards are pulled on non-blacks. I can say:

Blacks are carded more;

or

Blacks need to be carded more.

The evidence will show the same thing...more cards are pulled on black players. Which is the truth? That there is discrimination in officiating, or that blacks are unruly and need more watching?

We have this going on with the Oakland Raiders in the past 15-years. The Raiders consistently get more flags than any other team in any year. The Raiders also successfully sued the NFL twice in that span of time, and the lawyer for the League became the Commissioner for the League.

Which is the case? Are the Raiders breaking the rules more than other teams? Or, is the League retaliating against the Raiders for having twice successfully embarrassed the League in a court of law? Same facts make both cases.

_________________
“Little thieves are hanged, but great thieves are praised.” — Old Russian proverb, offered by Vladimir Putin to Donald J. Trump, Helsinki, July, 2018.

"I don't stand by anything."  ― Donald Trump, interview with John Dickerson, 5.1.17...

Normal is broken.

“That's libertarians for you — anarchists who want police protection from their slaves.” ― Kim Stanley Robinson, Green Mars
Original Quill
Original Quill

Posts : 29815
Join date : 2013-12-19
Age : 54
Location : Northern California

View user profile

Back to top Go down

Scientists can draw very different meanings from the same data, study shows Empty Re: Scientists can draw very different meanings from the same data, study shows

Post by Raggamuffin on Sun Jan 31, 2016 5:19 pm

Original Quill wrote:There's nothing magical about the term 'science'.  You just have to pay attention to what is being said.  People often say 'statistics are another way to lie', but it's all in the framing of the statement.  

Let's say in a tournament, overall 10 cards are pulled on blacks, and 8 cards are pulled on non-blacks.  I can say:

Blacks are carded more;

or

Blacks need to be carded more.

The evidence will show the same thing...more cards are pulled on black players.  Which is the truth?  That there is discrimination in officiating, or that blacks are unruly and need more watching?

We have this going on with the Oakland Raiders in the past 15-years.  The Raiders consistently get more flags than any other team in any year.  The Raiders also successfully sued the NFL twice in that span of time, and the lawyer for the League became the Commissioner for the League.  

Which is the case?  Are the Raiders breaking the rules more than other teams?  Or, is the League retaliating against the Raiders for having twice successfully embarrassed the League in a court of law?  Same facts make both cases.

It would also depend on how many black players they have as opposed to non-black players. I assume that the data in the experiment was adjusted for that.

Then you'd have to know if there was an unjustified bias, or if the black players simply broke the rules more often. The data wouldn't necessarily lead to a conclusion of "bias".

_________________

"There never was an Aaron, counselor".
Raggamuffin
Raggamuffin

Posts : 33746
Join date : 2014-02-10

View user profile

Back to top Go down

Scientists can draw very different meanings from the same data, study shows Empty Re: Scientists can draw very different meanings from the same data, study shows

Post by Original Quill on Sun Jan 31, 2016 5:31 pm

Raggamuffin wrote:
Original Quill wrote:There's nothing magical about the term 'science'.  You just have to pay attention to what is being said.  People often say 'statistics are another way to lie', but it's all in the framing of the statement.  

Let's say in a tournament, overall 10 cards are pulled on blacks, and 8 cards are pulled on non-blacks.  I can say:

Blacks are carded more;

or

Blacks need to be carded more.

The evidence will show the same thing...more cards are pulled on black players.  Which is the truth?  That there is discrimination in officiating, or that blacks are unruly and need more watching?

We have this going on with the Oakland Raiders in the past 15-years.  The Raiders consistently get more flags than any other team in any year.  The Raiders also successfully sued the NFL twice in that span of time, and the lawyer for the League became the Commissioner for the League.  

Which is the case?  Are the Raiders breaking the rules more than other teams?  Or, is the League retaliating against the Raiders for having twice successfully embarrassed the League in a court of law?  Same facts make both cases.

It would also depend on how many black players they have as opposed to non-black players. I assume that the data in the experiment was adjusted for that.

Then you'd have to know if there was an unjustified bias, or if the black players simply broke the rules more often. The data wouldn't necessarily lead to a conclusion of "bias".

Correct. The way they show bias in American law is by use of a bi-nomial analysis. The Supreme Court arbitrarily says if the two populations vary by more than 3 SD's, there is an extraneous reason for it.

_________________
“Little thieves are hanged, but great thieves are praised.” — Old Russian proverb, offered by Vladimir Putin to Donald J. Trump, Helsinki, July, 2018.

"I don't stand by anything."  ― Donald Trump, interview with John Dickerson, 5.1.17...

Normal is broken.

“That's libertarians for you — anarchists who want police protection from their slaves.” ― Kim Stanley Robinson, Green Mars
Original Quill
Original Quill

Posts : 29815
Join date : 2013-12-19
Age : 54
Location : Northern California

View user profile

Back to top Go down

Scientists can draw very different meanings from the same data, study shows Empty Re: Scientists can draw very different meanings from the same data, study shows

Post by Raggamuffin on Sun Jan 31, 2016 5:37 pm

Of course the experiment didn't specify black people versus non-black people, it merely says "dark-skinned" and "light-skinned". As I said, that would need to be defined because people have different perceptions of what is dark skinned or not.

It would also depend on how they looked at the data. If they put several photos of players side by side, they might define "dark-skinned" differently to someone who did not do that. For example, if I stood next to someone with blonde hair, someone might describe my hair as "dark", but if I stood next to someone with black hair, my hair might be described as "light".

_________________

"There never was an Aaron, counselor".
Raggamuffin
Raggamuffin

Posts : 33746
Join date : 2014-02-10

View user profile

Back to top Go down

Scientists can draw very different meanings from the same data, study shows Empty Re: Scientists can draw very different meanings from the same data, study shows

Post by veya_victaous on Sun Jan 31, 2016 10:14 pm

umm that is the point of science, it is called peer review.

But in addition that is a Social Science, like all psychology it is mocked by the real sciences because it fails to meet the requirements to be a real science. where one must be able to define and measure.

As raggs points out none of the variables in the above are definitively defined.

_________________
My job is to travel the world delivering Chaos and Candy.

We don't know the Questions... does that means we cannot seek the Answers?
veya_victaous
veya_victaous
The Mod Loki, Minister of Chaos & Candy, Emperor of the Southern Realms, Captain Kangaroo

Posts : 19095
Join date : 2013-01-23
Age : 36
Location : Australia

View user profile

Back to top Go down

Back to top

- Similar topics

 
Permissions in this forum:
You cannot reply to topics in this forum