Why too much evidence can be a bad thing

4) In science, theory and experiment go hand in hand and must support each other. In every experiment there is always ‘noise,’ and we must therefore expect some error. In the history of science there are a number of famous experiments where the results were ‘too good to be true.’ There are many examples that have been mired in controversy over the years, and the most famous are Millikan’s oil drop experiment for determining the charge on the electron and Mendel’s plant breeding experiments. If results are too clean and do not contain expected noise and outliers, then we can be led to suspect a form of confirmation bias introduced by an experimenter who cherry-picks the data.5) In many committee meetings, in today’s big organizations, there is a trend towards the idea that decisions must be unanimous. For example, a committee that ranks job applicants or evaluates key performance indicators (KPIs) often will argue until everyone in the room is in agreement. If one or two members are in disagreement, there is a tendency for the rest of the committee to win them over before moving on. A take-home message of our analysis is that the dissenting voice should be welcomed. A wise committee should accept that difference of opinion and simply record there was a disagreement. The recording of the disagreement is not a negative, but a positive that demonstrates that a systemic bias is less likely.6) Eugene Wigner once coined the phrase ‘the unreasonable effectiveness of mathematics’ to describe the rather odd feeling that math seems to be so perfectly suited to describing physical theories. In a way, Wigner was expressing the idea that math itself is ‘too good to be true.’ (See this article for more on this idea.) The reality is that modern devices and machines are no longer analyzed by neat analytical mathematical equations, but by empirical formulas embedded in simulation software tools. For some of the next big science questions, particularly in the area of complex systems, we are looking to big data and machine learning rather than math. Analytical math as we knew it was not the perfect glove that could fit every type of problem. So how did we get seduced to once thinking that math was ‘unreasonably effective’? It’s the systemic confirmation bias introduced by the fact that for every great scientific paper we read with an elegant formula, there are many more rejected formulas that are never published and we never get to see. The math we have today was cherry-picked.Lachlan J. Gunn (University of Adelaide) gives a seminar on the “paradox of unanimity” at École d’ingénieurs de l’université d’Angers at 10aam on the 12th January 2016.More information: Lachlan J. Gunn, et al. “Too good to be true: when overwhelming evidence fails to convince.” Proceedings of The Royal Society A. To be published. Arxiv pre-print: arxiv.org/abs/1601.00900© 2016 Phys.orgExplore furtherConfidence counts: Accuracy of eyewitness IDs increases with degree of certaintyDec 21, 2015When it comes to accurately identifying a criminal suspect, it makes a difference how sure an eyewitness is, finds a study led by a memory expert at the University of California, San Diego. The American justice system should …Additional suspect line-ups may help catch more bad guysMar 07, 2014A second viewing in a police line-up may help more eyewitnesses identify the culprit, new research from Flinders University reveals.Identifying the bad guyApr 17, 2012Flinders University psychologist Professor Neil Brewer is proposing a radical alternative to the traditional police line-up, arguing current eyewitness identification tests often fa

Source: Why too much evidence can be a bad thing

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s