Today’s red might be tomorrow’s blue ! All evaluations are flawed !

I had a similar expression when this thought came to me and I started pondering about it. Recently I became a part of a group called ” ASCEND-Science & Non Duality ” where like minded people discuss and debate on drawing parallelism between Science and Non Duality i.e., the teachings of the Advaita philosophy. The discussion on the group was usually on higher concepts of quantum physics and me being from an computer science and engineering background the discussions were was quite hard to comprehend or to get involved in them.

But few days ago I came across a paper “Producing Wrong Data Without Doing Anything Obviously Wrong!” by Mytkowicz et al. The paper deals about measurement bias and how this can lead to incorrect conclusions — particularly on “Does O3 optimisation level really offer any improvement over O2? “ This seems a rather simple problem right ? We just need to compile some programs at both optimisation levels and measure how long they take to run. But that obvious a sweeping assumption: that your handful measurements on those programs are representative of the general concept of compiling with different optimisation levels.

Of course it’s not, you might say — you can’t possibly compile all the programs in the world, for example, so your choice of programs might certainly influence the results you see. For example, consider a researcher who wants to determine if optimisation O is beneficial for system S. If she measures S and S + O in an experimental setup that favours S + O, she may overstate the effect of O or even conclude that O is beneficial even when it is not. This phenomenon is called measurement bias in the natural and social sciences.The best you can do is pick some standard, diverse, representative benchmarks and make the case that they’re reasonably representative. Similarly, while a truly robust finding would require measuring all the computers in the world that someone might ever run optimised binaries on, that’s clearly infeasible — so you can do your job as a scrupulous scientist by measuring a few popular machines and hoping for the best.

Even if we try all combinations of machines and programs there is still a probability of measurement bias because there might be some corner cases yet to discovered by the scientists !!! Now to flex my physics knowledge : consider the example of Newtonian mechanics. Earlier, the Newton mechanics was sufficient was to describe the motions of objects and the equations were tested on all possible scenarios which gave out the accurate answers. Hence, was accepted by the scientific community. But this was a case of measurement bias. These laws were applied only on macro objects like a table, a car or a monkey which favours the Newtonian mechanics and not on microscopic particles. Thus leading to an incorrect conclusion.

So the question is : How do you know when it’s time to stop and say your conclusions and inferences is good enough ? How many observations and experiments are required ?

I feel that the answer to this question is that we never know !! We never know exactly how many properties or programs has to verified before concluding. Who knows, today’s red might be tomorrow’s blue after considering some properties. Hence, we could say all evaluations are flawed since they all have measurement bias in them and all our lives are based on false world.

So the next question would be : Then what is the truth ? Are we on the right path of finding the truth ?

Honestly i don’t know. But here is something that I found interesting and made me question a lot. Consider a clay pot, a clay statue and a clay brick. For a person living in the 21st century these are three different things : a pot, a statue, a brick but for a man thousands of years ago everything is just clay for him. Now, we can extend this concept and consider everything is “Brahman” and whatever we see around us is just Brahman with certain man-made properties.

So here are my final thoughts : It was science and technology that added a layer of abstraction over clay to make it a pot or a brick. Similarly, is science responsible for adding all the layers of abstraction called “Maya” on top of Brahman ? So, is science or so called Vidya in today’s world actually Avidya ? Isn’t making further scientific discoveries actually increasing the abstractness ? Is science and the truth travelling in opposite directions ?

I know that instead of concluding and providing answers I have introduced more questions and confusion. Hopefully I can answer then in the upcoming blogs. Feel free to comment and suggest improvements in the blog.