“Anyone who has followed science in recent years has noticed something odd: science is less and less about a solitary scientist working alone in a lab. Scientists are working in networks, and those networks are gaining scope, speed, and power through the internet.”
Is there a better way to harness collective wisdom of the tribe? How might information and communication technologies (ICT) mediate interactions and collate diverse perspectives from tribe members?
In mathematical research, for example, there is an interesting case of how massively collaborative mathematics can rapidly improve upon the landmark result sparked by the insight of a solitary genius. Yitang Zhang, a lecturer at the University of New Hampshire, settled a long-standing open question about prime numbers on May 13, 2013 by demonstrating that even though primes get increasingly rare further out along the number line, one will never stop finding pairs of primes separated by at most 70 million.
Zhang’s finding was the first time anyone had managed to put a finite bound on the gaps between prime numbers. His result represents a major leap toward proving the centuries-old twin primes conjecture, which posits that there are infinitely many pairs of primes separated by only two (e.g., 11 and 13).
But why 70 million? There is nothing magical about the number, other than it served Zhang’s purpose and simplified his proof. While Zhang went for a nice round number, his method in fact gives 63,374,611 with just a little optimization. Soon, other mathematicians quickly realized that it should be possible to push this separation bound quite a bit lower, although not all the way down to two.
On May 28, simple tweaks to Zhang’s method brought the bound below 60 million. A flurry of activity ensued over a span of 5 days from May 30 to June 3, as mathematicians vied to improve on this number, setting one record after another: from 59,470,640 to 58,885,998 to 59,093,364 to 57,554,086 to 48,112,378 to 42,543,038 to 42,342,946 to 42,342,924 to 13,008,612 to 4,982,086 to 4,802,222. By June 4, Terence Tao, a Fields Medalist, set up a “Polymath project,” an open, online collaboration to improve the bound that attracted dozens of participants. For weeks, the project moved forward at a breathless pace. At times, the bound was going down every thirty minutes, according to Tao. By July 27, the Polymath team had succeeded in reducing the proven bound on prime gaps to a mere 4,680.
In a rather interesting turn of events, a post-doctoral researcher, James Maynard, working independently at the University of Montreal, has upped the ante. On November 19, just months after Zhang announced his result, Maynard presented an independent proof that pushes the gap down to 600. Maynard’s approach applies not just to pairs of primes, but to triples, quadruples, and larger collection of primes as well.
“Zhang’s work and, to a lesser degree, Maynard’s fits the archetype of the solitary mathematical genius, working for years in the proverbial garret until he is ready to dazzle the world with a great discovery. The Polymath project couldn’t be more different — fast and furious, massively collaborative, fueled by the instant gratification of setting a new world record.”
According to Tao, the solitary and collaborative approaches each have something to offer mathematics. “It’s important to have people who are willing to work in isolation and buck the conventional wisdom,” Tao said. Polymath, by contrast, is “entirely groupthink.” Not every math problem would lend itself to such collaboration, but this one did.
Fast and Furious: A whirlwind of mathematical activities bringing down the prime gap bound from 70 million to 600 in just half a year.
It turns out that Zhang’s constructive proof is very modular and involved three separate steps, each of which offered potential room for improvement on his 70 million bound. First, Zhang invoked some very deep mathematics to figure out where the prime numbers are likely to be hiding. Next, he used his result to figure out how many “teeth” his “comb” would need in order to guarantee that it would catch at least two prime numbers with its teeth infinitely often. Finally, he calculated how large a comb he had to start with so that enough teeth would be left after it has been snapped down to a condition of “admissibility” needed for catching prime numbers. The fact that these three steps could be separated made improving Zhang’s bound an ideal project for a crowd-sourced collaboration. People with different skills squeezed out what improvements they could.
So what is the secret formula of collaborative success through Polymath? The Polymath project attracted people with the right skills, perhaps more efficiently than if the project had been organized from the top down. “A Polymath project brings together people who wouldn’t have thought of coming together,” Tao said.
The culture of a Polymath project is such that everything was out in the open, so anybody could potentially contribute to any aspect. This allowed ideas to be explored from many different perspectives and allowed unanticipated connections to be made. Furthermore, a bedrock principle of the Polymath approach is that participants should throw any idea out to the crowd immediately, without stopping to ponder whether it is any good. It goes against people’s instincts, but great mathematicians make stupid mistakes, too. This makes the project much more efficient when everyone was more relaxed about saying “stupid things” in a supportive environment conducive to exploration and experimentation.
Is there any reason to think that a Polymath approach might not work for computational finance? What would be an ideal project for crowd-sourced collaboration in financial modeling or algorithmic trading?
References:
- Klarreich, Erica (2013, May 19). Unheralded Mathematician Bridges the Prime Gap. Quanta. Retrieved from: https://www.quantamagazine.org/20130519-unheralded-mathematician-bridges-the-prime-gap/
- Klarreich, Erica (2013, November 19). Together and Alone, Closing the Prime Gap. Quanta. Retrieved from: https://www.quantamagazine.org/20131119-together-and-alone-closing-the-prime-gap/
- Surowiecki, James (2004). The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations. Little, Brown.
- Shirky, Clay (2008). Here Comes Everybody: The Power of Organizing Without Organizations. Penguin Press.
- Nielsen, Michael (2008, July 17). The Future of Science. Retrieved from: http://michaelnielsen.org/blog/the-future-of-science-2/
- Nielsen, Michael (2009, January 26). Doing Science Online. Retrieved from: http://michaelnielsen.org/blog/doing-science-online/
- Gowers, Tim (2009, January 27). Is Massively Collaborative Mathematics Possible? Gowers’s Weblog. Retrieved from: https://gowers.wordpress.com/2009/01/27/is-massively-collaborative-mathematics-possible/
- Gowers, Tim and Nielsen, Michael (2009, October 15). Massively Collaborative Mathematics. Nature, 461(7266), pp. 879-881. Retrived from: http://michaelnielsen.org/papers/mcm.pdf
- Nielsen, Michael (2011). Reinventing Discovery: The New Era of Networked Science. Princeton NJ: Princeton University Press.
- Klein, Gary (2013). Seeing What Others Don’t: The Remarkable Way We Gain Insights. Public Affairs.