How much do you trust a multi-armed bandit to run experiments...?
The MAB Problem Formulation: The problem of dynamic pricing with limited supply is considered. A seller has k identical items for sale and is facing n potential buyers ("agents") that are arriving sequentially. Each agent is interested in buying one item. Each agent's value for an item is an IID sample from some fixed distribution with support [0,1]. The seller offers a take-it-or-leave-it price to each arriving agent (possibly different for different agents), and aims to maximize his expected revenue. It has been recognized by Babaioff et. al. that even in a setting with limited supply, the Multi-Armed bandit (MAB) approach can still be fruitfully applied.
Soldiers or bandits? It may not be that easy to tell them apart when they're at rest...
The Manchurian Bannermen is a hereditary occupational caste, ranked above others in society, whose members were expected to devote themselves to the state. In China proper, bannermen did not cultivate the fields (as they had in Manchuria) but rather lived off stipends, paid part in silver and part in grain. The dynasty supported banner soldiers and their families from cradle to grave, with special allocations for travel, weddings, and funerals. The banner population grew faster than the need for soldiers. Within a couple of generations, there were not enough positions in the banner armies for all adult males in the banners. Yet bannermen were not allowed to pursue occupations other than soldier or official. Consequently, many led lives of forced idleness, surviving on stipends.
References:
- Babaioff, Moshe, Dughmi, Shaddin, Kleinberg, Robert and Slivkins, Aleksandrs (2013). Dynamic Pricing with Limited Supply. Retrieved from: http://arxiv.org/abs/1108.4142
- Porter Eduardo and Farhad Manjoo (2016, March 8). A Future Without Jobs? Two Views of the Changing Workforce. New York Times. Retrieved from: http://www.nytimes.com/2016/03/09/business/economy/a-future-without-jobs-two-views-of-the-changing-work-force.html
- Rotman, David (2016, March 11). The Danger of the Universal Basic Income. MIT Technology Review. Retrieved from: https://www.technologyreview.com/s/601019/the-danger-of-the-universal-basic-income/