March 18 and 19, 2013
Shin-ichi Fukuda, University of Tokyo and TCER, Takeo Hoshi, University of California, San Diego and NBER, and Tetsushi Sonobe, GRIPS, Organizers

David McKenzie, Francisco Campos, Aidan Coville, Ana Fernandes, and Markus Goldstein, The World Bank

Learning from the Experiments that Never Happened: Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa

One of the most common policy instruments used by developing country governments to try to foster technological upgrading, innovation, exports, the use of business development services, and other activities leading to firm growth are matching grants. Because they involve subsidizing firms, though, the risk is that they could crowd out private investment by subsidizing activities that firms were planning to undertake anyway, or could lead to pure private gains, rather than generating public gains that might justify government intervention. As a result, such programs require rigorous evaluation. McKenzie and his co-authors attempted to implement randomized experiments to evaluate the impact of seven matching grant programs offered in six African countries, but in each case were unable to complete an experimental evaluation. One critique of randomized experiments is publication bias, whereby only those experiments with "interesting" results get published. The hope of these researchers was to mitigate this bias by learning from the experiments that never happened. They describe the three main proximate reasons for lack of implementation of the projects: continued delays, politicians not willing to allow random assignment, and low program take-up. They then delve into the underlying causes of these. They conclude that political economy, overly stringent eligibility criteria that do not take account of where value-added may be highest, a lack of attention to detail in "last mile" issues, incentives facing project implementation staff, and the way that impact evaluations are funded all help to explain the failure of randomization.

Yukichi Mano, Hitotsubashi University, and Tetsushi Sonobe

Teaching KAIZEN to Small Business Owners: An Experiment in a Metalworking Cluster in Nairobi

Mano and Sonobe assess the impacts of a management training program on the business performance of small enterprises in a metalworking cluster in Nairobi, Kenya. A previous study of this cluster observed that while several enterprises had successfully expanded operation, the majority had been experiencing declining profits because of increasing competition with imported products and with new entrants in the cluster. Based on the observed differences in management between successful and less successful enterprises, the authors design a management training program featuring the basics of KAIZEN -- an inexpensive, common sense approach to management, emphasizing the reduction of wasted work and materials -- for the less successful enterprises. Although their initial intention is to use this training program as a randomized experiment, in the end because of circumstances beyond their control, they give up randomization and allow every business owner interested in the program to participate in it. They find that business owners operating smaller enterprises tend to be self-selected into training participation. The training effects combined with the self-selection effect, which they estimate with panel data, are statistically significant and particularly stronger on profits than on sales revenues. Other training programs that did not teach KAIZEN had positive effects on sales revenues, but not profits. As a result, the participants caught up with and overtook the non-participants in terms of average sales revenues and average profits, respectively.

Alistair Munro, GRIPS, and Bereket Kebede, Marcela Tarazona-Gomez, and Arjan Verschoor, University of East Anglia

Autonomy and Efficiency. An Experiment on Household Decisions in Two Regions of India

Munro and his co-authors use experimental data to examine the implications of regional contrasts in female autonomy in India for the efficiency of family decision making. Analyzing a sample of 1200 couples (from one rural and one urban area in the North -- Uttar Pradesh-- and one rural area in the South, Tamil Nadu), they find large scale and robust evidence of inefficiency and of the hiding of assets when possible. Men invest more and are more generous to their partners, whereas women are more willing to invest in a common pool when their income is earned through working and when assets are publicly observable. The researchers find continuing significant differences between North and South and relatively little evidence that urban living is associated with changes in the nature of marital behavior. There are some differences between responses to treatment, but the key and striking difference between the North and the South is that in both the rural and urban sites in the North, household efficiency is considerably greater than in the South. This suggests a trade-off between autonomy and efficiency.

Tomoya Matsumoto, GRIPS

Disseminating New Farming Practice among Small Scale Farmers: Experimental Intervention in Uganda

After collecting information on the demand for the same inputs from sales workshops organized in 2009 and 2011 in which those inputs were sold, Matsumoto uses a randomized control trial to measure how the free distribution of modern inputs for maize production affects small-scale farmers' adoption of those methods in the subsequent seasons. He finds that the demand for inputs among those who received them for free was significantly higher in both 2009 and 2011 than the demand among those who did not receive them at all. The demand among neighbors of the recipients of free inputs fell somewhere in-between. The initial treatment assignment also has a persistent influence in the farmers' adoption over a two year period, whereas the difference across treatment status -- particularly, between the free-input recipients and their neighbors -- declines. The reduction in the gap in adoption is driven partly by learning through social networks.

Alex Oo and Russell Toth, University of Sydney

Using Framed Field Experiments to Understand Market Behavior in Developing Countries: Do Community-Sanctioned Social Pressures Constrain Microenterprise Growth?

Oo and Toth conduct a framed lab-in-field experiment to explore the hypothesis that a number of stylized facts about microenterprise behavior in developing countries – including product market homogeneity and lack of growth and innovation – can be explained by a social institution in which micro-entrepreneurs share the market to "buy a job." Two hundred and eighty present or prospective market trader women across four communities in rural Vietnam are randomized anonymously into pairs in order to play three "market game" treatments. Their interactions are framed to simulate real-world retail market competition. The participants compete in an effort task, with performance determining market returns. A highly incentivized individual round allows the authors to extract a measure of individual "ability" in the effort task. The subjects then compete in successive treatments, where in the final treatment the losing participant in a round can elect to "burn" the competitor's output, which is framed as the application of social pressure. The behavioral responses are significant and fit with a theoretical model of the social institution the researchers have in mind: even though subjects are from the same community, they are willing to punish ("apply social pressure"); the probability of punishment is increasing in the gap in ability in the pair; and this leads to a decrease in performance from higher-ability individuals. The study provides an example of the use of framed lab experiments to shed light on market behavior in developing countries.

Hisaki Kono, Institute of Developing Economies

Microcredit Games with Noisy Signals: Collusion or Free-Riding?

Compared with individual liability, joint liability can increase strategic default through collusion and free-riding. By using experimental repayment games which mimic micro-credit programs, Kono finds that joint liability increases strategic default when the signals are not precise or not available. His investigation on collusion and free-riding suggests that subjects free-ride under joint liability, but there is no evidence for free-riding under individual liability. One part of the results also support collusion under joint liability, but they too are consistent with free-riding. Subjects do not seem to respond to free-riding when they make decisions on shouldering their partners and on future repayment, which might explain why subjects choose free-riding.

Tahir Andrabi, Pomona College; Jishnu Das, The World Bank; and Asim Ijaz Khwaja, Harvard University and NBER

Understanding Educational Markets: A Sentinel Approach

Andrabi, Das, and Khwaja provide a summary of the LEAPS project, carried out in the Punjab province of Pakistan between 2003 and 2011. Both the research and data collection activities are still ongoing; they highlight the main characteristics of the project and the research completed thus far. This project is unusual in combining a long-term data collection exercise, similar to the Young Lives Project, with a variety of research that includes, but is not restricted to, descriptive research on education markets, analytical work on gender and private schooling, and evaluative research that examines the impact of prospective policy changes on schooling outcomes.

NBER Videos

National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email:

Contact Us