To determine the most important production constraints and associated yield losses for six major food crops in 13 farming systems with high poverty in Sub-Saharan Africa, South Asia and East Asia, surveys were conducted with 672 experts representing a diversity of backgrounds and experience. Respondents reported large gaps between highest achieved crop yield on smallholder farms and average yield on farm. Yield gaps were smallest for rice (about 60% of current average smallholder farm grain yields), mid size for wheat and cassava, and larger (sometimes double current farm yields) for sorghum, cowpea and chickpea. Gaps were also smaller in the high input and yield farming systems of East Asia and largest in the marginal, drier systems, particularly in Sub-Saharan Africa. Four categories of production constraint (abiotic, biotic, management and socio-economic) were considered important contributors to yield gaps. A diversity of specific constraints was reported for the crops in the different systems. The most severe and widespread specific constraints for wheat involved the deficiency, high cost and poor management of N fertilizer, and problems associated with drought stress at grain filling, mid season drought and irrigation management. Those for rice included N fertilizer problems, soil fertility depletion, various leaf, stem and head pests and diseases, weed competition and inadequate water management. Striga and weed competition, soil resource degradation, poor soil fertility management, and drought were the most severe specific constraints for sorghum. Insect pests of pod, leaf, stem and flower and the high cost of their control dominated the constraint set for cowpea. Helicoverpa pod borer, Botrytis grey mould and control costs were the most severe for chickpea. Unsuitable varieties/poor seed, soil infertility and fertilizer constraints were also widespread with the legumes. Marketing problems and lack of finance were concerns for cassava along with weed competition, African cassava mosaic virus and poor varieties/planting materials. The findings can help to inform priority setting for international agricultural research and development activities on important food crops in major farming systems occupying areas of high poverty.