AgMIP Protocols

AgMIP Core Wiki

AgMIP Protocols

AgMIP Global Workshops



Download the July 28, 2011 protocols in pdf format.


Principal Investigators: Cynthia Rosenzweig, Jim Jones, and Jerry Hatfield

Science Coordinator and Climate Scenarios Team Leader: Alex Ruane

Crop Modeling Team Leaders: Ken Boote and Peter Thorburn

Economics Team Leaders: John Antle and Jerry Nelson

Information Technologies Leaders: Cheryl Porter and Sander Janssen

Cross-Cutting Themes

Uncertainty: Daniel Wallach

Aggregation across Scales: Nadine Brisson, Jim Jones, and Alex Ruane

Representative Agricultural Pathways: John Antle, Jerry Nelson, and Cynthia Rosenzweig

Pilot Studies

Wheat Pilot Study: Senthold Asseng, Frank Ewert, Ken Boote, Nadine Brisson, Daniel Wallach, Alex Ruane, Maria Travasso, Soora Naresh Kumar, and Jim Jones

Maize Pilot Study: Nadine Brisson, Jean Louis Durand, Ken Boote, and Jon Lizaso 


Table of Contents

AgMIP Protocols


The Agricultural Model Intercomparison and Improvement Project (AgMIP) is a distributed research activity for historical and future climate conditions with participation of multiple crop and economic modeling groups around the world. The goals of AgMIP are to improve substantially the simulation tools that are used to characterize world food security due to climate change, to assess future world food security utilizing the improved models, and to enhance adaptation capacity in both developing and developed countries. By providing a coordinated set of model protocols, AgMIP enables linked multi-model climate change assessments for agriculture at both regional and global scales in order to place regional changes in agricultural production in a global context that reflects new trading opportunities, imbalances, and shortages in world markets resulting from climate change and other driving forces of agricultural supply and demand. The three-year project provides the necessary integrated, transdisciplinary framework to assess climate impacts on the agricultural sector and builds capacity for continuing agricultural assessment and management under changing climate conditions.


This document describes the AgMIP protocols that describe the process and tasks necessary to conduct the model intercomparisons, improvements, and assessments efficiently and comprehensively. AgMIP research activities are organized under four project teams (Climate Scenarios, Crop Modeling, Economics Modeling, and Information Technologies; see Figure 1), with guidance provided by the Leadership Team and Scientific Steering Group. Three crosscutting themes are emphasized: Uncertainty, Aggregation across Scales, and Representative Agricultural Pathways. The Uncertainty cross-cutting theme explores the contributions of each component to the uncertainty cascade. The Aggregation across Scales cross-cutting theme connects local, regional, and global agricultural information. The Representative Agricultural Pathways cross-cutting theme links to the new scenario process involving the Representative Concentration Pathways (RCPs; Taylor et al., 2011) and the Socio-economic Pathways (SSPs). In order to portray the world food system realistically in its suite of modeling activities, AgMIP is developing several key interactions related to Water Resources, Pests and Diseases, and Livestock. The outcomes of AgMIP include intercomparisons and improvements in crop models, agricultural economic models, scenario methods, and spatial aggregation methods, improved regional and global projections of climate change impacts on agriculture, and capacity building for regional vulnerability assessments, development of adaptation and mitigation strategies, testing the effectiveness of trade policy instruments, and enhancing technological exchange.


Figure 1: AgMIP components and expected outcomes.

The purpose in developing the protocols for the project is to provide guidance as to the operating procedures, progress evaluations, and anticipated deliverables from each project team and for the integration of the project as a whole. Since these teams are central to achieving the overall objectives, the protocols are organized by team; however they describe not only the research tasks for each team but also the interactions among the teams as well. The protocols facilitate the entrainment of a wide group of agricultural researchers into AgMIP activities and the evaluation of the progress of the project by the Leadership Team and the Scientific Steering Group.

General Principles

In developing the AgMIP protocols, the following general principles are emphasized:

  • AgMIP results should be reproducible by external researchers with access to the AgMIP models and database.
  • AgMIP is dedicated to recognizing and attributing fairly and accurately all contributions of data, models, and intellectual inputs.
  • AgMIP activities should address the telescoping set of scales of simulations and analyses from sites to sub-national regions, countries, groups of countries, and the world, with attention to improved characterization of uncertainty.
  • AgMIP methods should facilitate transdisciplinary integration and collaboration.
  • AgMIP activities include improvement of crop and economic models using the best data available, latest scientific knowledge, and best practice for coding models
  • AgMIP encourages the use and availability of open-source simulation codes, models, and methods.
  • AgMIP supports building scientific and adaptive capacity related to agriculture and climate variability and change.
  • AgMIP results are designed to contribute to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change and other assessments.

Science Approach

There are two primary tracks by which AgMIP achieves its goals (Figure 2). The first track is model intercomparisons and improvements. The second track is multi-model assessments of climate change effects on local, regional, national, and global food production and food security.


Figure 2: Two-track science approach to AgMIP research activities.

Both tracks are facilitated by a series of regional workshops to be held in each AgMIP region over a 3-year period of time as well as by global studies and workshops that focus on particular crops and on global analyses. Participants at the regional workshops conduct analyses at field-toregional scales and include both crop and economic model intercomparisons and improvement activities, as well as simulations with guided climate sensitivity tests and climate change scenarios (See Appendix I for the AgMIP Regional Workshop agenda template and the agenda for the first AgMIP Regional Workshop held in Campinas, Brazil, August 1-5, 2011).

The global crop studies are organized to test multiple sites across the agricultural regions worldwide with as many crop models as possible for each individual crop.

For the IPCC AR5, the goal is to contribute new regional and crop-based model intercomparison results as well as a global multi-model assessment, the latter utilizing current versions of crop and economic models. The regional workshop analyses will be designed to provide input to global analyses of climate change impacts and adaptations over time. Both of the parallel tracks will lead to critically important outputs, each targeting outcomes that contribute to AgMIP objectives.

Track One: Agricultural Model Intercomparisons and Improvements

Global Studies. Track One includes activities for model intercomparisons and improvement and for calibrating and evaluating models for use in different regions and for different crops. Teams are organized to compare and improve models for specific crops and regions and for regional and global economic models. Global teams will work on a crop-by-crop basis to intercompare crop models for accuracy and variability of model responses to climatic, management, soil, and growth-influencing factors (see Appendices I and II). Workshops will be held to facilitate model improvements by participating crop model groups through assembling leading experimentalists who can provide the best knowledge about crop responses to climate, soil, and management factors and can provide the best scientific data for improving the models. This includes, for example, assembling scientists responsible for experiments on CO2, temperature, water, and N responses. In the process of testing multiple crop models against such data, the model developers will learn from the data-comparison and each other, the best ways to modify code to improve the simulated responses to those factors. Economic modeling teams will work with crop modeling teams on a parallel set of activities, including the design procedures to aggregate crop model simulations to appropriate spatial scales, the design of representative agricultural pathways (RAPs) to be used for both crop and economic modeling runs, and uncertainty analysis. The economic teams will implement model runs based on a specified set of crop model outputs and RAPs for intercomparison and analysis. These model runs will also be used by regional teams to define price scenarios for their analyses.

Regional Workshops. A series of workshops in each region are designed to contribute to the goals of Track One (three workshops per region over a 3-year time period) and to simultaneously build capacity of scientists in the regions (see section on Regional Workshops. The workshop participants include regional experts as well as members of model development groups to work on specific crops, climate sensitivity and climate change scenarios, and on economic model data and analyses. A target number of major crops is selected for each region.

Participants at the workshop include regional contributors with expertise in agronomy, crop modeling, climate science, economic modeling and analyses, and information technologies as well as members of the AgMIP teams (Crop Modeling, Economic Modeling, Climate Scenarios, and Information Technology Teams). An AgMIP Workshop Report will document all activities, progress made on each of the four workshop objectives, and interactions with stakeholders.

Track Two: Regional and Global Multi-Model Agricultural Assessments

In Track 2, AgMIP conducts assessments of climate change effects on food production and food security at regional to global scales, including analyses of adaptation and mitigation measures over a range of agricultural futures designated Representative Agricultural Pathways (RAPs). AgMIP will produce a database of multiple crop model results from regional sentinel sites (see Crop Modeling Protocols) to create an input dataset for use in regional and global agricultural economic models. The regional assessments will be facilitated by the regional workshops and will make use of the best available methods for aggregation across scales and characterization of uncertainty for use by local to regional decision and policy makers.

The economic models will be used for assessment of world trade and economic impacts of different climate scenarios and agricultural pathways as well as impacts on food security. A fast track use of the aggregated regional datasets is to enable analyses for the IPCC AR5. Economic modelers from the International Food Policy Research Institute (IFPRI), Global Trade Analysis Project (GTAP), Terrestrial Ecosystem Model (TEM), IIASA Basic Linked System (BLS), and others have expressed interest in running their agricultural economic models with the AgMIP input dataset. The goal is for multiple economic models to use the same input dataset and RAPs so that model results regarding the effects of climate change on global agricultural production and trade, as well as food security can be rigorously compared. As more sites and models result from the regional activities, these will be added into the AgMIP datasets for simulations by the global agricultural economic models.

Track 2 utilizes the regional aggregation methods for both inputs and outputs developed in the regional workshops and in conjunction with the economic modelers. This work will integrate the regional results to facilitate simulations and transdisciplinary analyses across scales. The economic protocols detailed below will be used to guide the work of Track 2. Results from the AgMIP climate scenarios and multiple model runs will also be analyzed to gauge related uncertainties.

These efforts will be harmonized with current efforts of IFPRI, GTAP, and other groups. The regional analyses will contribute to the global analyses, which will be coordinated by the Leadership Team.

AgMIP Participation and Attribution Protocols

This section describes protocols for AgMIP participation among the distributed research groups to facilitate project roles and enhance communication among AgMIP scientists and guidelines to ensure fairness in attribution.

Entraining Participants

The goal of AgMIP is to entrain crop modelers and economic modelers throughout the world to participate in comparing and improving agricultural models and creating multi-model simulation of climate variability and change effects on crops, agricultural production, trade, and food security.

To initiate the project, the AgMIP Leadership Team will post on the AgMIP website,, an interest-and-commitment survey to present and potential AgMIP participants, asking for:

1) Commitment to participate

2) Regions of interest

3) Crops of interest

4) Models of interest

5) Role(s) – model user; developer

6) Commitment to provide observed agronomic data for testing and calibrating crop models, with brief description of such data including specific sites

7) Commitment to provide regional climate, soil, management, economic, or other relevant data, with brief description of such data including specific sites

This will be based on the recent survey undertaken through the CCAFS program, and will be disseminated to respondents from the CCAFS survey, the participants in the AgMIP Kick-off Workshop, ICASA, and other agricultural networks, forums, and meetings.

Participants in AgMIP undertake to maintain communication among project members; conform to protocols for site-based observations, simulations, GIS formats for data aggregation; documentation, publications; and share data and results, following the general principles described in this document.


AgMIP is dedicated to recognizing and attributing fairly and accurately all contributions of data, models, and intellectual inputs. Experimental data, other types of data, models, and intellectual property will not be used or published without permission, acknowledgment, and co-authorship. AgMIP values the contributions of all participants.

AgMIP Team Protocols

Climate Scenarios Team Protocols

Leader: Alex Ruane


The Climate Scenarios Team will assist in locating and processing climate data suitable to drive crop models during the historical period evaluation and inter-comparison, and will create protocols and tools for the use of climate model outputs as scenarios for future change. This team will work with Crop Modeling and Information Technologies Teams to facilitate the generation, archival, and dissemination of formatted climate input files for baseline climate, future climate, and climate sensitivity analyses.


1) Provide data, model, and scenario development documentation

2) Establish uniform data standard for AgMIP climate files

3) Provide historical climate information

4) Provide scenarios to test crop model sensitivity to key climate phenomena

5) Provide scenarios of future climate conditions under climate change

The .AgMIP Climate Data Standard File

The climate scenarios team will generate all AgMIP climate data files in a standard .AgMIP file format (see Appendix IV for details). Each file will contain daily information for solar radiation, maximum and minimum temperature, precipitation, surface wind, dewpoint temperature, relative humidity, and vapor pressure over a 31-year period (1980-2010 for the baseline period, for example). This period satisfies WMO recommendations for a climate period in which the climate signal is distinguishable from inter-annual variability, and allows at least 30 growing seasons even for locations where crops are planted at the end of the calendar year and are harvested in the new year. The .AgMIP file standard will serve to streamline tools (developed in conjunction with the IT Team) that will convert climate data and scenarios into the proper formats for each crop model.

Phased Approach

The following protocols will be followed for climate scenario information at all participating sites. For each activity, there are two types of scenarios generated. First phase experiments are scenarios that will be generated for simulations at all locations for consistent aggregation and intercomparison. Second phase experiments are scenarios created for interested researchers; simulations with these scenarios will enable a location to be included in the exploration of important research questions, but are not required of all researchers/locations.

These activities are summarized in Figure 3.

Activity 1: Baseline Analysis and Intercomparison

  • Observed daily climate data for the 1980-2010 climate baseline are sought for all crop modeling locations (Tmin, Tmax, Precipitation and Solar Radiation). In many cases solar radiation may be estimated from measurements of daily sunshine hours.
  • Where available, measurements of surface wind speed and at least one measurement of atmospheric humidity (dewpoint temperature, vapor pressure, specific humidity, and/or relative humidity at the time of day of maximum temperatures) are desired. If these variables are not measured they will be estimated from the NASA Modern Era Retrospective-Analysis for Research and Applications (MERRA; Bosilovich et al., 2010).
1.1. First phase experiments:
  • Local station observations: Collaborate with National Meteorological Agencies and National Agricultural Agencies to collect station observations for 1980-2010 period (as many as possible, especially at sentinel sites)
    • Assess quality of contributed datasets
    • Standardize format for inclusion in AgMIP online archive
  • Geospatial weather generators: 100 years of generated daily climate conditions for each time slice based upon local station observations
    • Weather generators will need to satisfactorily reproduce interannual distributions, include the core climate variables (Tmax, Tmin, Solar Radiation, Precipitation), capture the relationship between mean rainfall and its intensity and frequency, and integrate spatial correlations in and among agriculture regions.
    • The 100-year scenarios will be tested (T-test for mean; Levine’s test for variance; K-test for distribution) against the observed baseline for climate (Tmax, Tmin, Solar Radiation, Rainfall) and crop response (yield, biomass, phenology) variables, with results documented.
    • The 100-year scenarios will be the basis for investigations of interannual variability shifts and near-term climate impacts


1.2. Second phase experiments:
  • Alternative weather generators: Produce 100 years of daily climate time series using different products for weather generator intercomparison 
  • Gridded observational products: Utilize gridded observational datasets (reanalyses, University of Delaware rainfall, etc.) to fill in data gaps. 
  • Gridded satellite products: Utilize gridded satellite products (Meteosat, CMORPH, etc.) for data sparse regions

Activity 2: Climate Sensitivity Scenarios

2.1. First phase experiments:

  • Create weather input for crop model process sensitivity studies: mean temperature with delta approach; rainfall and variance changes with weather generators; CO2 levels according to CMIP3 and CMIP5 emissions scenarios
    • Respond to crop model need for test scenarios
    • Provide guidance on limits of crop sensitivity tests (from RCMs, published literature)
  • Impacts response surfaces for T, P, and CO2: work with the crop modeling team to generate mean change sensitivity scenarios to roughly cover wide range of plausible projections in these three variables.

2.2 Second phase experiments:

  • Temperature variability: Examine crop model responses to range of imposed standard deviation changes (-25%, -10%, -5%, +5%, +10%, +25%) 
  • Temperature extremes: Examine crop model responses to high and low temperature extremes (determined by crop/location) 
  • Rainfall variability: Examine crop model responses to range of imposed changes in the number of rainy days (-25%, -10%, -5%, +5%, +10%, +25%)
  • Rainfall extremes: Examine crop model responses to prolonged wet or dry period (determined by crop/location)
  • Sensitivity to data scaling: Use weather generators driven by reanalysis at GCM grid spacings for observed period to assess scaling uncertainties introduced by weather generators

Activity 3: Future Climate Scenarios

  • Climate scenarios will be generated for each of the following time slices
    • 2005-2034: Near-Term
    • 2040-2069: Mid-Century
    • 2070-2099: End-of-Century 
  • and for each of the following GCM/emissions scenarios combinations
    • 16 GCMs for A2 and B1
    • CMIP5 GCMs for RCP3PD, RCP4.5, RCP6, RCP8.5 (when available)


3.1. First phase experiments:


  • -Enhanced Delta method: Monthly mean changes from GCMs (future 30-year time slice compared to 30-year baseline) imposed on baseline time series
  • Geospatial weather generators: 100 years of generated daily climate conditions for each time slice based upon monthly mean changes from GCMs (as above) with more realistic associations between mean changes and high-frequency variability
    • Top goal – Adjust parameters for mean and inter-annual variability changes from climate time slices
    • Secondary goal– impose additional variation of sub-seasonal metrics


3.2. Second phase experiments:


  • Alternative weather generators: Produce 100 years of daily climate time series using different products for weather generator intercomparison
  • Regional Climate Model (RCM)–based mean and variability scenarios: Where RCM experiments are available, mean and variability shifts may be used in lieu of GCM means to produce future scenarios using either the Delta or Weather generator approaches.


Figure 3: Prioritized scenarios for each climate scenarios activity.

Crop Modeling Team Protocols

Co-leaders: Ken Boote, Peter Thorburn


To evaluate the accuracy of climate and carbon dioxide responsiveness in interaction with management factors for different crop models by comparison to historical climate and yield records so that the models will be effective tools to predict global change effects on crop production around the world, and to make consistent scenario-based projections of future crop production for economic and food security analysis at regional and global scales.


  1. To evaluate different crop models by comparison to observed growth and yield data and published crop responses to climate change variables and their interactions, including methods for simulating responses to carbon dioxide, temperature, water shortage or water excess, as well as management factors.
    1. To evaluate different methods for modeling responses to carbon dioxide, temperature, soil water, and management
    2. To modify model code and parameters to improve predictability
  2. To document and evaluate the uncertainties in modeled outcomes relative to uncertainty of model formulation (comparing different models for the same crop), uncertainties of soil and weather inputs, and uncertainties of model parameters.
  3. To calibrate baseline models for agricultural regions of the world accounting for:
    1. Regional crop management in terms of sowing dates, rotations, irrigation and fertilization practices
    2. Regional variations in soil fertility and water-holding capacities of soils
    3. Regional variations in cultivars (based on generic knowledge on anthesis, maturity, etc., rather than any one model’s specific genetic coefficients).
    4. Yield gap factors related to biotic pests (not related to water or N supply covered in 3a and 3b)
  4. To evaluate adaptation strategies for use in assessing implications for regional adaptation as well as for input to global economic analyses. These strategies include changes in management and genotypic improvement for future climate.


Different crop models are contrasted to each other, highlighting the similarities and differences in model strengths and weaknesses. This helps to determine the effectiveness of different simulation approaches for a particular stress such as CO2, temperature, water deficit, water excess, N stress, extreme events, ozone, or other, and their interactions. Simulations of different models are compared for sites where the historical records of crop yield variations are strongly linked to variation in temperature and rainfall. RMSE and other statistics are used to evaluate degree of prediction accuracy. Yield gaps, soil infertility, and pests will need to be considered. Sites where yields are strongly affected by yield gaps, soil fertility, weeds, insects, or diseases will be analyzed with appropriate consideration of those limiting factors.

In addition, methods for scaling up field-scale simulation results to the regional scale will be developed and compared. Aggregation issues include the spatial scales relevant to climate data, as well as variability of soils and management practices. Uncertainties and biases introduced by aggregation will be explored (see section on Aggregation Cross-Cutting Theme).

Crop model outputs for the sets of calibration and validation tests are archived in the AgMIP database (see model inputs and outputs described in the Appendix II) with documentation and discussion of the model parameterizations and outputs for the historical baseline and future scenarios.

Crops, Crop Models, and Data Requirements

Crops. All crops will be included for which there are simulation models that fit the AgMIP requirements (see below). These include but are not limited to rice, wheat, maize, sorghum, millet, barley, soybean, peanut, common bean, chickpea, faba bean, cowpea, potato, sunflower, canola/rapeseed, sugarbeet. tomato, sugarcane, and cotton.

Crop Models. All models will be included that fit the AgMIP requirements (see below). These include but are not limited to DSSAT, APSIM, STICS, CropSyst, ALMANAC, EPIC, SALUS, APES (Seamless), and INFOCROP.

Crop Model Requirements. For a crop model to be incorporated into the AgMIP evaluation and intercomparison process, the following requirements must be met:

1) Peer-reviewed journal article(s) that describe the development, structure, functions and performance of the crop model, with citations provided.

2) Crop model documentation, with citations.

3) Database with crop model inputs (weather, soil, and management) with documentation on how to extract data for use in running the crop model and for adding additional data.

4) Set of standardized crop model outputs for intercomparisons, improvement, and analyses of regional and global impacts and adaptation options.

Data Requirements: Data requirements for the crop models participating in AgMIP are found in Appendix II.

Activity 1. Testing and Improving Crop Model Responses to Main Growth Factors

Intercomparison and improvement of crop models for accuracy of response to climatic factors of CO2, temperature, soil water, and nitrogen, and their interactions, through comparison to detailed intensive time-series data.

This will be done on a crop-by-crop basis at sentinel sites by global crop teams that include crop modelers familiar with a given crop and scientists who collected the primary data used in the intercomparisons (Free Air CO2 Enrichment (FACE) experiments, temperature transect data, water or N variation data, etc.) (See Appendix III for Wheat Pilot Intercomparison Studies Protocols). This effort will be presented and discussed at Regional Workshops by the global crop teams, with the entrainment of interested regional participants.. Initially, the effort will involve those crop modelers who have models for a particular crop, such as DSSAT, APSIM, STICS, CropSyst, EPIC, SALUS, and APES. The number of models evaluated and improved will vary with crop and participant interest.

1.1 Intercomparison and Testing of Crop Model Responses to Growth Factors

1.1.1 CO2 Sensitivity Tests

a. Document methods by which crop models simulate and predict CO2 responses.

b. Compare simulations to growth and yield dynamics (grain yield, total biomass, canopy evapotranspiration, crop life cycle, etc.) for each given crop in response to specific CO2 treatments, water application, N fertilization, and weather conditions observed in Free Air CO2 Enrichment studies. Initially calibrate models only for ambient CO2 level treatments, prior to testing response to elevated CO2.

c. Compare percent simulated responses to higher levels of CO2 as documented in observed metadata (relative CO2 responses) for variables such as grain yield, total biomass, canopy evapotranspiration, crop life cycle, etc. in a simulation environment as close as possible to the experimental environment. (e.g., data from phytotrons, growth chambers, and FACE experiments).

d. Inter-compare models for their relative response to CO2 in several environments for each crop (cool, moderate, warm, moderate temperature with water deficit, moderate temperature with optimum irrigation, and moderate temperature with N deficiency if cereal). Contrast to metadata.

e. Document observed and simulated responses to CO2 and its interactions with other growth factors.

1.1.2. Temperature Sensitivity Tests

a. Document and inter-compare methods by which models simulate temperature sensitivity.

b. Document and compare to published data, the thermal sensitivity of phasic development including vernalization, “below” and “above-optimum” temperature on processes of development, photosynthesis, grain-set, and grain growth.

c. Compare simulations to absolute growth and yield dynamics (grain yield, total biomass, canopy evapotranspiration, crop life cycle, etc.) for each given crop in response to specific experiments in which temperature variation is present (e.g., transects associated with elevation, latitude, sowing date, or different weather years). Include interactions with irrigation and N fertilization.

d. Simulate response to full temperature range and compare relative responses to percent simulated responses reported in literature (phytotron or statistical weather “regression” trends).

e. Test effects of extreme events such as frost, freeze, and stressfully high temperatures that affect pollination and grain set in simulation experiments and compare to observations. Document observed and simulated responses to temperature and its interactions with other growth factors.

1.1.3. Water Balance

Analyses (with steps similar to 1.1.1 and 1.1.2) will be conducted to compare and test how the crop models simulate soil water balance with emphasis on root growth and on how soil water deficit and soil water excess (saturation to near flooding) affect crop growth processes. Observed and simulated responses to water and its interactions with other growth factors will be documented.

1.1.4. Nitrogen

Analyses (with steps similar to 1.1.1 and 1.1.2) will be conducted to compare and test how the crop models handle soil organic matter mineralization and availability and effects of N supply, in interaction with water stress, on production and grain quality. This will include comparison and testing of how the crop models handle long-term effects in terms of soil water and organic matter. Observed and simulated responses to nitrogen and its interactions with other growth factors will be documented.

1.2 Crop Model Improvement

As an outcome of the model intercomparisons to detailed sentinel site data on responses to CO2, temperature, soils, water, and N, the model developers will learn where their current model coding has deficiencies and they will work with experimentalists/scientists to improve model code and parameterization to improve the accuracy of model responses to these growth-influencing factors.

1.2.1. Improve model codes and parameterizations for responses to CO2 and its interactions with other growth factors based on these tests.

Outcome: Improved parameterization of crop responses to higher CO2 and its interactions with other growth factors in a range of crop models.

1.2.2. Improve model code and parameterization for response to temperature and its interactions with other growth factors, based on these comparisons and tests.

Outcome: Improved parameterization of crop responses to mean temperature and extreme events and interactions with other growth factors in a range of crop models.

1.2.3. Improve model code and parameterization for simulation of crop water balance and its interactions with other growth factors.

Outcome: Improved parameterization of crop water balance including effects of mean changes and extreme events and its interactions with other growth factors in a range of crop models.

1.2.4. Improve model code and parameterization for simulation of nitrogen processes and their interactions with other growth factors.

Outcome: Improved parameterization of nitrogen processes and their interactions with other growth factors in a range of crop models.

Overall Outcome: Improved crop models for use in climate change assessments.

Activity 2. Climate Change Uncertainty Analysis

Evaluation of uncertainty of prediction across an ensemble of crop models, relative to model sensitivity to

1) climatic factors, nitrogen fertilizer inputs, management and soils, and

2) quantity and quality of input data.

This will be done on a crop-by-crop basis with global teams that include crop modelers familiar with a given crop, in parallel to Activity 1The first goal of Activity 2 is to evaluate uncertainties in projected responses to climate change that arises from the use of multiple crop models in a manner similar to the uncertainty analyses used to characterize the range of responses to greenhouse gas forcing of global climate models (GCMs). A second goal is to characterize uncertainty in simulations arising from the quantity and quality of input data.

Uncertainties related to climatic factors and N fertilizer inputs will be determined. The emphasis in Activity 2 is on characterizing uncertainty of model behavior rather than on testing or improving of models as in Activity 1. Effort will include as many models for a given crop as possible. Pilot studies will be conducted for four sentinel sites that differ in agroecological environment with a single treatment per site, in both “model-calibrated” mode and “uncalibrated” mode (simulations to be done prior to calibration) (See Appendix III for Pilot Study Crop Modeling Protocols).

The relative responses of multiple crop models to specified climatic and management changes will allow the quantification of crop model uncertainty to climatic factors, nitrogen inputs, management and soil characteristics in contrasting yielding environments. The uncertainty of an ensemble of crop models for a given crop is comparable to evaluating the uncertainty of climate change scenarios using multiple Global Circulation Models. This will be done with both uncalibrated and calibrated crop models in order to learn whether the uncertainty of relative response of models is influenced by lack of site-specific calibration (as is the case in many agricultural regions). Presentation of the uncalibrated results will not identify individual models.

While the primary focus of the sensitivity analyses in Activity 2 is to determine crop model uncertainty in projecting climate change impacts (by simulating the same input changes using multiple models), the relative response of the different models to CO2, temperature, and rainfall will be compared to published literature and metadata, in conjunction with Activity 1. The central tendencies of the metadata will be presented to indicate what the crop response to those factors should be. (Note: where the uncertainty exercises are completed first, this comparison may inform subsequent model comparisons to data in FACE and temperature-gradient studies).

2.1 Uncertainty Analysis related to Climatic Factors, Nitrogen Fertilizer Inputs, Management, and Soils

Minimum protocols for crop model intercomparisons for Activity 2.1 include the following tests. (Teams may choose to add additional factors and analyses for specific crops and purposes). In all tests, the baseline is simulated with 30 years of historical weather per site.

2.1.1 Sensitivity to Climatic Factors and N Fertilizer Inputs (full multi-factorial)

a. CO2 levels (350, 450, 550, 650, 750 ppm)

b. Mean temperature (Tmax and Tmin changed together) (-3, 0, +3, +6, +9 C)

c. Rainfall (-50, 0, +50% of current)

d. N fertilizer (0, 25, 50, 100, and 150% of reference N application for site)

2.1.2 Sensitivity to Selected Management and Soils Treatments

Current Climate

a. Baseline

b. Baseline + 14 days of Tmax = 35 C (may vary with crop) centered on anthesis

c. Baseline with 10-day earlier sowing date

d. Baseline with 10 day later sowing date

e. Baseline with 50% less N fertilizer

f. Baseline with 50% more N fertilizer

g. Baseline with 20% less available soil water

h. Baseline with 20% more available soil water

Future Climate

a. A single A2-End-of-Century scenario (734ppm)

b. A single A2-End-of-Century scenario (734ppm) + 14 days of Tmax = 35 C (may vary with crop) centered on anthesis

c. A single A2-End-of-Century scenario (734ppm) with 10-day earlier sowing date

d. A single A2-End-of-Century scenario (734ppm) with 10 day later sowing date

e. A single A2-End-of-Century scenario (734ppm) with 50% less N fertilizer

f. A single A2-End-of-Century scenario (734ppm) with 50% more N fertilizer

g. A single A2-End-of-Century scenario (734ppm) with 20% less available soil water

h. A single A2-End-of-Century scenario (734ppm) with 20% more available soil water

Outcome: Multi-model uncertainty analyses of climatic factors, nitrogen fertilizer inputs, management, and soils in current and future climates, including potential changes in means and extreme events.

2.2 Uncertainty Analysis related to Quantity and Quality of Input Data

2.2.1 Initial uncalibrated model simulations. In the Pilot Studies, crop modelers will be given daily weather data, soil characteristics, management, initial conditions and elementary cultivar information (including anthesis and maturity dates) for 4 sites. A first set of simulations for a 30-year-baseline + one single 30-year-GCM generated future scenario will be done using those limited inputs information, but with no other changes to the model (default parameter values).

2.2.2 Model calibration. Prior to the second set of “uncertainty” sensitivity tests, modelers will be given detailed information on measured phenology and time-series data on growth (LAI, biomass, grain, soil water, and N uptake dynamics) to the extent available from those sites. Model calibrations will be allowed for the single treatments per site, to the extent of site-specific parameters and cultivar/genetic traits (but with no change of source code). Calibrations will be documented.

Outcome: Multi-model uncertainty analyses of quantity and quality of input data.

Activity 3. – Regional Climate Change Projections

Model calibration and climate change simulations for sites and regions, using site-specific and non-site-specific regional data. Available data for calibration and aggregation will include site-specific data from time-series experiments and high-quality yield trials, as well as regional yields (e.g., from county, state, and national databases, the FAO, and other sources)) that are not site-specific.

This activity will be done in close cooperation with the climate and economic teams.

The site-specific data needed to calibrate the crop models are given in Appendix II. There will be sites with detailed (intensive time-series) crop measurements as well as yield trials with minimal end of season and phenological development data from more sites (extensive).

Regional yield data (e.g., from the FAO, national statistics) are not site-specific, but are aggregated over soil types, sowing dates, cultivars, and different crop management (fertility and pest control). Methods for improving these regional calibrations and evaluations will be developed jointly by climate scientists, crop modelers, economists, and regional/country specialists who know the distribution of management, sowing dates, flowering dates, maturity dates, cultivar types, soil limitations, resource limitations, and management limitations represented in their region. (These activities will be planned and initiated at the 1st Regional Workshops and continued throughout the period of the AgMIP project, with updates at each of the later Regional Workshops). Each region will develop the major target crops appropriate for that region. At minimum, the workshops would use DSSAT and APSIM models in each region, and selections by regional participants of other models, such as STICS, CropSyst, EPIC, SALUS, and APES, as well as possibly the AEZ model. Uncertainties related to methods of aggregation across spatial resolutions of different data sources will be explored as a source of uncertainty, in collaboration with regional model calibration and aggregation (see both Uncertainties and Aggregation Cross-cutting Themes).

3.1 Detailed intercomparison, evaluation, and calibration of models with site-specific data

There is some overlap with Activity 1, but here the focus is on creating regional projections.

3.1.1 Intensive time-series data collected at platinum sentinel sites: Try for at least 4 locations per country per crop, with available detailed time-series growth, soil water, N uptake and other data exist at field scale, corresponding to treatments related to soil water supply, N fertilization, sowing dates, and other treatments. The calibration will involve minimizing RMSE or a similar criterion. Model developers will be allowed to develop their own crop and cultivar coefficients. Crop and cultivar traits (for same cultivar) will not be allowed to vary across sites, but soil calibration will be allowed (but later contrasted to learn what users did). In this step, we will ask questions of: “Why did this happen? What did we learn? How can the models be improved in regard to overall performance?” This step will help determine accuracy of time-series patterns of growth, water use, N uptake for the different crops and cultivars for the different crop models.

3.1.2 Intercomparisons and calibration of models against end-of-season data from gold sentinel sites (extensive site-specific variety trials): Evaluate and calibrate the models at 10-20 sites in a country/region where site-specific end-of-season data exist on weather, soil traits, input management, cultivars, and life cycle-yield data from variety trials or reliable agronomic trials. It is recommended that some sites include intensive data from experiments that have in-season observations of crop and soil variables to help ensure reliable calibrations for those variety trials and regions. The calibration will involve minimizing RMSE or a similar criterion. Model developers will be allowed to develop their own crop and cultivar coefficients. Crop and cultivar traits (for same cultivar) will not be allowed to vary across sites, but soil calibration will be allowed (but later contrasted to learn what users did).

Outcome: Set of site-based crop modeling results for use in regional aggregation.

3.2 Crop model calibration against region-wide yield data that is not site-specific

A range of aggregation techniques will be tested with climate and economic team members to improved aggregation techniques across soils, management, site-specific weather, and cultivars. Yield gaps will be addressed in this activity. (Crop models will be calibrated against regional data on crop yield using estimated soils, surveyed management, surveyed sowing dates, weather, etc.)

3.2.1 Crop model calibration with region-wide data Crop model calibration will be conducted by crop modelers working with regional/country specialists who know the management, sowing dates, flowering dates, maturity dates, cultivar types, soil limitations, resource limitations, and management limitations typical for their region. Typical farmer fertilization will be used. Sowing dates will come from regional knowledge or sowing date surveys. Cultivar “life cycle traits” will be set from regionally known anthesis and maturity dates. Simulations will be conducted over multiple soils within the regions, using nearest weather sources for soil units. Then simulated yields will be aggregated over soil types, sowing dates, cultivars, and crop management (fertility and pest control). Model calibration will involve minimizing RMSE or similar criteria by comparison to observed yields and other available data in the site datasets over years within regions. At this step, crop model growth patterns are assumed correct from step 3.1.1 and 3.1.2 above, and that model calibration will involve site aspects such as: varying rooting-depth or soilwater-holding traits where yield is underpredicted in low rainfall years, and varying soil “fertility” traits and soil organic matter “pools” to set yield level in higher rainfall seasons. Reducing productivity by modifying cultivar “yield traits” beyond those “yield traits” already set from site-specific trials (in prior steps) is discouraged as there are many yield gap factors, including those with biotic stresses or inability of models to predict excess water effects.

3.2.2 Accounting for Yield Gaps

Where district-wide crop yields over multiple years (10-30 years) are available, the crop models will be simulated against them to determine: 1) whether the crop models are able to accurately simulate yield variation caused by abiotic factors (rainfall, soil type, and temperature) and 2) the extent of yield gaps that exist and the disease, insect, and management factors that cause the yield gaps. Regional country specialists will provide knowledge about pest damage on given crops.

Outcome: Set of crop modeling results for use in regional aggregation based on region-wide soils, management, weather, and yield gap information.

3.3 Regional climate change projections based on crop model results, climate, soils, management, and economic data inputs.

Here all available information will be brought together to create regional climate change projections that can be used in economic model simulations. A range of aggregation techniques will be tested with climate and economic team members to improved aggregation techniques across soils, management, site-specific weather, and cultivars. This activity will be coordinated with the regional and global economic models.

Outcome: Set of regional climate change projections for use in regional planning and global economic model assessments.

Activity 4. – Adaptation and Mitigation

Developing and Evaluating Adaptation and Mitigation Strategies under Future Climate

This activity will use the improved crop models as tools to evaluate adaptation and mitigation strategies in the agricultural regions of the world. There will be multiple models running for each crop, baseline and adaptation pathway on a regional basis. (This activity will be introduced in the 1st Regional Workshops and will continue to be developed in the later workshops.) The initial focus of Activity 4 will be on adaptation; work on mitigation will be developed in the course of the project.

4.1 Adaptation by Crop Management

Sowing date, crop type or cultivar, irrigation, N fertilization, crop rotation will be evaluated as strategies for adaptation to the AgMIP climate scenarios.

4.2 Adaptation by Genetic Improvement

Genetic attributes to improve production and increase tolerance to drought and heat stress will be evaluated for the different crops as strategies for adaptation to climate scenarios.

4.3 Mitigation Strategies

These will be conducted for those models that are capable of predicting soil C and N balance and/or greenhouse gas emissions. These models will be tested to demonstrate and improve capability of simulating the crop and soil processes that affect carbon and nitrogen balance..

Outcome: Quantified evaluations of specific management and genetic adaptation strategies and mitigation options to AgMIP climate change scenarios.

Economics Team Protocols

Co-Leaders: John Antle and Jerry Nelson


The goal of the economics team is to establish a methodological and procedural foundation for the systematic comparison and improvement of global and regional land use, production and trade models used for analysis of climate change impact, mitigation and adaptation in the agricultural sector.

1. Data standardization and documentation

1.1. Identify input and output variables – See Data Appendix

1.2. Define output variables, spatial and temporal resolution for inter-comparisons

1.3. Survey and catalog existing datasets for regional and global models

1.4 Model identification and documentation

-Identify global and regional models

-Review existing documentation, policies for model code etc.

-Establish recommendation for documentation and open code

2. Representative agricultural pathways (RAPs)

2.1 Review literature on IPCC scenarios and RCP, SSP concepts.

2.2 Develop strategy for global and regional RAPs

-identify range of development “storylines” that include detail needed at both global and regional scales, e.g., crop and livestock productivity, farm and family size, management, cost of production

-identify exogenous variables at global and regional scales

3. Methods for Crop-Economic Model Linkages and Aggregation

3.1 Interpretation and use of simulated yields: e.g., absolute vs relative changes

3.2 Spatial and temporal resolution and aggregation

3.3 Develop and compare approaches to statistical characterization of yield distributions

-IMPACT approach (pixel simulations aggregated to production region)

-statistical aggregation approach

-combination of econometric production and crop simulation models

-identify regions with data adequate to undertake comparison of alternative methods

3.4 Identify GHG and environmental variables to be used in regional impact assessment

-soil C, N2O, methane

-other environmental variables: soil nutrient losses, soil erosion, water quality

4. Economic modeling methods

4.1 Identify methodological issues at regional and global scales and strategy for investigating them, e.g.,

-sources of uncertainty, sensitivity analysis

-defining impact Indicators at global and regional scales (income, poverty, environment, social)

-characterizing adaptation (land use, management, technology)

4.2 Global-regional model linkages

-global to regional prices

-consistency between global and regional outputs and costs of production

-consistency across spatial units in global and regional models

4.3 Model validation and inter-comparison methodologies at global and regional scales

-develop validation concepts and strategies, e.g., using historical data

5. Implementation of economic model inter-comparison and validation

5.1 Global models

-identify collaborating global teams and implementation strategy

5.2 Regional models

-identify regions for inter-comparison where TOA-MD and other models have been implemented, develop implementation strategy

-identify regions where global model results can be compared to regional models, reach agreement on variables and spatial scales for inter-comparison

Information Technologies Team Protocols

Co-leaders: Cheryl Porter, Sander Janssen


The goals of the AgMIP IT team are to facilitate the compilation, archiving, and exchange of data and information for the AgMIP research community and stakeholders.


  1. Develop an IT infrastructure for the AgMIP project that allows easy and secure access to shared data, models and results of researchers in the AgMIP consortium, with both a short and long term perspective;
  2. Facilitate primarily the use of data by models and exchange of model results and secondly, the linking of models relevant for the AgMIP project to allow for reproducible and repeatable applications;
  3. Explore the potential of state-of-the-art Information & Communications Technologies, relevant to improve modeling with a long-term perspective, including web-based model executions and Service Oriented Architecture (Bio-informatics)
  4. Organize the online dissemination of AgMIP data and outputs.


The IT team will provide support for all AgMIP participants and teams by facilitating the sharing of data, documents, models, procedures and final outputs with both contributing researchers and project stakeholders. The IT team will focus on data sharing and availability, as we consider these to be the most important critical success factor to model intercomparison and shared use for capacity building. For successful data sharing and availability, data storage, common data access and standardized data formats need to be ensured. Figure 4 conceptualizes a data sharing system in which AgMIP researchers will access data and models through a web-based research interface. The proposed system will likely be a distributed system of databases, with a shared web-based portal, rather than a single centralized storage system. Appendix II of this document presents a preliminary list of data which will be stored in the online database.

A second critical success factor is the availability of a separate interactive stakeholder interface to explore and investigate project outputs by a wider audience of policymakers and food security analysts. This interactive stakeholder interface has to be usable and understandable without modelers or experts help. Ultimately, it increases the chance of re-use of AgMIP results and the transparency of AgMIP throughout the process.

In addition to facilitating the availability of data and model linkages, the IT team will promote and enforce procedures for data quality standards within the AgMIP research community, including metadata standards. The IT team will also recommend guidelines for software development for AgMIP tools, models and applications, including version control, issue tracking, and documentation.

A list of activities developed to meet the goals and objectives of the IT Team are listed below. The AgMIP IT team will seek funding for activities from different sources across projects. These protocols present a vision that ties all the activities and projects together. Some projects can be more research oriented while others have to be more operational in provide real support to users

(i.e. AgMIP researchers and stakeholders). Projects are encouraged to adopt activities (or parts of activities) to contribute to the larger whole. Each of the activities is carried out according to a prototyping approach in which progressively more advanced versions can be delivered and early usability is first versions of tools can be made available fast, to be replaced with more advanced versions of tools resulting from more advanced prototypes. First prototypes must be first developed for activities 1 (already available), 2, 4 and 5.

Figure 4: Database management and exchange system schematic. All project data and models will be accessed via a model and data interface to a common database. A separate community interface will allow project results to be shared openly with a broader community.

Activity 1: Communication

Methods will be developed to enable the distributed AgMIP research groups to communicate effectively; share data, ideas and documents; and to share project products with other participants and with stakeholders. An AgMIP web presence will be the first priority and will be used to present information related to the Regional Workshops, provide a portal for data and models, as well as to enable coherent management for the entire AgMIP project. Other communication methods may include emailing lists, conference calls, online project management sites (such as Central Desktop), web meetings, a project WIKI, as well as planning and coordination among teams at the Regional Workshops.

Activity 2: Model dataset specifications

A global review of available data standards and model input/output sets will help determine possible templates that could be used for description of AgMIP data. An extensible data standard will be developed for use with the project prior to the first Regional Workshop. The IT Team will work in conjunction with Climate, Crop Modeling and Economic Modeling Teams to describe minimum datasets for input, output and data exchange interfaces. These data consist of inputs to the crop and economic models, including outputs from the climate models; data required to interface between the crop and economic models; and experimental data used for model evaluation and intercomparison. These datasets will be designed to be flexible and extensible such that models and data can be added or modified as the project progresses. Specifications for common units and formatting for model data will be generated. Data “filters” will be developed to allow data harmonization from various sources, as depicted in Figure 5. Model outputs needed to generate project result summaries will be identified at the first regional workshops, including variables needed to quantify model uncertainty. All data specifications will be documented in an AgMIP report which will be made available with the data (Activity #5).

Figure 5. Harmonized data storage available for use by models.

Activity 3: Inventory of existing tools

As much as possible, existing methods and tools will be used for data collection and dissemination, model and data interfaces, model evaluation and intercomparison, data aggregation, preparation of presentation materials for stakeholders, etc. An inventory of existing data and methods, including those developed by the iPlant Collaborative, IRI data library, CIAT harmonized data, Victoria Department of Primary Industries, FAO, IIASA, Planetary Skin project and others will be assessed for their applicability to long term storage and retrieval of AgMIP data and products. The inventory will commence prior to the first regional workshops and will continue throughout the life of the project, as each phase of the project continues and as tools generated by complimentary projects become available.

Activity 4: Use case analyses

The IT Team will work with the Climate, Crop Modeling and Economic Modeling Teams to perform a requirements analysis of the type of usages (use cases) that AgMIP teams would expect of the IT infrastructure. An extensible collection of use cases will be produced by the crop, climate and economic modeling teams (see example in Appendix III), which will be the basis for designing of the IT infrastructure (Activity #5). These use cases are crucial for the IT team to achieve an understanding about the biggest IT obstacles to fulfill the objectives of AgMIP from the perspective of crop, climate and economic teams. With the help of the Leadership Team and Steering Committee, stakeholders will be identified and use cases developed based on an assessment of the stakeholders needs for AgMIP outputs. These use cases will form the basis for design of the Stakeholder interactive interface (Activity #6).

Activity 5: AgMIP research data and model interface

The IT Team will design an information architecture that consistently and coherently achieves the requirements for data availability and use and model integration for AgMIP researchers, as depicted in Figure 3. The system will include the database structure for storage of model input and output data; tools which can be used to upload, view, share, search databases; and tools to explore metadata. User account control software will allow creation of new user accounts, authentication, and user-and content-level control of the access to data. Development of this product will occur over time and will be distributed among the regions. This data interface will enable users to reproduce crop or economic model simulations, to modify and re-simulate scenarios, and also serves as an archive for the entire project. The system will provide the needed tools and databases to allow regional users to 1) perform analyses of additional climate change adaptation pathways (management and policies) for the region and specific countries in the region and 2) perform additional analyses on impacts of climate change as new climate scenarios are produced by the global scientific community and as new information on policies and markets become available. Databases and AgMIP user interfaces will be refined based on input from the use cases and Regional Workshops.

One goal of the AgMIP IT Team is to ensure long-term availability of data, models, tools, training materials and methodologies used in the AgMIP project, including crop and economic models, methods for aggregation and uncertainty analyses, interactive tools for analysis of data and prepared reports and presentation materials. This includes choosing an open source / open access license, determining an appropriate distribution server, and establishing procedures to maintain and manage the repository.

Activity 6: Stakeholder interactive interface

A separate interactive interface will be developed to enable stakeholders open access to project results and analyses (Figure 3). These stakeholders may include researchers working on other global food security research projects, for example, those of the Consultative Group on International Agricultural Research (CGIAR) Centers, USAID as well as the global crop modeling community. Summaries of integrations and analyses can be presented in multiple formats including online archive for impacts scenarios and outputs using GIS-based formats; model-based probabilities and uncertainties of regional and global agricultural yields and prices. Highly interactive visualization tools will allow users to combine project output data to create unique graphs, maps or animations relevant to particular regions, commodities or other metrics. (See for an example of a separate interactive stakeholder interface:

Activity 7: Training

The IT Team will develop training materials to allow the AgMIP IT tools and procedures to be used by a wide group of researchers and stakeholders. The IT team will assist other AgMIP teams to develop their training materials. Training materials for researchers will include videos, reports, web pages, hands-on exercises and other techniques to teach AgMIP participants how to use the methods and tools produced by the project. Training materials for stakeholders will include tutorials for the interactive tools for generating maps and graphical outputs. These tools will be made available to AgMIP researchers at the Regional Workshops and will be a focus of discussions with policy-makers.

Activity 8: Facilitate model integration

A longer term objective, dependent on funds and resources available, is to collect or develop methods and procedures that will allow consistent data flows between climate, crop and economic models and to effectively present results of analyses. Examples of such tools are OMS, TIME, KEPLER, FRAMES, MODCOM, OpenMI, etc., Where appropriate, web services could be developed allowing researchers access to these methods using a distributed, web-based, high-performance system. Workflow management tools would be particularly useful for the Track 2, global assessment simulations, where large amounts of data will be accessed, processed, generated and stored.

Cross-cutting Themes

Cross-Cutting Theme 1. Uncertainty in the Estimation of the Impact of Climate Change on Crop Yields

Daniel Wallach


Major objective.

The major objective is to evaluate uncertainty in the estimated effect of climate change on crop yields, on a world wide basis. We want to include uncertainties in climate projections and also uncertainties in yield predictions. The emphasis will be on taking into account uncertainties in yield prediction, which has been studied much less than uncertainties in climate projections. The basic idea is to use an ensemble of crop models as the basis for evaluating uncertainty in yield predictions, in analogy with the use of an ensemble of climate models to evaluate uncertainty in climate projections. AgMIP is uniquely positioned to do this, thanks to the fact that most of the major modelling groups participate in AgMIP.

Secondary objective.

The secondary objective is to improve crop models, in particular their ability to predict the effect of climate change. The major pathway will be through model intercomparison, and sharing of data sets for model evaluation. Model improvement is an open-ended process. We should not make model improvement a prerequisite before looking at the impact of climate change on yield. That would run the risk of putting off the major objective of AgMIP for a long time. AgMIP has two parallel streams that advance at the same time: first using models essentially “as is” (with some screening nonetheless) to evaluate the effect of climate change on yield, and secondly model improvement. There is no universally ‘best’ model. Different models will perform well under different circumstances. It is important to allow modelling teams to interact and to compare their models, and that should lead to some convergence between models and some model improvement, but I don’t think it will lead to a single model at the end. We will use an ensemble of models because there is a legitimate diversity in crop models, which represents the uncertainty in predicting crop yield.

Proposal, for the final step in evaluation of the impact of climate change

Assumption 1

I am assuming that historical year by year crop yields for the major crops and the major growing regions worldwide are available at a spatial scale useful for the economic models and compatible with global climate models. I will refer to these data as data at the “regional” level, though this could be yield data for 0.5° x 0.5° grid cells, for administrative regions, for agro-environmental zones or other. I am a total novice concerning world yield data, but I have seen several papers that use such data such as (Tao et al., 2009) or (Challinor and Wheeler, 2008).

To simplify notation, we consider a single region. Suppose we have observed yields for say the years 1992-2005. These are noted Y1992,K,Y2005.

Assumption 2

I further assume that for each region, several of the AgMIP models could be run. That is, the requisite input variables (soil, weather, management and initial conditions) are available. These would be data from one or a few representative sites and management options per region. (These data could be obtained from regional workshops?)


The basic idea of the procedure is to split the estimation of the effect of climate change on regional yield into two parts: 1. the effect on attainable yield (limited by water and nutrients) at representative sites, estimated using crop models 2) a correction factor that accounts for the difference between the representative sites and the overall region, for the yield gap and for model error.




A crop model is only fully defined when the parameter values are specified

When we talk of DSSAT or APSIM or another model, we have identified the equations, but there may be several versions that differ in the values of the parameters. To fully specify the model, we must also specify the parameter values. What parameter values should be used in the above procedure for using an ensemble of crop models? (A word on vocabulary. I distinguish parameters and input variables. The input variables are conditions which change for each use of the model: daily weather, soil characteristics, management (including the name of the variety) and initial conditions. We have assumed that the input variables are available for the representative sites in each region).

Proposal: The idea would be to provide to all modelling teams whatever information is available about the major variety or varieties in each region (maturity class for example). Then let each modelling team determine the parameters in whatever way they feel is best. For example, this could be to use the default parameter values that are usually provided with each model, except for the genetic parameters which would be based on the information about the major variety or varieties. The choice of parameter values is an important determinant of yield prediction, and each crop modelling team knows best how to parameterize their model. An essential advantage of AgMIP is the participation of the modelling teams, so that we can have the most informed determination of parameter values.

Remark: We would not provide the modelling teams with the historic yield data in each region. Of course it would be possible to calibrate the crop models using that data, but that would not be a good idea. We want to separate the prediction of attainable yield for representative sites (the crop models give this) and correction factors due in particular to heterogeneity in the region and yield gap (given by the correction factor). Calibrating with the yield data would mix them together.

Reality checks

The proposed procedure will be a good approximation to the true impact of climate change on yield if

    • Most of the effect of climate change comes from changes in attainable yield at representative sites. The effect of climate change on the correction factor should be slight. We could partially check this on historical data. We would want the correction factor to be fairly constant, with only mild variation from year to year.
    • The crop models give good predictions of the effect of climate change on attainable yield. We could examine the historical data to see how much of the variability is accounted for by the crop models. For example, we could calculate


    • We are more limited in how to check that the effects of elevated CO2 are reasonably estimated. The crop model protocols cover this.
    • We also want the estimated uncertainty to be reasonable. The uncertainty in predicting attainable yield is represented by using an ensemble of models. We would want to check that attainable yield is within the range of model predictions. This could perhaps best be done by using data from specific sites, rather than regional data. We would run the different models at various sites, and verify that the observed yield is within the range of predicted yields. The crop model protocols cover this.


    1. Challinor,A.J., andT.R.Wheeler. 2008. Use of a crop model ensemble to quantify CO2 stimulation of water-stressed and well-watered crops. Agricultural and Forest Meteorology 148:1062-1077.
    2. Tao,F., Z.Zhang, J.Liu, and M.Yokozawa. 2009. Modelling the impacts of weather and climate variability on crop productivity over a large area: A new super-ensemblebased probabilistic projection. Agricultural and Forest Meteorology 149:1266-1278.

Cross-Cutting Theme 2. Aggregation across Scales

Nadine Brisson, Jim Jones, Alex Ruane

AgMIP research initiatives must overcome significant obstacles in scale dependence in connecting field-level crop model output to regional and global scale economic models. Additionally, the climate information that drives these simulations ranges from in situ observations to gridded observational products and climate simulations. The Aggregation across Scales Theme will develop procedures to scale field-level outputs up to regional and country scales, as well as to downscale climate and economic scenarios to drive field-level experiments.

Aggregation and disaggregation are facilitated by the availability of quality geographic data regarding the spatial distribution of climate (daily weather), topography, soils, land-use, farm-level management, and socioeconomic conditions throughout a region of interest. While excellent data exist in some regions, the most data-sparse regions are often also the regions with the largest spatial heterogeneity in farming conditions and practices. For these regions AgMIP will investigate the potential of satellite, remote-sensing, and other observational products to fill in data gaps.

Aggregation of field-scale crop model outputs to a regional or larger-scale economic model generally follows one of several approaches (e.g., Hansen and Jones, 2000). One approach involves disaggregating the region into approximately homogeneous sub-regions in a type of biophysical typology (Hazeu et al., 2010) with calibrated sentinel sites for crop model simulations, and then converting yields to a regional production using planted areas in each subregion (Burke et al., 1991; de Jager et al., 1998; Yu et al., 2010; Ruane et al., 2011).

Another approach uses multivariate sensitivity tests to cast probabilistic distributions of farm-level conditions into an estimate of regional production (Haskett et al., 1995). In a third approach, farm behavior is explicitly taken into account, and crop models are linked to farm economic models to provide farm production estimates, which can subsequently be upscaled through response functions (Pérez Domínguez et al. 2009; Ewert et al., 2009).

A fourth approach is to make crop model simulations at high spatial resolution but with relatively coarse management differences. Relative responses to different climate futures are then aggregated up to economic units of analysis and used to adjust exogenously determined changes in productivity (Nelson et al., 2010).

Cross-Cutting Theme 3. Representative Agricultural Pathways

John Antle, Jerry Nelson, Cynthia Rosenzweig

To enable a simulation framework with consistent climate, economics, and field-level assumptions across a range of scales, a cross-cutting activity is building on previous and current agricultural scenario development to create a set of Representative Agricultural Pathways (RAPs) that provide a linked set of necessary variables for field-level crop models and regional and global economic models in AgMIP assessments (Figure 13). These scenarios will bound uncertainty in each region to allow stakeholders and policymakers to assess risk, and will also contribute to monitoring, evaluation, and decision-making.

In order to ensure that the climate and agricultural scenarios are not contradictory, the starting basis for these scenarios will be the SRES emissions scenarios and RCPs used in the IPCC AR4 and AR5, respectively (SRES, 2000; Taylor et al., 2009). The scenario’s description of national, regional and global policy will also link to the socio-economic scenarios developed for IPCC AR4 and being developed for AR5 (Moss et al., 2010). Potential scenario variables for economic models include population growth, income growth, technology changes, as well as trade, investment, energy, and agricultural policy.

AgMIP RAPs will also act to capture plausible farm-level improvements, as climate change impacts assessments that assume static farm management are generally pessimistic in their lack of development and adaptation (Burton et al., 2001). To better model crops at the farm scale, the economic, technological, and scientific development of each agricultural region will be used to specify plausible regional land use, irrigation, fertilizer and chemical applications, and the improved genetic characteristics of cultivars that may be developed or more widely distributed in the coming decades.

Several RAPs will be created and will specify evolving conditions for farm-level management options and country/regional-level economic policies over the 21st century. AgMIP RAPs will facilitate an important assessment of the scale-dependent and intertwining roles of climate change, economic development, and adaptation on the agricultural sector. It is also our hope that AgMIP RAPs will act to standardize agricultural model simulations of future conditions for research projects in the coming years, allowing independent researchers to directly compare their results and building the transdisciplinary community of agricultural modeling.

Regional Workshops

Below is are descriptions of the goals and activities of the three workshops to be held in each AgMIP region.

AgMIP Regional Workshop 1. Getting Going: Data and Model Assembly, Model Evaluation, Preliminary Climate Sensitivity Responses, and Guiding Questions

The first workshop, to be held in 2011-2012 in each target region, is designed to focus primarily on comparing crop model relationships for simulating climate change variables and on evaluating the crop models using data from the region. Regional data will be used to the extent possible; ways that CO2, temperature, water, nitrogen, and management are included in the models will be documented and reported.

There is substantial preparation ahead of the workshops for the AgMIP teams and the regional partners. This preparation include data and model assembly for crops, climate, and economics, e.g., crop model cultivar, soils, and management inputs, weather data, and economic data for 4 to 10 sites per region. Climate observations provided to the climate scenarios team with adequate lead time will be used as a basis for climate scenarios that will be explored at the workshop. Objectives of the first workshop, which will address aspects of both Track 1 and Track 2, are to:

1) Evaluate Crop Model Responses. The participants will, at a minimum, conduct crop model evaluation runs comparing observed and simulated yields, simulate crop responses to T and CO2, run sensitivity analyses relative to soils, current weather and management practices in the region.

2) Initiate Crop Model Inter-Comparison. The task is to conduct an initial comparison of at least two crop models with and accompanying uncertainty analyses using regional sentinel datasets, assembled jointly by AgMIP regional partners and teams (to be continued in Workshop 2). The goal is to simulate a initial set of the first priority climate change scenarios with the crop models for 4 to 10 sites per region.

3) Define Yield Gaps, Develop Prototype Aggregation Method, and Draft Regional Agricultural Pathways. The tasks here are to define and characterize the yield gaps in the region, develop a prototype of aggregation methods to translate crop model outputs to economic model inputs, and draft plausible Agricultural Pathways for production and adaptation for the regional impact assessments (management practices at field, farm, and regional scales), based on guiding questions from regional stakeholders.

4) Develop Preliminary Regional Impact Assessment. Based on activities of the first three steps, develop a ‘strawman’ production response from climate change scenarios for analysis and possible implementation of the Minimum-Data Tradeoff Analysis (TOA-MD) and other regional economic models if available. This assessment tool is a simulation model designed to be used by multi-disciplinary research or technology deployment projects to carry out quantitative assessments of regional economic, environmental and social impacts associated with agriculture.

All Workshop activities, including stakeholder interactions, will be documented in a Workshop Report.

See next section for a generalized structure and agenda for the first set of workshops in each region.

AgMIP Regional Workshop 2. Mid-Course Milestones: Crop Model Intercomparison, Regionalization, Climate Change Responses, and Guiding Questions

Preparations for the Second AgMIP Regional Workshops include continued refinement of the research tasks covered in the first workshop for both Tracks 1 and 2. The second set of regional workshops will be held in the second half 2012/first half 2013. Stakeholder interactions will either be part of the workshop or will be organized as site visits to relevant regional agricultural agencies and organizations. Objectives for the second regional workshop are to:

1) Deepen Progress in Crop Model Validation and Improvement. The task here is to continue the crop model evaluation and improvement based on Workshop 1 activities, subsequent ongoing work by project participants in their institutions, and intervening site visits.

2) Conduct Crop Model Inter-Comparisons. The tasks here are to continue the comparison and documentation of at least two crop models and associated uncertainty analyses using regional sentinel and other datasets developed in the course of the project and the first-priority climate change scenarios, assembled jointly by AgMIP regional partners and teams. Complete simulations of first priority climate change scenarios with the crop models for the region.

3) Refine Yield Gaps Analysis, Aggregation Methods, and Regional Agricultural Pathways. This task is to refine the yield gap analysis, aggregation methods to translate crop model outputs to economic model inputs, and the plausible Agricultural Pathways for production and adaptation for the regional impact assessments (management practices at field, farm, and regional scales), based on guiding questions from regional stakeholders.

4) Continue Development of Regional Impact Assessment. Based on continuing activities of the first three steps, continue the implementation of the Minimum-Data Tradeoff Analysis (TOAMD), and other regional economic models if available. This will include a critical assessment of regional data bases assembled, example simulations and analyses, and procedures for going from field to regional scales.

All Workshop activities, including stakeholder interactions, will be documented in a Workshop Report.

AgMIP Regional Workshop 3 Finalizing the AgMIP Regional Assessments

A final workshop will be held in each region in the last half of 2013 or in early 2014 with both scientific and stakeholder parts of the agenda, working to finalize results for both Tracks 1 and 2.

In the scientific part, model results from each of the models used in the analyses to produce intercomparison results will be analyzed and evaluated. An assessment of results will be completed followed by decisions to make any final simulation runs to complete the regional assessments. In the stakeholder part of the workshop, a stakeholder forum will be invited to present and discuss findings and possible implications for regional food production and adaptation to climate variability and climate change. This stakeholder engagement will also provide information on future needs for the region relative to policies and research.

1) Finalize Crop Model Evaluation and Improvement. The task here is to finalize the crop model evaluation and model improvement based on Workshops 1 and 2, subsequent on-going work by project participants in their institutions, and intervening site visits.

2) Finalize Crop Model Inter-Comparisons. The tasks here are to finalize the comparison of at least two crop models and associated uncertainty analyses using regional sentinel datasets and other datasets developed by the project and the first priority climate change scenarios, assembled jointly by AgMIP regional partners and teams.

3) Finalize Yield Gap Analysis, Aggregation Method, and Regional Agricultural Pathways. This task is to finalize the yield gap analysis, aggregation method to translate crop model outputs to economic model inputs, and plausible Agricultural Pathways for production and adaptation for the regional impact assessments (management practices at field, farm, and regional scales), based on guiding questions from regional stakeholders.

4) Finalize Regional Impact Assessment. Based on continuing activities of the first three steps, continue the implementation of the Minimum-Data Tradeoff Analysis (TOA-MD), and other regional economic models if available. This will include a critical assessment of regional data bases assembled, example simulations and analyses procedures for field-to-regional scales.

All Workshop activities, including stakeholder interactions, will be documented in a Final Workshop Report

Draft Structure and Agenda for AgMIP Regional Workshop 1

Getting Going: Data and Model Assembly, Model Evaluation, Preliminary Climate Sensitivity Responses, and Guiding Questions

Purposes: Evaluate crop model responses; initiate crop model inter-comparison; define methods to account for yield gaps, define prototype aggregation method, and develop draft regional agricultural pathways; develop preliminary impact simulations at a few selected locations in the region. Accomplishing these tasks will include: Characterizing crop model responses to climate variables, performing crop model intercomparisons and evaluations under varying climate and other conditions, and scoping economic model(s) to be used at regional scales.

Day 1

  • Welcome and Introductions of Participants, Stakeholders in attendance for preliminary guiding questions and inputs
  • Objectives of Workshop
  • Stakeholder Panel: Key Challenges for Agriculture in Region
  • Description of AgMIP
  • Overview of Crop Models (DSSAT, APSIM, plus 2 or more additional models)
  • Overview of Climate and Climate Change in AgMIP
  • Overview of Regional Impact Assessment in AgMIP including economics
  • Crop Modelers: Description of how each model simulates effects of temperature, CO2, and water availability on crop growth and yield (Procedures for doing this are in the Crop Modeling Procedures/Protocols section. Prior to the workshop, the crop modeling team will prepare a brief (about 5 pages) overview, with references, to characterize how response to each of these climate variables is modeled. These will be discussed at the workshop.)
  • Climate Specialists: Discuss sources of climate information, climate projections, and related uncertainties.
  • Economists: Description of variables and core processes of economic models.

Day 2

  • Morning Check-in with all participants
  • Crop Modelers: Intercompare models using the same baseline weather, soil, and management conditions. These runs will establish the baseline for intercomparisons among models prior to improvements, and these intercomparisons will be for specific climate, soil, and management conditions of the region. Prior to the workshop, regional crop participants will work with the leaders of the Crop Modeling Team to assemble these regional baseline climate, soil, and management conditions for the intercomparisons and they will summarize the conditions to document the baseline. There will not be observed crop response data for these comparisons. Scenarios of temperature changes (on each day), CO2 changes, and rainfall changes will be created for use in creating simulated responses to those ranges of climate inputs by each model. We recommend that these runs be made prior to the workshop so that results can be summarized for each crop and climate factor variation (single factor T, CO2, and rainfall variations, as well as limited combinations). However, participants will also run examples of these combinations to explore sensitivities of model results for their region. These simulations will be performed for rainfed and no-water stress (potential yield) conditions.
  • The results from each model will be combined during the workshop to facilitate discussions and information that may help improve the models relative to their responses when compared with published metadata on CO2 and temperature responses.
  • Climate Specialists: Inventory, and do quality assurance and quality control on all weather and climate datasets. Introduce delta method, types and application of weather generators, and structured climate scenarios for sensitivity analyses.
  • Economists: Inventory, and do quality assurance and quality control on all economics datasets for the region.
  • End-of-Day Check-in

Day 3

  • Crop Modelers: Prepare sentinel data from time-series experiments and high quality yield trials for model evaluation. The data needed to evaluate the models are given in Appendix II, however we envision that there will be experiments with detailed (intensive time-series) crop measurements as well as yield trials with minimal end of season and phenological development data from more sites (extensive). The data would again have to be assembled prior to the workshop and loaded into the web site by the IT team for accessibility. The number of data sets are likely to vary by region, and only relatively few data sets are likely to be used during the workshop. These datasets will be made available for use with each crop model and each crop that is being analyzed in each region. A goal is to get at least 4 intensive datasets to work with each crop model.
  • Participants working with each crop model will perform any needed calibration (parameter estimation) and simulate the experiments, carefully checking the quality of data input to the models.
  • Results from each model and crop will be assembled for intercomparison against observed data and evaluation of the models’ abilities to simulate historical yields for range of climate, soil, and management conditions.
  • The intercomparisons will be discussed by the participants with the goal of identifying possible needs for model improvement and for deciding on additional intercomparison and evaluation analyses to perform following the workshop.

Days 4

  • Introduction to regional economic models (e.g., the TOA-MD model and others if available). This includes what the model will provide relative to assessing impacts and adaptation pathways in a variable and changing climate
  • Training of participants on use of the TOA-MD model (and others) and presentation of data requirements for implementation in the region
  • Plan for data collection and workshop II during which the PTIA model will be implemented for the region

Days 5 Refining and Preparing for Workshop II

  • Presentation and discussion of methods for scaling up to aggregate production (scaling up team members)
  • Presentation and discussion of how to account for yield gaps while scaling up.
  • Presentation and discussion of methods for uncertainty analyses (uncertainty team members)
  • Presentation from regional experts on data sources (how to access climate, soil, management, yields, economics) for conducting climate change assessments over broad regions, to include scaling up crop model analyses and economic analyses
  • Introduce the concepts of adaptation pathways and define plausible adaptation and mitigation pathways for the region (crop management and policies as appropriate)
  • Presentation and discussion of climate scenarios and other scenarios to be used for the regional analyses

Stakeholder Interactions: Meet with Stakeholders to Present Progress and Elicit Guiding Questions and Feedback


Appendix I: Data Requirements

Crop Model Input Data:

1) Weather: Daily solar radiation, Tmax, Tmin, Rainfall, CO2

2) (Desired: wind speed and humidity measurement, e.g. dewpoint temperature, vapor pressure, specific humidity, relative humidity)

3) Soils: Soil texture, soil surface color, depth to bottom layer suitable for roots (not crop specific). Then, for each soil layer: percent sand, percent silt, percent clay, coarse fraction, percent organic C, moist bulk density, pH (Desired: sum of bases (CEC, cmol/kg), extractable aluminum (cmol/kg), exchangeable Ca (cmol/kg)). NOTE: Crop models need to derive SAT, DUL, LL from soil characteristics. We need to agree on pedotransfer algorithm.

4) Management: sowing date, sowing depth, row spacing, plant population, cultivar, tillage operations and dates.

5) Fertility: N, P, K, and other fertilizer applications (kg/ha and DOY)

6) Irrigation: amounts (mm depth) and dates (DOY) of application

7) Weed control: herbicides used, dates, & amounts applied, degree & timing of mechanical weed control

8) Agrichemicals for disease and insect control, product, amount, DOY applied

9) Crop cultivar traits (Either these are model-specific genetic coefficients, or we must define them in a specified environment such as optimum warm temperature or short days or long days): date of anthesis, date of beginning grain growth, date of physiological maturity, potential grain size (mg/grain)

Crop Model Output Data

1) Minimum In-season (leaf area index, leaf, stem-culm-petiole, grain, total crop mass, total crop N mass, grain N mass, daily transpiration, daily ET)

2) Minimum End-of-season (grain yield, grain size, maturity date, total crop mass, total crop N uptake, grain N off-take, irrigation requirement, seasonal N mineralized from soil, changes in soil C by layer)

3) Time Series (soils): soil water content by layers, soil nitrate and ammonium concentration by layers, soil C by layer, N leaching by layer

Trade Model Input Data

1) supply side:

  • prices,
  • quantities of inputs and outputs,
  • transport 

2) demand side:

  • income,
  • population,
  • demographic 

3) policy:

  • domestic,
  • trade interventions,
  • consumption subsidies

Data for linkages between crop models and economic models

1) Definition of spatial and temporal units and format for crop model outputs

2) Crop model output variables

  • single crops, crop rotations, simple and complex intercrops
  • crop variety

3) Crop model management variables

  • fertilizer, seed, agrichemicals, water, labor, animal and mechanical power, pest management
  • management systems, e.g., tillage
  • spatial and temporal resolution

Climate Scenario Data A large storage archive will be needed to collect the datasets required for scenario generation. An archive of publicly-available gridded datasets (retrospective analyses, GCM outputs, etc.) will be constructed at Columbia University/GISS. The end goal of the climate scenarios team is to produce climate scenarios for AgMIP simulations, which will be placed online for AgMIP users. All data used will be documented (including links to public dissemination sites).

Climate datasets of interest include:

1) Historical observations from meteorological stations in agricultural regions (not always publically available)

2) Reanalysis model outputs (NCEP/NCAR Reanalysis-1, NCEP/DOE Reanalysis-2, NASA MERRA, ERA-Interim, GLDAS, NCEP CFSR)

3) Observational products (CRU/Tyndall Center, WorldClim, Sheffield et al., 2006, NASA POWER)

4) Baseline and future climate model output (CMIP3; Meehl et al., 2004; CMIP5; Taylor et al., 2010)

5) Baseline and future climate dynamically downscaled output (e.g. NARCCAP, CORDEX, CLIMAS, ENSEMBLES, PRUDENCE)

6) Baseline and future climate statistically downscaled output (Sheffield et al., 2006; Maurer et al., 2007; Robertson et al., 2010)

Climate variables in focus:

1) Daily rainfall (mm/day)

2) Maximum, minimum, and mean daily surface (2 meter) temperatures (degrees Celsius;

maximum and minimum temperatures not always kept in observations and models)

3) Solar radiation (MJ/day; may be calculated from sunshine hours)

4) Dewpoint temperature (degrees Celsius; more difficult to obtain in observations and

model output)

5) Surface (10 meter) wind speed (more difficult to obtain in observations and model output)

Appendix II. Wheat Pilot Study

Intercomparison of Crop Models for a Given Crop to Evaluate Uncertainty and Responsiveness to Climatic and Management Factors

Senthold Asseng, Frank Ewert, Ken Boote, Nadine Brisson, Daniel Wallach, Alex Ruane, Maria Travasso, Soora Naresh Kumar, and Jim Jones

Improved and fine-tuned during the Amsterdam workshop on April 8/9, 2011

May 18, 2011

Multiple models for a single crop will be compared. This exercise will be initiated prior to the AgMIP Regional Workshops by crop modelers who have interest (initially, for wheat crop models, until modelers of other crops do the same). The focus is on uncertainty of crop model responses to climate factors, obtained by having multiple crop models.

Objective: The objective of this model intercomparison is to quantify the uncertainty of crop model simulations for climate change impact and adaptation studies at a global scale, with an initial focus on wheat grain yield and grain protein.

Main research question:

What is the uncertainty of crop models to project climate change impact on wheat production?

Aims (year 1 + 1/2):

1. To quantify the relative uncertainty of wheat crop models simulating climate change impacts:

a. for four different yielding environments,

b. for ‘less system-knowledge situations’ and for ‘detailed system-knowledge’ (which allows calibration of the model) separately,

c. for selected crop management variations and soils

d. in relation to GCM-generated climate change scenarios (part II of pilot).

2. To analysis sources of uncertainty and compare with metadata.


The simulation experiments/runs by the individual modeling groups need to be finalized by late 2011 (part I) and early 2012 (part II) and prepared for scientific publications by mid 2012. See detailed deadlines below.

Approach: The wheat pilot (pilot-part I).

i. ‘less system-knowledge situations’ Modellers will be given daily weather data, soil characteristics, management, initial conditions and elementary cultivar information (including anthesis and maturity dates) for 4 locations with different growing conditions. A first set of simulations for a 30-year-baseline + one single 30year-GCM generated future scenario will be done using those limited inputs information, but with no other changes to the model (default parameter values).

ii. ‘detailed system-knowledge ‘ A second and main part of simulations will be done after model calibration using measured grain yield and grain protein% and detailed measurements based on growth, water and N uptake and water and N dynamic data (if available) from the locations.

The first simulations represent uncertainty when yield data and detailed information are not available (as will often be the case in a worldwide study due to lack of measurements) or when limited resources restrict the calibration of models against detailed observations.

The simulations after calibration will indicate if the uncertainty of crop model projections will differ with different system-knowledge information.

Possible analysis of results:

  1. The analysis will include absolute and relative yield and STD change with CO2, temperature, rainfall an N. To analyse absolute and relative values it will be useful here to look at other outputs that could help to understand why the models give those results. Each model could output in addition to yield and GP% e.g. total biomass production, duration of growing season (emergence-harvest), duration of grain filling period, phenological dates (emergence, flowering, harvest), maximum LAI, an overall measure of water stress and overall measure of N stress (for periods of emergence-anthesis and anthesis-maturity) and annual water amount drained and nitrate leached under the potential maximum root depth of a crop. This would allow additional analysis of the simulated data base and the analysis could concentrate on why different models give different results, and not just on the differences in the final results. A detailed list of output variables will be supplied.
  2. Interactions with management: The emphasis is on ‘if and how much’ the uncertainty in simulating climate change impact will differ with changes in management, soil and seasonal variability. E.g. does the relative mean and relative STD of simulated climate change impact differ under different management, soil and seasonal variability?
  3. During the Amsterdam workshop, we decided to add N factors across T x CO2 x rainfall to the simulation experiment to allow a separate analysis on uncertainty of climate change impact on grain protein% simulations to be analyzed by Pierre Martre.

a. Simulate four contrasting production levels (pilot-part I).

‘less system-knowledge situation’:

Four contrasting data sets from:

  1. The Netherlands,
  2. India,
  3. Argentina and
  4. Australia

will be supplied to each modeling group. These data sets will be well documented and “straight forward”. The modelers will be asked to simulate these data sets, first without any calibration of the model against the observed growth and yield data (as growth and yield data will not be supplied initially). This will be the situation for most of the future climate change impact and adaptation studies where no growth and yield data will be available for model calibration. The aim is not to see who gets closest to the observed data but to supply the same input variable to all the crop modelers. The spread of the model results will be a measure of model uncertainty in the absence of detailed system-knowledge. As the individual model performance is not relevant individually in the uncertainty analysis, the individual results will not be identified.

The modelers will be asked to simulate the 30-year baseline and a single A2 End-of-Century scenario (including 734ppm CO2) for each location with climate data supplied to them. No other simulations will be required for the less system-knowledge situation.

Main simulation experiment-‘detailed system-knowledge’: A comprehensive sensitivity analysis will be done with each individual crop model after crop modelers had the chance to calibrate, if necessary, their model to bring the simulation results close to the observed data, in particularly the yield and grain protein%. The aim again is not to see which model gets closest to the observed data but to bring each model to a similar production level (a high and a low production level with the same/right pathways). This will create a common base for the “sensitivity analysis”, repeated for the four locations.

Models will receive additional information on soil characteristics, variety, management, observed dynamics of crop (biomass, LAI, N content), soil (water, N in layers), yield (grain yield, grain size, grain number, grain N). These data sets are mostly publically available and can be distributed with acknowledging the original source.

b. Sensitivity analysis (pilot-part I)A sensitivity analyses with each individual model will be asked to be carried out on the four common locations.

First, the modelers will be asked to simulate the 30-year baseline and a single A2-End-of-Century scenario (including 734ppm CO2) for each location again, but this time with their calibrated models.

This will be followed by a detailed sensitivity analysis.

The sensitivity analysis will include a multi-factorial analysis on climate factors and N and a selected set of individual factor combinations.

total of 384 runs each for 30 years and for each of the four locations will be required from each individual model.

Full multi-factorial (T x rain x CO2 x N) analyses on climate impact (for 30 years of historical data)The modeling groups will be supplied with manipulate weather data sets (or requested to manipulate their weather data sets accordingly) for the locations by creating weather records with changed daily values for:

  1. Baseline + Mean Temperature (Max & Min changed together) (-2, 0, +2, +4, +8oC offset from current)
  2. Baseline + Rainfall (-50, 0, +50 of current)
  3. Baseline + CO2 levels (350, 450, 550, 650, 750 ppm)
  4. Baseline + N fertilizer (0, 25, 50, 100, 150% of reference N application at specific location)

All these steps will be done as a full multi-factorial experiment (5x3x5x5=375 simulations).

Selected treatment analysis. The following nine simulations are selected treatments to analyze heat stress impact and climate change interaction effects with changes in management and soils.

  1. Baseline + 7 days of Tmax=35 oC start at measured anthesis dateA
  2. A single A2 End-of-Century scenario (734ppm)
  3. A single A2 End-of-Century scenario (734ppm) + 50mm irrigation at anthesis
  4. A single A2 End-of-Century scenario (734ppm) -10 days in sowing date
  5. A single A2 End-of-Century scenario (734ppm) 10 days in sowing date
  6. A single A2 End-of-Century scenario (734ppm) -50% N fertilizer
  7. A single A2 End-of-Century scenario (734ppm) 50% N fertilizer
  8. A single A2 End-of-Century scenario (734ppm) -20% PAW of soil
  9. A single A2 End-of-Century scenario (734ppm) 20% PAW of soil

– same measure anthesis date assumed for each of 30 years in baseline

These steps will be done as selected-treatments (9 runs for each location and each for 30 years). The outputs will be used to analyze the simulated heat stress response and if the relative impact to climate change will differ under various management and soils.

Outputs used for the uncertainty analysis will include grain yields (t/ha with 0% moisture) and grain protein (%).

c. Uncertainty of model response to climatic and management factors (pilot-part I)The responses of multiple crop models to these changes will allow us to quantify the model uncertainty to climate change and in combination with management changes, and soil characteristics uncertainty in a contrasting yielding environments. This evaluation of an ensemble of crop models is comparable to evaluating uncertainty of climate change scenarios using several Global Circulation Models.

d. Comparison of model response to literature and metadata (pilot-part I)While the primary focus of the sensitivity analyses is to determine crop model uncertainty in projecting climate change impact (by simulating the same input change with multiple models), the relative response of the different models to CO2, temperature and rainfall should be discussed and compared to published literature and metadata indicating what the crop response to those factors should be. At a later stage (after the initial wheat pilot), we will try direct comparisons to available FACE and temperature-gradient data sets that exist.

e. Uncertainty of model response to climate change in relation to uncertainty in GCM scenarios and downscaling methods (part II of pilot).

Future climate change scenarios for several future time periods from several GCM’s and several downscaled methods will be supplied for each of the locations to the individual modeling groups. Simulation results with these future scenarios will be used to divide the uncertainties in climate change impact simulations between crop models and GCM scenarios/downscaling methods.

All the simulations by the individual modeling groups will be done, starting with wheat in May 2011, via email over the next 10 months while aiming for publications (with each full-model simulation contribution acknowledged through a co-authorship) on uncertainties in projecting climate change impact. Most of the papers focus will be on “uncertainty” per se, but there will be an attempt, involving all co-authors, to contrast the simulated responses of CO2, temperature, rainfall and N etc., to published metadata.

Schedule and Deadlines:

The simulation experiments carried out by the individual modeling groups need to be finalized by early 2012 and prepared for scientific publications by mid 2012. S. Asseng will send out the four data sets to the individual modelers and will be the receiving end for all individual simulation results following a detailed schedule below. He will pass on the simulation results on grain protein% to P. Martre and for the climate change scenarios to Alex Ruane for separate analysis.

  1. late-May 2011: modelers will start receiving limited information data sets (including daily weather data of experiments, soil characteristics, initial soil conditions, crop management, anthesis and maturity date) and a 30-year baseline climate record and one single GCM-generated scenario (30 years daily weather data) for each of the 4 locations.
  2. mid-September 2011 (limited-information scenarios):
    1. submit simulated grain yield and grain protein % for experiments of each locations (i.e. GY and GP% and standard set of other outputs (TBA) for each location),
    2. submit grain yield and grain protein % for 30 years of baseline (as under (a)) and for the single GCM generated 30 years climate scenario – not to be confused with the full climate change scenarios which will follow later in this pilot (described under pilot part II).
  3. end-September: receive detailed information for experiments at each location (including measured grain yields, grain protein%, yield components, biomass, LAI, crop N uptake dynamics, soil water and soil N dynamics if available) to assist with calibration of models to grain yield and grain protein%.
  4. end-November 2011 (after calibration with detailed experimental information):
    1. submit new grain yield and grain protein% for experiments of locations (and other outputs as before).
    2. submit results from simulation experiment (CO2 x T x rainfall x N plus extra treatment-combinations based on manipulated baseline climate data [30 years] for each location.
    3. receive multi-GCM based climate scenarios for each location (for pilot part II).
  5. end-February 2012: submit simulation results from climate scenarios (pilot part II)
  6. July 2012: submit scientific publications on:
    • grain yield uncertainty (Asseng + all involved crop modelers)
    • grain protein% uncertainty (Martre + all involved crop modelers)
    • climate scenarios (Ruane + all involved crop modelers)

Additional preparation tasks and requests:

  1. Decide on future time periods and assemble GCM and downscaling scenarios for individual locations for pilot part II.
  2. Submit individual name + affiliation of crop modelers (after a template has been supplied).
  3. Submit paragraph on individual model, including references for model description, testing with measured data and applications if applicable, plus full reference list (after a template has been supplied).

Appendix IV: .AgMIP Climate Data File Standard

AgMIP climate data and scenarios will be archived in the standard .AgMIP data format described in this appendix.

Naming standards: The naming convention for .AgMIP files indicates the time series’ station location, time period, the climate models/data used to generate the scenario, the type of downscaling approach applied, and the type of scenario represented. Following the DSSAT standard format, all climate files have 8-character names:

First 2 characters describe region

e.g. PA for Panama, FL for Florida, NL for Netherlands

Next 2 characters describe station

This could be coded by station name (e.g. HA for Haarweg; DE for Delhi) or by station number (for example 96 stations in Florida may each be described with first four digits of FL01, FL02, etc.)

Fifth character indicates time period and emissions scenario (additional values will be generated to indicate CMIP5 RCP simulations)

0 = 1980-2009 baseline

1 = A2 -2005-2035 (Near-term)

2 = B1 -2005-2035 (Near-term)

3 = A2 -2040-2069 (Mid-Century)

4 = B1 -2040-2069 (Mid-Century)

5 = A2 -2070-2099 (End-of-Century)

6 = B1 -2070-2099 (End-of-Century)

A = RCP8.5 -2005-2035 (Near-term)

B = RCP4.5 -2005-2035 (Near-term)

C = RCP8.5 -2040-2069 (Mid-Century)

D = RCP4.5 -2040-2069 (Mid-Century)

E = RCP8.5 -2070-2099 (End-of-Century)

F = RCP4.5 -2070-2099 (End-of-Century)

Sixth character is GCM/data source (here listed as CMIP3, subject to change for CMIP5):

X = no GCM used (observations)

0 = imposed values (sensitivity tests)

A = bccr

B = cccma cgcm3

C = cnrm

D = csiro

E = gfdl 2.0

F = gfdl 2.1

G = giss er

H = inmcm 3.0

I = ipsl cm4

J = miroc3 2 medres

K = miub echo g

L = mpi echam5 M = mri cgcm2

N = ncar ccsm3

O = ncar pcm1

P = ukmo hadcm3



Z = NCEP/DoE Reanalysis-2

Seventh character describes downscaling/scenario methodology

X = no additional downscaling

0 = imposed values (sensitivity tests)

1 = WRF

2 = RegCM3

3 = ecpc

4 = hrm3

5 = crcm

6 = mm5i

7 = RegCM4

A = GiST

B = MarkSIM

C = WM2

D = 1/8 degree BCSD

E = 1/2 degree BCSD

Eighth Digit is Type of Scenario:

X = Observations (no scenario)

A = Mean Change from GCM

B = Mean Change from RCM

C = Mean Change from GCM modified by RCM

D = Mean Temperature Changes Only

E = Mean Precipitation Changes Only

F = Mean and daily variability change for Tmax, Tmin, and P

G = P, Tmax and Tmin daily variability change only

H = Tmax and Tmin daily variability and mean change only

I = P daily variability and mean change pm;y

J = Tmax and Tmin daily variability change only

K = P daily variability change only

Additional values may also be introduced according to the participation/inclusion of new methods for scenario generation and requested scenarios.

Header: The first line in the .AgMIP file contains full information about the location and scenario represented. Lines 3-4 contain information about the location, including an abbreviated name, latitude and longitude (degrees), elevation (meters), average temperature and the amplitude of diurnal variability (ºC), and the reference instrument height for temperature and wind (meters).

Daily time series data:

Time series data contain the date as a concatenation of the year and Julian day, but also as individual columns for the year, month, and day of month.

Climate data include:

Solar radiation (MJ)

Maximum temperature (ºC; preferably measured at 2m)

Minimum temperature (ºC; preferably measured at 2m)

Precipitation (mm)

Wind speed (m/s; preferably measured at 2m)

Dewpoint temperature (ºC; preferably measured at 2m)

Vapor pressure (hPa)

Relative humidity corresponding to the time of day of maximum temperature (%). This approximates daily minimum relative humidity.

Note that missing data are assigned a value of -99.

.AgMIP file example:



Appendix V: AgMIP Sentinel Sites

Jonathan Hickman

Data for Model Evaluation/Improvement, Model Calibration, and Regional Climate Impact Assessment:

The full complement of data that can be used for model calibrations, simulations, and evaluations requires an intensive and time-consuming series of measurements, and is not readily available for many sites. To address the wide variation in data availability and quality, AgMIP is employing a tiered sentinel site approach, with tiers corresponding to the amount and quality of data available for model use (Figure 3).

Selected “platinum sentinel sites” with adequate data for complete and rigorous point model intercomparisions represent the top tier of data availability and quality. Platinum sentinel sites, which will be located in different agroecological zones, feature a wide breadth of data for crop model calibration, operation, and evaluation, including in-season measurements such as biomass accumulation and changes in soil and plant nutrient concentrations (Appendix II). In addition, efforts will be made to ensure that the minimum dataset for sentinel sites is of the highest possible quality (e.g., weather data based exclusively or primarily on observations, soil parameters based on direct measurements rather than pedo-transfer functions). Efforts will also be made to acquire complementary data which are not strictly necessary for assessment of crop growth and productivity, but which can be used for evaluation of model processes (e.g., in-season soil N mineralization rates). The identification of additional platinum sentinel sites within a country or region will be used for fine-tuning crop models for regional projections.

Figure 3. AgMIP Sentinel Site Pyramid.

The second tier, or “gold sentinel sites,” are sites that have the minimum data required for model calibration, operation, and evaluation, but that may lack the breadth of variables or depth of data available in sentinel sites, and may have some data that do not meet the quality standards of platinum sentinel sites. The lower data requirements for gold sentinel sites is intended to provide greater regional representation for crop model intercomparisons and a more representative foundation for regional projections. Though gold sentinel sites will rely more heavily on end of season data for calibration, regions should include some gold sentinel sites with in-season measurements for calibration of cultivars.

At the third tier (“silver sentinel sites”) are sites where adequate data is available for model operation, but not for calibration or evaluation, (Appendix II, “Crop Model Input Data”). These data may be obtained from a variety of sources, and frequently may not be based on direct observation. For example, daily weather data may be obtained from remote sensing, downscaling, or reanalysis products, and soil profiles may be built using the FAO Soil Map of the world, representative profiles from ISRIC-WISE database complemented with generic profile data, or a mixture of these data with some observed data.

Finally, regional data—including regional estimates of average management, sowing dates, flowering dates, maturity dates, and regional yield data—will be used to conduct regional projections. Simulations using spatially-paired soil and weather data from multiple locations in the region will be aggregated to develop regional projections.