The outputs of OVERSEER, as with every model, involve a level of uncertainty. OVERSEER was designed to model easily obtainable data which means the model uses simplifications of complex processes. This results in a level of uncertainty in the modelled estimates.
Obtaining actual measurements of leaching carries with it a level of uncertainty, and generating an accepted value for farm-scale nutrient loss is considered extremely challenging1.
There are multiple sources of uncertainty in OVERSEER – they can be generally grouped into measured or modelled uncertainties.
Measured data contains variability in space and time inherent in all biological systems, such as the differences in soil moisture measured across a paddock. This uncertainty relates to data that comes from a measured source bringing uncertainty with it.
Modelling uncertainty includes all the variability associated with data entry as well as the uncertainty within the modelling procedures including:
The diagram below provides a conceptual example of the relationship between measured and modelled uncertainty.
Quantifying and accounting for sources of uncertainty in models is particularly challenging, especially for a model describing complex farm systems like OVERSEER. A report by Ledgard and Waller (2001)2 estimated uncertainty of 25-30% for model predictions for N, which has since been widely quoted. However, this estimate didn’t include errors associated with measurements, or uncertainty from data inputs, providing only part of the full picture of quantifying uncertainty, and is therefore limited.
Since 2001, the OVERSEER model has had a number of changes, which likely result in a different uncertainty estimate for the current model version. An updated uncertainty analysis would be useful; however, quantifying all sources of uncertainty involved in the N leaching value produced by OVERSEER is impossible. Therefore, reducing uncertainty is more appropriate, for which there are several options.
All models should be calibrated (adjusted to match measured values) to confirm that the model is delivering a known outcome for specific conditions. To provide further confidence in the model, results should also be validated (compared to an independent data set)3. Together calibration and validation techniques are known as evaluation. The diagram below demonstrates how these processes work.
OVERSEER is a complex model with multiple sub-models generating different results. The pastoral N leaching model has been continually validated and calibrated as it has been developed, whereas the P loss model is based on a single calibration process.
Because of its wide coverage, the OVERSEER model is not evaluated for all conditions or types of systems in New Zealand. The reason for this is that there isnt data available to do this. It is unrealistic to get data for all of the varied farm systems and conditions in New Zealand. However, continuing to source data from a greater range of systems and conditions is important for ongoing validation and calibration of OVERSEER to increase confidence in model outputs and reduce their modelling uncertainty.
Uncertainty around model estimates tends to be lower when they reflect conditions similar to those in the calibration data set (where we have the most information). The model is capable of extrapolating outside of these conditions; however, the uncertainty is greater because there is less available information supporting the modelled estimate. The relative difference in prediction uncertainty within and outside the calibration conditions is described in the diagram below. An example is annual rainfall, whereby N leaching estimates under high rainfall (>1500 mm per year) conditions have far greater uncertainty, due to lack of measured data from these situations, than those under moderate rainfall.
Most of the calibration data used to date is from flat, pastoral, dairy enterprises, with free-draining soils and moderate rainfall. Therefore, to strengthen the calibration dataset and to reduce uncertainty in model results, datasets from outside these conditions are required e.g. cropping, beef and sheep enterprises, clay and shallow soil types, rainfall zones > 1200 mm.