National Aeronautics and Space Administration

This paper presents a Stage 3 validation of the recently released Collection 6 NASA MCD64A1 500 m global burned area product. The product is validated by comparison with Landsat 8 Operational Land Imager (OLI) image pairs acquired 16 days apart that were visually interpreted. These independent reference data were selected using a stratified random sampling approach that allows for probability sampling of Landsat data in both time and in space. A total of 558 Landsat 8 OLI image pairs (1116 images), acquired between March 1, 2014 and March 19, 2015, were selected and used to validate the MCD64A1 product. The areal accuracy of the MCD64A1 product was characterized at the 30 m resolution of the Landsat independent reference data using standard accuracy metrics derived from global and from biome specific confusion matrices. Because a probability based Stage 3 sampling protocol was followed, unbiased estimators of the accuracy metrics and associated standard errors could be used. Globally, the MCD64A1 product had an estimated 40.2% commission error and 72.6% omission error; the prevalence of omission errors is reflected by a negative estimated bias of the mapped global area burned relative to the Landsat independent reference data (−54.1%). Globally, the standard errors of the accuracy metrics were less than 6%. The lowest errors were observed in the boreal forest biome (27.0% omission and 23.9% estimated commission errors) where burned areas tend to be large and distinct, and remain on the landscape for long periods, and the highest errors were in the Tropical Forest, Temperate Forest, and Mediterranean biomes (estimated > 90% omission error and >50% commission error). The product accuracy was also characterized at coarser scale using metrics derived from the regression between the proportion of coarse resolution grid cells detected as burned by MCD64A1 and the proportion mapped in the Landsat 8 interpreted maps. The errors of omission and commission observed at 30 m resolution compensate to a considerable extent at coarser resolution, as indicated by the coefficient of determination (r² > 0.70), slope (>0.79) and intercept (−0.0030) of the regression between the MCD64A1 product and the Landsat independent reference data in 3 km, 4 km, 5 km, and 6 km coarse resolution cells. The Boreal Forest, Desert and Xeric Shrublands, Temperate Savannah and Tropical Savannah biomes had higher r² and slopes closer to unity than the Temperate Forest, Mediterranean, and Tropical Forest biomes. The analysis of the deviations between the proportion of area burned mapped by the MCD64A1 product and by the independent reference data, performed using 3  km × 3  km and 6  km × 6  km coarse resolution cells, indicates that the large negative bias in global area burned is primarily due to the systematic underestimation of smaller burned areas in the MCD64A1 product.