Methods for rapid quality assessment for national-scale land surface change monitoring

Abstract

Providing rapid access to land surface change data and information is a goal of the U.S. Geological Survey. Through the Land Change Monitoring, Assessment, and Projection (LCMAP) initiative, we have initiated a monitoring capability that involves generating a suite of ten annual land cover and land surface change datasets across the United States at a 30-m spatial resolution. During the LCMAP automated production on a tile-by-tile basis, erroneous data can occasionally be generated due to hardware or software failure. While crucial to assure the quality of the data, rapid evaluation of results at the pixel level during production is a substantial challenge because of the massive data volumes. Traditionally, product quality relies on the validation after production, which is inefficient to reproduce the whole product when an error occurs. This paper presents a method for automatically evaluating LCMAP results during the production phase based on fourteen indices to quickly find and flag erroneous tiles in the LCMAP products. The methods involved two types of comparisons: comparing LCMAP values across the temporal record to measure internal consistency and calculating agreement with multiple intervals of the National Land Cover Database (NLCD) data to measure the consistency with existing products. We developed indices on a tile-by-tile basis in order to quickly find and flag potential erroneous tiles by comparing with surrounding tiles using local outlier factor analysis. The analysis integrates all indices into a local outlier score (LOS) to detect erroneous tiles distinct from neighbor tiles. Our analysis showed that the methods were sensitive to partially erroneous tiles in the simulated data with a LOS higher than 2. The rapid quality assessment methods also successfully identified erroneous tiles during the LCMAP production, in which land surface change results were not properly saved to the products. The LOS map and indices for rapid quality assessment also point to directions for further investigations. A map of all LOS values by tile for the published LCMAP shows all LOS values are below 2. We also investigated tiles with high LOS to ensure the distinction with neighboring tiles was reasonable. An index in this study shows the overall agreement between LCMAP and NLCD on a tile basis is above 71.5% and has an average at 89.1% across the 422 tiles in the conterminous U.S. The workflow is suitable for other studies with a large volume of image products.