Comparative Performance of Multi-Source Reference Data to Assess the Accuracy of Classified Remotely Sensed Imagery: Example of Landsat 8 OLI Across Kigali City-Rwanda 2015

Accuracy assessment of remote sensed classified images is considered the backbone of remote sensing image processing to be considered credible. However, reference data to perform this task is also a considerable challenge for the remote sensing analyst. This study was carried out over Kigali city using Landsat remotely sensed imagery acquired on July 15, 2015, to compare multi-sourced reference data performance to assess the accuracy of classified Landsat remote sensed imagery. To achieve this objective, GeoEye-1, WorldView-2, Google earth high-resolution image, and GIS layers have been used to verify the accuracy of remote-sensed data classification.   In this study, we applied different reference data sources to Landsat 2015 classified images to assess the accuracy. Therefore, results from GEOEYE-1 image as reference data source displayed the total accuracy and kappa coefficient of 98.5% and 0.98 respectively. WorldView-2 MS Image revealed 97.25% of total accuracy and a 0.96 Kappa coefficient agreement. High-resolution rectified images generated using El-Shayal Smart GIS Editor also show its capabilities to assess the accuracy of Landsat remote sensed  data whose results were 94% and 0.92%, respectively, for overall accuracy and total Kappa statistics. Furthermore, the remote sensing analyst should not worry about where or how to find reference data to assess image classification so long as they possess GIS shape files. GIS shape files provide good results where the overall accuracy was 92% and a Kappa coefficient of 0.90. Moreover, GIS shape files results showed a slightly lower accuracy because of data properties; it is recommended to check projection before using any spatial data. This paper strongly focused on soft features during ground reference data collection. Test data from GEOEYE-1 images have shown the best thematic accuracy after being overlaid with Kigali 2015 thematic map. All of the referenced data sources, in general, showed the ability to assess remote sensed classified map in the range of 90% to 98.5% for both total accuracies of the map and kappa accuracy.

Felicien Nkomeje, "Comparative Performance of Multi-Source Reference Data to Assess the Accuracy of Classified Remotely Sensed Imagery: Example of Landsat 8 OLI Across Kigali City-Rwanda 2015" International Journal of Engineering Works, Vol. 4, Issue 1, PP. 10-20, January 2017. 

1. [1]   K. R. Lillesand TM, Chipman JW Remote Sensing and Image Interpretation, 5th ed., 2004.
2. [2]   G. M. Foody, "Harshness in image classification accuracy assessment," International Journal of Remote Sensing, 29, 3137-3158. , 2008.
3. [3]   S. A. ADEFIOYE, "Quantitative Image Classification Accuracy Assessment Program for Sustainable Geospatial Technology Applications.," 2014.
4. [4]   a. K. J. Mohd Hasmadi Ismail, "Satellite Data Classification Accuracy Assessment Based from reference Dataset," World Academy of Science, Engineering and Technology, International Journal of Environmental, Chemical, Ecological, Geological and Geophysical Engineering, vol. Vol:2, No:3, 2008.
5. [5]   C. R. J., "Sample size for testing differences in proportions for the paired-sample design," Biometrics. 1987 Mar;43(1):207-11., 1987.
6. [6]   G. Gutman, "Assessment of the NASA–USGS Global Land Survey (GLS) datasets," Remote Sensing of Environment 134:249–265 2013.
7. [7]   C. F. B. Gary M. Senseman, and Scott A. Tweddale, "Accuracy Assessment of the Discrete Classification of Remotely-Sensed Digital Data for Landcover Mapping," EN-95/04 1995.
8. [8]   K. G. Russell G. Congalton, Assessing the Accuracy of Remotely Sensed Data: Principles and Practices, 2nd ed., 2008.
9. [9]   J. v. Vliet, "Assessing the Accuracy of Changes  in Spatial Explicit Land Use Change Models " presented at the 12th AGILE International Conference on Geographic Information Science 2009 Leibniz Universität Hannover, Germany 2009.
10. [10] R. G. Congalton, "Accuracy assessment: a critical component of land cover mapping.," 1996.
11. [11] G. F.-L. Rosenfield, K., " A coefficient of agreement as a measure of thematic classification accuracy.," Photogrammetric Engineering and Remote Sensing, 1986.
12. [12] G. H. Rosenfield, "The analysis of areal data in thematic mapping experiments.," Photogrammetric Engineering and Remote Sensing, 1982.
13. [13] G. Foody, "On the Compensation for Chance Agreement in Image Classification Accuracy Assessment," Photogrammetric Engineering & Remote Sensing, vol. Vol. 58. No.10., 1992.
14. [14] P. S. Thenkabail, " Inter-sensor relationships between IKONOS and Landsat-7 ETM+ NDVI data in three ecoregions of Africa.," International Journal of Remote Sensing 25(2), 389-408. , 2004.
15. [15] Q. W. Assefa M. Melesse , Prasad S.Thenkabail  and Gabriel B. Senay "Remote Sensing Sensors and Applications in Environmental Resources Mapping and Modelling," Sensors 2007, 7, 3209-3241, 2007.
16. [16] R. L. Demin Xiong, J. Bo Saulsbury, Elizabeth L. Lanzer, Albert Perez, Remote sensing applications for environmental Analysis in transportation planning:  : Application to The Washington state i-405 corridor. WA-RD-593-1: Environmental Affairs Office, Washington State Department of Transportation, 2004.
17. [17] R. G. Congalton, "Thematic and Positional Accuracy Assessment of Digital Remotely Sensed Data," 2005.
18. [18] S. V. S. a. R. L. Czaplewski, "Design and Analysis for Thematic Map Accuracy Assessment: Fundamental Principles," REMOTE SENS. ENVIRON., vol. 64:331-344 (1998) 1998.
19. [19] E. J. O. Charles, "Is 80% accuracy Good Enough?," Michigan Tech Research Institute, Ann Arbor, Michigan 48105, 2008.
20. [20] E. E. H. JAMES R. ANDERSON, JOHN T. ROACH, and RICHARD E. WITMER "Land Use And Land Cover Classification System For Use With Remote Sensor Data," US Geological Survey Professional Paper No. 964. Washington, DC, 1976.
21. [21] C. H. Limin Yang, Collin G. Homer, Bruce K. Wylie and Michael J., "An approach for mapping large-area impervious surfaces:  Synergistic use of Landsat 7 ETM+ and high spatial resolution imagery " Canadian Journal of Remote Sensing, Coan Raytheon Corporation, U.S. Geological Survey EROS Data Center, Sioux Falls, SD 57198 USA, No date.
22. [22] G. M. Foody, "Sample size determination for image classification accuracy assessment and comparison," International Journal of Remote Sensing vol. Volume 30, 2009.
23. [23] G. M. Foody, "Sample Size Determination for Image Classification Accuracy Assessment and Comparison " Shanghai, P. R. China, June 25-27, 2008.
24. [24] R. Peacock, "Accuracy Assessment Of Supervised And Unsupervised Classification Using Landsat Imagery Of Little Rock, Arkansas," 2014.
25. [25] A. C. Curtis, "Thematic Accuracy Assessment Procedures, National Park Service Vegetation Inventory,," National Park Service, Natural Resource Program Center atural Resource Report NPS/NRPC/NRR—2010/204, 2010.
26. [26] G. H. a. F.-L. Rosenfield, K., "A coefficient of agreement as a measure of thematic classification accuracy," Photogrammetric Engineering and Remote Sensing, vol. 52, 1986.
27. [27] C. G. a. C. Jenkins, "Land cover mapping of Greater Mesoamerica using MODIS data," Can. J. Remote Sensing, vol. Vol. 31, No. 4, 2005.
28. [28] U. Yusuf. ( 2013, October 5, 2016). How to Download BIG Rectified Google Earth Imagery to get big High Resolution Image Available: https://www.youtube.com/watch?v=HlpCpi8plXw
29. [29] M. E. Elshayal, "Elshayal Smart GIS Map Editor and Surface Analysis," Programming of Geographic Information Systems (GIS)-Elshayal Smart GIS Map Editor, 2010.
30. [30] J. E. Nichol and M. S. Wong, "High resolution remote sensing of densely urbanised regions: a case study of Hong Kong," Sensors (Basel), vol. 9, pp. 4695-708, 2009.
31. [31] I. Mohd Hasmadi, Alias Mohd,S. and Norizah,K., "Reclassifying forest type to a new forest class based on vegetation and lithology characteristics using geographic information system at southern Johore, Malaysia," INTERNATIONAL JOURNAL of ENERGY and ENVIRONMENT, vol. Volume 2, 2008.
32. [32] V. Manirakiza, "Promoting inclusive approaches to address urbanisation challenges in Kigali," African Review of Economics and Finance, vol. Vol. 6, No. 1June 2014 pp. 161-180, 2014.
33. [33] REMA, "Kigali State of Environment and Outlook Report 2013," Rwanda Environment Management Authority, Kigali, Rwanda2013.
34. [34] P. Avino, D. Brocco, L. Lepore, M. V. Russo, and I. Ventrone, "Remote sensing measurements for evaluation of air quality in an urban area," Ann Chim, vol. 94, pp. 707-14, Sep-Oct 2004.
35. [35] R. ESRI, How to solve the data shift problem in Rwandan Projections. Kigali, Rwanda, 2012.
36. [36] W. F. a. B. Kraus, Working with projections and Datum Transformation in ArcGIS: Theory an practical examples. Points Verlag norden Halmstad: 360 gages, richly illustrated paper back 2005.
37. [37] J. Jensen, Introductory Digital Image Processing, 3rd. ed., 2005.
38. [38] P. S. Thenkabail, Land Resources Monitoring, Modeling, and Mapping with Remote Sensing, Vol.2. vol. 2, 2016.
39. [39] J. A. R. Ned Horning, Eleanor J. Sterling, Remote Sensing for Ecology and Conservation: A Handbook of Techniques, 2005.
40. [40] G. M. Foody, "Status of land cover classification accuracy assessment," Remote Sensing of Environment 80 (2002) 185–201, 2002.
41. [41] D. G. a. J. Montero, "Determining the accuracy in image supervised classification problems," vol. EUSFLAT-LFA, 2011.
42. [42] T. A. M. University. (2013, 15 May). Accuracy Assessment of an Image Classification in ArcMap Available: https://www.youtube.com/watch?v=FaZGAUS_Nlo
43. [43] R. G. Congalton, "A review of assessing the accuracy of classifications of remotely sensed Data," Remote Sensing of Environment, 1991.
44. [44] B. D. E. a. M. Glass, "The Kappa Statistic: A Second Look," Association for Computational Linguistics, vol. Volume 30, 2004.
45. [45] M. J. Warrens, "Properties of the quantity disagreement and the allocation disagreement," International Journal of Remote Sensing vol. Volume 36, 2015-Issue 5 2015.
46. [46] F. Bishop Y.M.M., S.E. and Holland, P.W.  , "Agreement as a special case of association. Discrete Multivariate analysis.," Cambridge MA, MIT press, 393 - 400., 1975.
47. [47] K. Bahadur K.C, "Improving Landsat and IRS Image Classification: Evaluation of Unsupervised and Supervised Classification through Band Ratios and DEM in a Mountainous Landscape in Nepal," Remote Sensing, vol. 1, pp. 1257-1272, 2009.
48. [48] R. G. B. Congalton, G.S. , "A pilot study evaluating ground reference data collection efforts for use in forest inventory. Photogrammetric Engineering and Remote Sensing.58(12): 1669-1671.," 1992.
49. [49] R. D. Tortora, "A Note on Sample Size Estimation for Multinomial Populations," The American Statistician, 32, 100-102., 1978.
50. [50] T. Yamane, "Statistics, An Introductory Analysis," 2nd Ed., New York: Harper and Row, 1967.
51. [51] J. R. Jensen, " Introductory Digital Image Processing " Prentice-Hall, Englewood Cliffs, NJ, 1986.
52. [52] W. G. Cochran, "Sampling techniques 3rd ed. New York: Wiley.," 1977.
53. [53] R. G. G. Congalton, K., "Assessing the accuracy of remotely sensed data: principles and practices," Boca Raton, FL: Lewis Publishers., 1999.
54. [54] B.-C. a. L. Tai, J. , "Sample size and power calculations for comparing two independent proportions in a ‘negative’ trial," Psychiatry Research, 80, 197-200. , 1998.
55. [55] G. D. Israel, "Glenn D. Israel2_Determining Sample Size," Fact Sheet PEOD-6, November 1992, Florida cooperative extension services, 1992.
56. [56] S. V. Stehman, "Comparison of Systematic and Random Sampling for Estimating the Accuracy of Maps Generated From Remotely-Sensed Data," Photogrammetric Engineering & Remote Sensing, vol. vol 58, No. 9, 1992.
57. [57] J. L. a. L. van Genderen, B.F. , "Testing land use map accuracy," Photogrammetric Engineering and Remote Sensing. 43: 1135-1137., 1977.