FOREST STAND EXTRACTION: WHICH OPTIMAL REMOTE SENSING DATA SOURCE(S)?

Clément Dechesne, Clément Mallet, Arnaud Le Bris, Valérie Gouet-Brunet

Univ. Paris-Est, LASTIG MATIS, IGN, ENSG, F-94160 Saint-Mande, France

ABSTRACT

It has been now widely assessed in the literature that bothmulti/hyperspectral optical images and 3D lidar point cloudsare necessary inputs for tree species based forest stand de-tection. Nevertheless, no comprehensive analysis of the gen-uine relevance of each data source has been performed so far:existing strategies are limited to a single spatial and spectralresolution. This paper investigates which is the optimal com-bination of geospatial optical images and lidar point clouds.A supervised semantic segmentation framework is fed withvarious sources (multispectral satellite and airborne images,hyperspectral airborne images, low, medium and high den-sity lidar point clouds), ablation cases are defined, and thediscrimination performance of several fusion schemes is as-sessed under a challenging mountainous area in France.

Index Terms— Hyperspectral, high density lidar, fusion,feature selection, supervised classification, regularization,forest stand delineation, tree species.

1. INTRODUCTION

The fusion of multimodal data has been widely investigated inthe remote sensing literature in the two decades [1, 2, 3]. Datafusion remains a major field of investigation with the spreadof satellite missions delivering complementary images in thespectral, spatial and temporal domains. Recent surveys dis-cuss the different levels of fusion (original measured domain,feature spaces and/or classification level). They underline thesuperior performance of the joint use of lidar and hyperspec-tral data for various classification challenges. While urbanareas are heavily documented, no genuine study has been car-ried out for forested environments so far.The study of forested areas from a remote sensing point ofview can be operated at different levels: pixel, object (mainlytrees) or stand. Forest stands are among the basic units forforest analysis. They can be defined in many different terms(specie, age, height, maturity). Most of the time in nationalforest inventories, for reliability purposes, each area is man-ually interpreted by human operators using Very High spatialResolution (VHR) geospatial images with an infra-red chan-nel [4].Airborne laser scanning (ALS) and optical images are bothwell adapted remote sensing data for automatic stand segmen-

tation. ALS provides information about the vertical distribu-tion of the trees (height, strata, penetration ratios), while opti-cal images are useful for the tree species discrimination. Sur-prisingly, only a few methods have addressed the forest standsegmentation issue. Most of the literature focus on the de-lineation of forest stands using only a low density lidar pointcloud (<5 pts/m2) and VHR multispectral images (4 chan-nels) [5, 6, 7]. While such data combination corresponds tocurrent operational constraints, the emergence of new sensorsraises the question of their relevance in such a context:

• Hyperspectral images have shown to be relevant forthe classification of tree species at the tree level in [8]:therefore it may be assessed how it can be beneficialfor forest stand segmentation. It may also be investi-gated whether a limited number of channels (e.g., 6,corresponding to superspectral devices) is sufficient.

• In [9, 10], the high sampling of trees with lidar devicesallows to derive specific structural features for the dis-crimination of tree species. With the arrival of newlidar devices (Single Photon Lidar, Geiger, multispec-tral linear systems), higher point densities (>10 pts/m2)should therefore be employed for enhanced classifica-tion results.

This paper will focus on the method proposed in [11]. Italready allows to accurately delineate forest stands, accordingto the tree species. The method has been positively evaluatedover various forest compositions and tree species. Here, weinvestigate the contribution of superior ALS density and spec-tral resolution of optical images. More specifically, the aim ofthis paper is to determine if the use of data with higher reso-lution (spatial for ALS, and spectral for optical images) has asignificant impact on the accuracy of forest stand delineation.

2. METHODS

The investigated method is composed of three main steps(Figure 1, more details in [11]). It mainly relies on a super-vised classification step. Data fusion is performed both at thefeature and decision levels. Here, the strategy for assessingthe relevance of the data sources consists in feeding the pro-cess with various feature sets.

Lidar data Optical images

Pixel-based features

Object-based features

Classification posterior probabilities

Final segmentation

Feature extraction

Object extraction

Classification

Regularization

Multispectral

Superspectral

Hyperspectral

LowMediumHigh

Pointdensity

Fig. 1. Method proposed for forest stand extraction, with thevarious remote sensing inputs.

First, 25 features are derived from the airborne lidar pointcloud, whatever the point density. They are described in [11].They are based both on 3D and intensity information, at threedifferent scales.Then, features are extracted from one optical image.

• For the hyperspectral image, for computational issues,the raw channels (416) are the sole features;

• For the superspectral image, 10 statistical features arecomputed for each of the 6 raw channels;

• For the multispectral image, the same statistical fea-tures are computed for the 4 spectral bands and for 3standard vegetation indices (NDVI, DVI and RVI).

All features are computed at the pixel level, at the spatialresolution of the optical image, for the subsequent energyminimization process. They are also generated at the ob-ject level (retrieved through superpixel computation [12] ofoptical image). Indeed, the features are more consistent atthis scale which improves the discrimination process [13].Then, a SFFS-based feature selection step is carried out fora subsequent classification of the vegetation type (mainly treespecies), using a Random Forest classifier. Training samplesare automatically extracted from the existing French nationalland-cover (LC) forest database [11]. Last, a smoothing reg-ularization process based on an energy minimization frame-work is carried out in order to obtain the final segments.

Given the different input data (3 optical images × 3 lidarpoint densities), 9 scenarii are investigated in order to assessthe contribution of these data sources. They correspond todistinct feature sets and cardinalities (see Table 1).

3. DATA

The input data for the proposed method are lidar point cloudsand optical images. Data specifications are provided below.Both sources were acquired simultaneously (October 2015),and are perfectly co-registered, following a well-establishedprocedure [14]. The area of interest is a 2.5km×2.5km moun-tainous area in the East of France (Vosges mountains, see Fig-ure 2). The vegetation is very varied due to the very hetero-geneous environmental conditions (altitude, climate, topog-raphy, soil). 5 tree species co-exist ( Deciduous oaks,Beech, Scots pine, Fir or Spruce, Douglas fir).

3.1. Lidar

The lidar data has been acquired with a LEICA ALS 60 sen-sor with a very high point density (∼ 40 points/m2). Withsuch a high density point cloud, it is possible to artificially de-rive lower density point clouds by sub-sampling. The initialdataset has therefore been reduced to a medium point density(∼20 pts/m2), and a low point density (∼4 pts/m2). The den-sity reduction is performed by exploring all the points. Foreach point, a random number is generated according to a uni-form distribution and the processed point is kept or removedfrom the final points cloud, according to the value of the gen-erated number.

3.2. Optical images

The optical images are all derived from the hyperspectral im-age, acquired with a HySpex device. It exhibits 716 spectralbands (160 in the VNIR (400-1000 nm) domain and 256 inthe SWIR (1000-2400 nm) one), with a spatial resolution of1.4 m. On the one hand, a simulation of a superspectral im-age with 6 spectral channels is derived: the channels are se-lected among the most relevant VNIR bands as a good com-promise for multiple classification problems (both urban andforested environment). Such image is targeted for the designof a new camera in IGN where a trade-off between a veryhigh spatial sampling and a suitable spectral resolution has tobe found. On the other hand, a standard multispectral image(red, green, blue, near infra-red bands simulating Pleiades’spectral configuration) is also generated from the hyperspec-tral image. With a 1.4 m resolution, it can be compared to a

ImagesLidar

Medium density Low density

Hyperspectral VNIR+SWIR HS+MD : 416+25 HS+LD : 416+25Hyperspectral VNIR HV+MD : 160+25 HS+LD : 160+25Superspectral S+LD : 60+25 S+HD : 60+25Multispectral M+LD : 70+25 M+LD : 70+25

Table 1. Investigated fusion scenarii, with associated codesand number of features fed into the segmentation framework.

VHR satellite image such as SPOT6/7 ones.

4. RESULTS

The data set was processed for the 8 configurations and re-sults were assessed before and after the regularization withstandard accuracy metrics : Intersection-over-Union (IoU), F-Score, Overall Accuracy (OA), and κ. Results can be seen inTables 2 and 3. Obtained accuracies are very similar whateverthe configuration. They are generally slightly better whenmedium points density Lidar is involved, especially for rawclassification. Considering spectral information, superspec-tral and hyperspectral VNIR data do not improve results com-pared to 4 band multispectral configuration. There are severalpossible explanations to this. First, spectral information isvery redundant in the VNIR domain and the problem onlyinvolves 5 classes. Thus, a 4 band configuration may be suf-ficient to sum useful information. Second, the image was ac-quired at the very end of summer and differences betweendeciduous and evergreen species appear even in RGB images.Thus, this period is not the best to assess the relevance ofenriched spectal data. On the opposite, accuracies increasewhen using hyperspectral VNIR+SWIR data, while no tex-ture information is derived from such data. Indeed, SWIRcontains useful non redundant information with VNIR. At theend, results can only be evaluated on areas supposed to bepure but only at 75%. Thus, obtained results are not suf-ficiently different to consider real differences from accura-cies results. Results for 4 configurations (M+MD, HS+MD,M+LD, HS+LD) are presented on Figure 2 before and afterregularization. Compared to the reference data, pure speciesare correctly retrieved. Differences between results can beobserved in mixed forest areas, but remain quite slight.

5. CONCLUSION

Several sensor configurations involving images and Lidarpoint clouds were simulated and compared for forest standextraction. At the end, all of them led to very similar results.However, it must be taken into account that the classificationproblem remained quite specific (small area, evaluation onlyon 5 pure species areas, images captured at a specific season).

κ OA mean FScore mean IoUS+LD 96.4 97.8 97.4 95.0S+MD 96.7 97.9 97.7 95.5M+LD 96.6 97.9 97.4 95.0M+MD 96.9 98.1 97.8 95.7HV+LD 96.7 97.9 97.5 95.1HV+MD 96.6 97.9 97.5 95.1HS+LD 97.2 98.3 97.7 95.6HS+MD 97.3 98.3 98.0 96.1

Table 2. Accuracies (%) for the raw classification results.

6. ACKNOWLEDGMENTS

The authors would like to thank ONERA for providing theHySpex sensor for the Vosges survey. This work was sup-ported by the French National Research Agency (ANR)through the HYEP project on Hyperspectral imagery forEnvironmental urban Planning (ANR-14-CE22-0016).

7. REFERENCES

[1] M. Dalla Mura, S. Prasad, F. Pacifici, P. Gamba,J. Chanussot, and J.A. Benediktsson, “Challenges andOpportunities of Multimodality and Data Fusion in Re-mote Sensing,” Proceedings of the IEEE, vol. 103, no.9, pp. 1585–1601, 2015.

[2] L. Gomez-Chova, D. Tuia, G. Moser, and G. Camps-Valls, “Multimodal classification of remote sensing im-ages: A review and future directions,” Proceedings ofthe IEEE, vol. 103, no. 9, pp. 1560–1584, 2015.

[3] M. Schmitt and X. X. Zhu, “Data fusion and remotesensing: An ever-growing relationship,” IEEE Geo-science and Remote Sensing Magazine, vol. 4, no. 4, pp.6–23, 2016.

[4] A. Kangas and M. Maltamo, Forest inventory: method-ology and applications, vol. 10, Springer Science &Business Media, 2006.

[5] D. Tiede, T. Blaschke, and M. Heurich, “Object-basedsemi automatic mapping of forest stands with laser scan-ner and multi-spectral data,” International Archives ofPhotogrammetry, Remote Sensing and Spatial Informa-tion Sciences, vol. 36(8/W2), pp. 328–333, 2004.

[6] O. Diedershagen, B. Koch, and H. Weinacker, “Auto-matic segmentation and characterisation of forest standparameters using airborne lidar data, multispectral andfogis data,” International Archives of Photogrammetry,Remote Sensing and Spatial Information Sciences, vol.36(8/W2), pp. 208–212, 2004.

[7] V. Leppänen, T. Tokola, M. Maltamo, L. Mehtätalo,T. Pusa, and J. Mustonen, “Automatic delineation offorest stands from lidar data,” International Archives ofthe Photogrammetry, Remote Sensing and Spatial Infor-mation Sciences 38(4/C1), pp. 5–8, 2008.

[8] M. Dalponte, H. O. Ørka, L. T. Ene, T. Gobakken, andE. Næsset, “Tree crown delineation and tree species

κ OA mean FScore mean IoUS+LD 98.0 98.7 98.5 97.1S+MD 98.1 98.8 98.6 97.2M+LD 98.2 98.9 98.7 97.4M+MD 98.0 98.7 98.6 97.2HV+LD 98.5 99.1 98.8 97.6HV+MD 97.9 98.7 98.5 97.1HS+LD 98.4 99.0 98.7 97.5HS+MD 98.4 99.0 98.8 97.6

Table 3. Accuracies (%) for the classification after regular-ization.

(a) Multispectral orthoimage(RGB channels).

(b) Multispectral orthoimage(CIR).

(c) Normalized Digital SurfaceModel (lidar)

(d) Forest land-cover vectordatabase (training and valida-tion).

(e) M+MD raw (f) HS+MD raw (g) M+MD final (h) HS+MD final

(i) M+LD raw (j) HS+LD raw (k) M+LD final (l) HS+LD final

Fig. 2. Overview of the area of interest (2.5 km×2.5 km) and examples of obtained results before and after regularization (colorcode: Deciduous oaks, Beech, Robinia, Fir or Spruce, Douglas fir)

classification in boreal forests using hyperspectral andals data,” Remote sensing of environment, vol. 140, pp.306–317, 2014.

[9] J. Li, B. Hu, and T. L. Noland, “Classification of treespecies based on structural features derived from highdensity lidar data,” Agricultural and forest meteorology,vol. 171, pp. 104–114, 2013.

[10] R. Blomley, A. Hovi, M. Weinmann, S. Hinz, I. Korpela,and B. Jutzi, “Tree species classification using withincrown localization of waveform lidar attributes,” ISPRSJournal of Photogrammetry and Remote Sensing, vol.133, pp. 142 – 156, 2017.

[11] C. Dechesne, C. Mallet, A. Le Bris, and V. Gouet-Brunet, “Semantic segmentation of forest stands of purespecies combining airborne lidar data and very high res-olution multispectral imagery,” ISPRS Journal of Pho-togrammetry and Remote Sensing, vol. 126, pp. 129–145, 2017.

[12] P. Felzenszwalb and D. Huttenlocher, “Efficient graph-based image segmentation,” International Journal ofComputer Vision, vol. 59, no. 2, pp. 167–181, 2004.

[13] C. Dechesne, C. Mallet, A. Le Bris, V. Gouet, andA. Hervieu, “Forest stand segmentation using airbornelidar data and very high resolution multispectral im-agery,” International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences, vol.XLI-B3, pp. 207–214, 2016.

[14] K. Adeline, A. Le Bris, F. Coubard, X. Briottet, andN. Paparoditis, “Description de la campagne aéroportéeUMBRA : étude de l’impact anthropique sur les écosys-tèmes urbains and naturels avec des images THR mul-tispectrales and hyperspectrales,” Revue Française dePhotogrammétrie and de Télédétection, vol. 202, pp.79–92, 2014.