Semi-Automated DIRSIG Scene Modeling from 3D LIDAR and Passive Imaging Sources

Research Team: Steve Lach (Ph.D. student) and John Kerekes.

Task Scope: This research is developing techniques for the fusion of 3D LIDAR data with passive imagery to semi-automate several of the required tasks in the DIRSIG scene creation process.

Task Status: This first year has focused on the extraction of the geometric terrain and surface objects necessary to describe the scene. In creating the terrain model, the irregularly-sampled LIDAR point cloud may be processed directly, or the data may first be interpolated onto a regular grid to form a range image. In either case, the fundamental concept is to separate the ground points from the non-ground points, then to remove the non-ground points. Recent literature describes several complex techniques that may be applied directly to the 3D point data, but it was found that satisfactory results could often be obtained easily by applying image processing techniques to the range image alone. Figure 4.3-1: Depiction of the Modified Median Filter. The median value of data in the smaller square region is computed. Data points in the larger square region more than a given threshold above this median value are then flagged as non-groundA simple approach that has been found to be effective in many cases is to use a variation of the spatial sliding window median filter in order to identify points that are significantly higher than their neighbors. Care must be taken to ensure that the kernel is large enough to span the roof structures of the largest buildings in the scene. If it is not, the central points of large objects may not be flagged as being non-ground. However, when the kernel is large, this technique may fail in regions where there is a low ratio of ground to non-ground points. These issues are avoided by computing the median value for a small region (typically 10m x 10m), and then identifying points in a larger region that are significantly higher than this median value. This modified median filter also has the advantage of being much more computationally efficient, and a similar technique may be employed directly on the point cloud, if desired. Figure 4.3-1 illustrates this filtering concept for a single location of the sliding windows while Figure 4.3-2 depicts the progression of the entire process.

Figure 4.3-2: Original scene (upper left), points identified for removal (upper right), points identified for scene after removal of points (bottom left), interpolated and smoothed Digital Terrain Model - DTM (bottom right)

Through the use of the modified median filter, a set of points were extracted that were significantly higher than their neighbors (the initial set of non-ground points). In many cases, it may be assumed that these points represent buildings and trees, and the next logical task is to identify to which class each of these points belongs. The spectral reflectance of plants has a distinct signature, and therefore multispectral and hyperspectral classification techniques excel at differentiating vegetation from most manmade objects. In general, use of spectral information is the most robust method that we considered for separating building and tree regions, although in many images the required spectral information may not be available.

In the event that spectral information is not available, spatial-based approaches may also be used. A range-image based morphological technique has been successfully implemented in which each region was opened by a specific amount. By sizing the structuring element to preserve just a few pixels from the smallest building in the scene, tree regions were effectively removed. This held true even for large areas containing many trees, as several ground pixels are present in almost all tree canopy regions.

Objects that contained pixels in common with the opened image were specified as being buildings, and the remaining objects were classified as trees.

Once the building and tree points have been categorized it is necessary to determine the geometrical properties of each object and represent them as a set of facetized models.

For the autonomous geometric reconstruction of trees, high resolution aerial imagery is blurred in order to emphasize the lower-frequency features of the trees in the image. By then performing correlations with different sized circle functions, regions with high radial symmetry were identified. This process is augmented by also using the height information provided by the LIDAR data. Once tree sizes and locations are determined, a specific tree model is selected from a library of objects previously created with the Tree Professional software.

For buildings, a different process is pursued. It is assumed that buildings are composed of many smaller geometric entities, and by modeling the structure of these entities, the overall building form may be achieved. Each building is first separated into multiple layers based on height, and each layer is then further grouped into spatial clusters. The outer wall of each cluster is found using a technique known as alpha shapes, while interior ridgelines are found as intersections of the dominant planes in each cluster.

These techniques have been applied to a portion of the RIT campus in order to produce a small scale scene consisting of varied terrain, trees, roads, buildings, and other common features. Using the algorithms described above, the terrain was successfully modeled, and all buildings and most of the individual trees were found. Additionally, two building were geometrically reconstructed with an accuracy that is satisfactory for many DIRSIG applications. Figure 4.3-3 provides an example result for one of the buildings.

Figure 4.3-3: CAD model of building successfully extracted from semi-automated processing of the 3D LIDAR imagery