METRICS FOR IMAGE SURFACE APPROXIMATION BASED ON TRIANGULAR MESHES

Surface approximation plays an important role in several application ﬁelds, such as computer-aided design, computer graphics, remote sensing, computer vision, robotics, architecture, and manufacturing. A common problem present in these areas is to develop efﬁcient methods for generating, processing, analyzing, and visualizing large amount of 3D data. Triangular meshes constitute a ﬂexible representation of sampled points that are not regularly distributed in space, such that the model can be adaptively adjusted to the data density. The choice of metrics for building the triangular meshes is crucial to produce high quality models. This paper proposes and evaluates different measures to incrementally reﬁne a Delaunay triangular mesh for image surface approximation until either a certain accuracy is obtained or a maximum number of iterations is achieved. Experiments on several data sets are performed to compare the quality of the resulting meshes.


INTRODUCTION
Advances in techniques and equipments for image acquisition have allowed the extraction of large data volumes.Several knowledge domains require the construction of surface models with high level of details, such as computer vision, digital entertainment, cartography, computer graphics, finite element methods, scientific visualization, manufacturing, architecture, remote sensing, and virtual reality.
Despite the reduction of costs with storage devices, the availability of images at higher resolutions demands the development of efficient mechanisms for storing, manipulating, transmitting and visualizing data.
A digital image is commonly represented by a two-dimensional matrix that stores intensity values sampled at a discrete set of points.The matrix grid must have a resolution sufficiently small to capture the details necessary for the application under consideration.In order to provide a variableresolution approximation of digital images, some hierarchical representations have been developed in the literature, such as quadtrees (Sullivan and Baker, 1994;Scholefield and Dragotti, 2014) and pyramids (Burt and Adelson, 1983;Tran et al., 1987).
In contrast to the use of regular grid representations, where a set of sampled data points is stored at uniform intervals, polygonal meshes (Araújo et al., 2015;Bommes et al., 2013;Botsch et al., 2010;Maglo et al., 2012) can be more adaptive to irregularity of the data.In particular, triangular meshes are a common structure for modeling surface as a network of adjacent triangles, whose vertices represent the data samples.Important features, such as corners or edges, can be directly incorporated into the model.Furthermore, the triangular structure can be adjusted to fit variable density of data points, avoiding redundancy, In fact, triangular meshes can be used to compress (Maglo et al., 2012) data, while preserving relevant surface characteristics.Such data reduction improves storage requirements, transmission capacities, and rendering performance.
Several methods for constructing triangular meshes from dense data sets have been proposed in the literature (Lipman, 2012;Ma et al., 2013;Moraes et al., 2017;Nivoliers et al., 2012).Most surface approximation approaches can be classified as refinement or decimation methods.Refinement methods (Boissonnat et al., 2009;Brown, 1997;Hu et al., 2009;Dey and Ray, 2010) start with a simple initial approximation of the surface and incrementally add new data points to the triangulation until a given approximation criterion is achieved, such as desired accuracy or maximum number of iterations.On the other hand, decimation methods (Cignoni et al., 1998;Shaffer and Garland, 2001;Luebke et al., 2002;Salinas et al., 2015) start with a triangulation containing the entire set of sampled data points and iteratively simplify it until a specified approximation criterion is satisfied.
The vertex selection metric used in the refinement or decimation process is crucial to produce compact and precise triangular meshes, since it determines the degree of fidelity between the original data and the approximated model.Therefore, adequate measures for determining the relevance of a point must be carefully designed and evaluated.
The main contribution of this paper is to propose and evaluate different metrics for inserting new points to an initial triangulation with accuracy within a specified error tolerance or maximum number of points.More specifically, we extend upon the set of mesh refinement metrics developed in a previous work (van Kaick and Pedrini, 2006), such that five new criteria for image surface approximation are defined and analyzed in this work.Several data sets are used to assess the effectiveness of the metrics.
The paper is organized as follows.Section 2 briefly describes relevant concepts and works related to surface modeling and image approximation.Section 3 presents the different metrics used to approximate a set of sampled data points through triangular meshes.In Section 4, experiments are conducted on several data sets to assess the effectiveness of the evaluated metrics.Section 5 concludes the paper with some final remarks and directions for future research.

BACKGROUND
Different spatial data structures can be employed to model surface from a set of points, however, they have critical effects in terms of storage requirements, degree of accuracy, retrieval flexibility, scalability, among other factors.Examples of spatial data structures (Samet, 1990;2006), illustrated in Fig. 1, include a grid representation of regularly distributed data points, a triangular mesh, a quadtree and a bintree.
Several surface representations have been proposed in the literature (Araújo et al., 2015;Bommes et al., 2013;Botsch et al., 2010;Maglo et al., 2012;Mostafavian and Adams, 2015;Berger et al., 2013;Garland and Heckbert, 1997), however, polygonal models are probably the most common choice for representing sampled data sets since they are supported by the vast majority of tools for modeling and rendering purposes.
A polygonal surface is a piecewise-linear surface defined by a set of polygons, typically a set of triangles.In addition to their simplicity and flexibility, polygonal models can represent surfaces at different levels of detail (Hu et al., 2009;Luebke et al., 2002;Morigi and Rucci, 2013) and accuracy, such that a coarse representation can be used to describe less relevant areas, while high resolution can be focused on specific parts of interest.Furthermore, discrete definitions of differential operators have been proposed (Meyer et al., 2003), such as curvature measure (Kim et al., 2002) and geodesic curves (Polthier and Schmies, 2006).Techniques for constructing triangle meshes can be generally categorized into refinement and decimation.In the refinement methods (Boissonnat et al., 2009;Brown, 1997;Hu et al., 2009;Dey and Ray, 2010), a new data point is iteratively inserted into the mesh, typically the point that presents the largest approximation error according to a specific metric.In the decimation methods (Cignoni et al., 1998;Shaffer and Garland, 2001;Luebke et al., 2002;Salinas et al., 2015), the point with the smallest approximation error is iteratively removed from the current triangulation.In both approaches, local adjustments must be performed after each point operation in order to rebuilt the mesh.The process of refinement or decimation finishes when a required number of points is satisfied or an error tolerance is reached.
Two common strategies for generating and maintaining triangles meshes for image surface approximation purpose are Delaunay triangulation and data-dependent triangulation.
Delaunay triangulation (de Berg et al., 2008;Boissonnat et al., 2015;Cheng et al., 2012) guarantees certain geometric properties.The circumcircle of any triangle in a Delaunay triangulation contains no other data points in its interior.Another interesting property is that the Delaunay triangulation of a set of points is dual to its Voronoi diagram (de Berg et al., 2000;Preparata and Shamos, 1985).The Delaunay triangulation also maximizes the minimum angle of all triangles of the mesh, reducing the occurrence of thin triangles that tend to cause undesirable effects, such as numerical instability and visual artifacts (Scarlatos, 1992;Rippa, 1992a).Based on the property of maximizing the minimum angle, Lawson (1977) proposed a local optimization procedure to swap the diagonals of a strictly convex quadrilateral to produce a most equiangular triangulation.An important consequence is that, after a finite number of swaps, the successive application of this procedure to all internal edges of an arbitrary triangulation will produce a Delaunay triangulation.
Data-dependent triangulation (Brown, 1991;Dyn et al., 1990;Quak and Schumaker, 1990) maintains the mesh topology based on information of the input points to approximate the image surface.Although data-dependent triangulation methods can generate compact meshes, long and thin triangles are usually produced, which can lead to certain disadvantages for rendering purpose (Kolingerová et al., 2010;Scarlatos, 1992;Rippa, 1992a).However, such approaches can be suitable for the reconstruction of features, such as edges in images, where a function has high secondorder derivatives in one direction when compared to other directions (Rippa, 1992b;Tóth et al., 2007;Wang et al., 2001).
In this work, we used the Delaunay triangulation to incrementally approximate image surfaces due to its efficient operations to maintain a valid topology after each vertex insertion, as well as visual quality of the generated meshes.
The development of compact representation schemes for approximating image surface plays an important role in the reduction of storage and transmission costs.Associated with this subject, a large number of lossless and lossy image compression techniques have been proposed in the literature (Held and Marshall, 1996;Rabbani and Jones, 1991), such as transform coding (DeVore et al., 1992;Skodras et al., 2001), deflation (Rigler et al., 2007;Xie et al., 2008), chain codes (Ren et al., 2002;Sánchez-Cruz et al., 2007), chroma subsampling (Chen et al., 2009;Lin et al., 2013), fractal compression (Barnsley and Hurd, 1993;Fisher, 2012), among others.
In the context of image surface approximation, several approaches have been developed to compress triangular meshes (Peng et al., 2005) in order to reduce the size of the resulting models.These methods usually explore two types of information, geometry and topology.
Geometry compression methods (Chow, 1997;Van Aerschot et al., 2009;Khodakovsky et al., 2000) aim to reduce the numerical information associated with the mesh vertices, such as position, intensity, normal vectors, and texture.Deering (1995) introduced the concept of geometry compression, where vertex positions, normals and colors were quantized to less than 16 bits through delta compression and modified Huffman encoding.Taubin and Rossignac (1998) developed a geometry compression technique where vertex positions were quantized through a spanning tree, achieving 12 bits per vertex.
Topology compression methods aim to reduce the connectivity information which associates vertices of the mesh with edges and faces.Hoppe (1996) described a technique for transferring a mesh progressively, from a coarse approximation to a refined sequence of new vertices.Each vertex is transferred once, where 5 bits were used to identify two adjacent vertices.Other progressive compression methods were developed by Alliez and Desbrun (2001); Cohen-Or et al. (1999).Rossignac (1999) developed a method for compressing 3D triangular meshes using 1.3 and 2 bits per triangle.Decomposition of triangle meshes into triangle strips were explored to encode connectivity (Chow, 1997;Evans et al., 1996;van Kaick et al., 2004;Silva et al., 2002).Turán (1984) explored the mesh connectivity information as a triangulated planar graph encoded with less than 12n bits, in a triangulation with n vertices.Keeler and Westbrook (1995) improved Turan's results by encoding planar graphs with 4.6n bits through a triangle-spanning tree.

MATERIAL AND METHODS
The surface representation of an image can be generally viewed as a 2 1 2 -dimensional modeling problem (Pedrini, 2000), where a function of two variables, z = f (x, y), expresses the altitude z of the surface at a point (x, y).
In our approach, a triangular mesh is constructed from a set of irregularly spaced data points.A Delaunay triangulation is adopted due to the suitable geometric properties mentioned in the previous section.The triangulation of a set of data points in the plane can be defined in terms of a planar graph, where pairs of vertices are connected by edges intersected only at their endpoints, forming triangular faces.
A minimal approximation of the surface is initially constructed by consisting of two triangles over the data domain.This coarse mesh is repeatedly refined by the insertion of new points until either a specified error tolerance is achieved or a given number of points is reached.The mesh is updated after each point insertion in order to maintain the Delaunay triangulation.
The order in which the points are inserted into the triangulation is crucial to produce a compact and accurate approximation.A common strategy is to iteratively add the worst fitting point to the current triangulation according to the maximum absolute error (vertical difference) between the original and approximated models until all points are fit within a specified error tolerance.This process is straightforward to implement and generates compact triangulated meshes, however, it is sensitive to noise or outliers due to the selection of points with highest error.
In this work, we propose and evaluate new criteria for mesh refinement, extending upon the set of metrics described by van Kaick and Pedrini (2006).Each metric associates an error to a triangle ∆.Let f (p) and g(p) be the pixel intensity values at a point p(x, y) in the original and approximated images, respectively.It is worth mentioning that, at each step of the refinement algorithm, the point with the highest error in the triangle ∆ according to one of the evaluated metrics is inserted into the triangulation.
The following metrics for point insertion are evaluated in our method: maximum absolute error (MAE), absolute vertical sum (AVS), squared vertical sum (SVS), Laplacian maximum error (LMAE), Laplacian absolute vertical sum (LAVS), maximum absolute error weighted by the triangle area (MAEA), maximum absolute error weighted by the standard deviation of the errors (MAES), absolute vertical sum weighted by the standard deviation of the errors (AVSS), squared vertical sum weighted by the triangle area and standard deviation of the errors (SVSAS), Jaccard coefficient (JC), Czenakowski distance (CZD), and squared vertical sum weighted by the triangle area and the mean of the values in the original model (SVSAM).These metrics are defined in the next equations: , ( 11) where k(p) > 0 is the curvature at point p calculated with a Laplacian filter, σ (∆) is the standard deviation of the errors calculated in triangle ∆, A(∆) denotes the area of triangle ∆, and f (p) is the mean of the values f (p) in triangle ∆.Metrics MAEA, MAES, AVSS, SVSAS and SVSAM are proposed in this work.
Algorithm 1 shows the main steps of the mesh refinement process for a set of points chosen from an input image.Initially, a coarse triangular mesh is constructed with the four corners of the image (Lines 1-2).At each iteration, the Delaunay triangulation is refined by inserting the point with highest error among all triangles according to the metric under consideration, that is, one of the metrics defined in Equation 1 to 12.The intensity value in each triangle point is then approximated through an interpolation process, which is performed by computing the plane defined by the three vertices of the triangle and evaluating the plane equation at the point under consideration.This process is repeated until either a specified error tolerance is satisfied or a given number of points is achieved (Lines 3-7).The selected subset The evaluation of the resulting surface approximation is performed through two common image quality measures, the root mean squared error (RMSE) and peak signal to noise ratio (PSNR), expressed as: where M and N correspond to the image dimensions and L is the maximum intensity value of the image.

RESULTS
Experiments were conducted on several data sets to demonstrate the effectiveness of the evaluated metrics for mesh refinement.To allow a further quantitative and qualitative analysis of the results, Fig. 2 shows three tested images with their respective dimensions in pixels, as well as minimum and maximum intensity (elevation) values.The metrics AVS, SVS, LAVS, AVSS, SVSAS and SVSAM generated triangulations with a smaller number of points, fitting to the image features adequately for a fixed error rate.Other metrics, such as MAE, LMAE, MAES and JC concentrated points on specific portions of the image, particularly regions with high curvature, requiring more points.Some metrics (AVS, LAVS, MAEA, CZD and SVSAM) produced meshes with points more regularly distributed across the image, although MAEA and CZD did not generate compact triangulations.
In this work, the common criteria for mesh refinement based on the vertical error between the original and approximated models were modified in order to make them more adaptable to the data.Metrics weighted by triangle area favor the partitioning of large triangles into smaller ones, which can be more suitable for high curvature regions.On the other hand, metrics weighted by the standard deviation of the errors can prevent the selection of points based on local errors in the presence of noise (outliers).Fig. 6 shows the number of points required for each mesh refinement metric to achieve a fixed tolerance level (RMSE ou PSNR) for each tested  image.It is worth mentioning that the point selection criteria do not guarantee a monotonically convergence, however, the approximation errors show a decreasing behavior.From the results, it is possible to observe that new strategies for point selection can provide higher adaptability to the data when compared to the vertical error measure.
Triangulations and respective reconstructed images for the tested data sets with metric AVSS using only 1% of the original number of points are shown in Fig. 7. Despite the small number of points, the triangular meshes correspond to suitable approximations of the images, resulting in an efficient compression scheme.Error maps, computed as the difference between the intensity values of the original and approximated images, are also illustrated.
The proposed approach provides a good tradeoff between performance and flexibility to construct triangular meshes from different data sets.

DISCUSSION
In this work, we evaluated several metrics for refining triangular meshes constructed to approximate surfaces from a set of sampled points.The main benefit is the generation of more compact and accurate meshes, which can be applied to a number of different practical problems.
As demonstrated in our experiments, a substantial reduction in the size of triangular meshes can be achieved through the employment of more effective meshes, improving storage requirements, transmission capacities, and rendering performance.
Directions for future work include the proposition of new heuristics for mesh refinement and evaluation of features present in the data, such as corners and lines, to guide the triangulation construction process.

Figs. 3
Figs. 3, 4 and 5 show the triangular meshes constructed for approximations of the three images used in our experiments with the evaluated measures.All triangulations satisfy the same error level (RMSE = 15 or PSNR = 25), and the results show the spatial distribution of points generated by each metric.