Personal tools
You are here: Home → Research → Mobility Exchange Reports → Minica Panchetti → Research Summary
Navigation
 
Document Actions

Minica Panchetti

NIRG AIM@SHAPE Short Mobility Report: between LSIS, Equipe IMS - CNRS UMR 6168 (Aix-en-Provence, France) and IMATI - CNR (Genova, Italia), May, 9th - June, 6th 2006.

Research Summary

Images
Figure 1. A possible scenario for filling hole process using images information. Figure 1. A possible scenario for filling hole process using images information.
Figure 2. On the left, the mechanical part mesh with a hole inside; on the right, one real image of the object. Figure 2. On the left, the mechanical part mesh with a hole inside; on the right, one real image of the object.
Figure 3. The reconstructed cloud of points using SFS methods. Figure 3. The reconstructed cloud of points using SFS methods.
Figure 4. The resulting mesh formed of the initial one and of the patch. Figure 4. The resulting mesh formed of the initial one and of the patch.
Figure 5. A grey-scale image of an object is the input of SFS algorithm Figure 5. A grey-scale image of an object is the input of SFS algorithm
Figure 6. Three examples of results of different SFS methods. Figure 6. Three examples of results of different SFS methods.
The Reverse Engineering process consists in creating a digital representation of a physical object. The resulting geometric model is often a triangle mesh built from a point cloud acquired with a scanner. According to both the scanning process and the object complexity, some areas may be inaccessible thus resulting in a lack of information and, as a consequence, in a set of undesired holes inside the reconstructed geometric model. For example, when scanning an historical edifice with a scanner lying on the floor, the roof may not be accessible thus resulting in an incomplete representation of the monument. This is simply not acceptable. In this project we plan to overcome the limits of the actual scanning process by using additional information coming from images that may be taken during, or after, the scanning operation.

Approach


One possible scenario for a filling hole process using images information could be as shown in the figure 1.

The first step should be the identification of the hole, both in the image and in the scanned object (here, the object is a mechanical foundry part, figure 2).  Then, it should be performed one Shape From Shading (SFS) algorithm to this portion corresponding to the hole.  SFS methods aim at retrieving the 3D coordinates of an object from one gray-scale image of this object.  These methods are based on the surface reflectance theories.  The output of SFS is a cloud of 3D points where the (x,y) correspond to the pixels coordinates and where the z coordinate is an elevation computed from the information of light intensity I(x,y).

The second step should be the merge of the initial mesh and the cloud of points: thus, the cloud of points should be first normalized and simplified (since the number of points is the same as the number of pixels in the image, the cloud is too dense), and then aligned with the initial mesh (figure 3., red and yellow points are the reconstructed points).

Then the cloud of points should either triangulate or used to constraint the deformation of a coarse patch done by a classical filling holes algorithm (figure 4).

The work performed at IMATI focused on the SFS state of the art and tests.

SFS tests


Giving a gray-scale image such as this one in figure 5. and knowing the reflectance law of the surface and the light source position, SFS methods have to solve a linear equation with three unknowns, when the surface is described in terms of the surface normal, and a nonlinear equation with two unknowns when the surface is described in terms of the surface gradient.  Hence, finding a unique solution is difficult and additional constraints are required. 

We have made tests with the three most representative methods of SFS with real images (figure 6).  The first method, the Falcone and Sacona [1] (figure 6.a.) method is the resolution of a partial equation, the second one, the Danial and Durou [2] (figure 6.b.) method uses optimization and the last one, the Tsai and Shah [3] (figure 6.c.) method approximates the image irradiance law.

Conclusion


As shown in the figure 6., SFS methods don't succeed in well retrieving the shape of a 3D object from an image.  Indeed, SFS methods try to compute 3D points from pixels without any further information but light properties, therefore, they are likely to enerate approximate 3D shapes.  However, SfS could be useful in the sense that we have correspondences between the 3D points of the hole contour.  The next step of my work will thus be trying to use this information as boundary conditions to drive a SFS process.  Anothe scenario could also be such as using intensity information in order to constraint a coarse mesh using a deformation mechanical model.

References


[1] M. Falcone, and M. Sagona, An Algorithm for the Global Solution of the Shape-from-Shading Model Proc. Ninth IEEE Int'l Conf. Image Analysis and Processing, vol.1, pp.596-603, Sept. 1997.
[2] P. Daniel, and J.-D. Durou, From Deterministic to Stochastic Methods for Shape From Shading Proc. Fourth Asian Conf. Computer Vision, pp. 187-192, Jan. 2000.
[3] P.-S. Tsai, and M. Shah, Shape from Shading Using Linear Approximation, Image and Vision Computing, vol 12, no. 8, pp. 487-498, Oct. 1994.
1 2 3
by Odd_Andersen — last modified 2006-06-14 12:18
« May 2012 »
Su Mo Tu We Th Fr Sa
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31