CALL FOR PARTICIPATION
SHREC'06 - 3D Shape Retrieval Contest


*Introduction*
The Network of Excellence AIM@SHAPE is taking the initiative to organize a 3D shape retrieval evaluation event: SHREC - 3D Shape Retrieval Contest. The general objective is to evaluate the effectiveness of 3D-shape retrieval algorithms. The contest is organized in conjunction with the SMI06 conference (Shape Modeling International) where the evaluation results will be presented. Because of the short term, this year's contest will consist of a single track. For subsequent years it is our intention to organize a multi-track contest.

*Task description*
The task set for the 3D shape retrieval contest is to retrieve a ranked list of similar 3D objects, given a query 3D object, based on the shape characteristics of the 3D objects (no other aspects such as color). The collection to search in consists of 3D polygonal models that are 'polygon soups', i.e., not necessarily watertight.

*Models description*
The collection to search in consist of all the classified models from the Princeton Shape Benchmark Version 1, both the models from the training set and the test set (1814 models), extended with extra 3D polygonal models from the AIM@SHAPE Shape Repository (say N models). The total collection of 1814 models will be distributed via the AIM@SHAPE website as a set of renamed files in a single directory, such that the classification of the objects cannot be derived from the path and filename. Each of the participants must provide 5 candidate polygonal query models to the organization. These models should not already be in the Princeton Shape Benchmark, but should fit in its classification. Participation in the contest
without providing these models is not allowed. From the total set of submitted query objects, the organization selects a number of models for performing the evaluation (for example 30 queries).

*Evaluation Methodology*
The retrieval result must be a ranked list, which will be assessed by a number of performance measures, such as precision and recall, cumulative gain, etc. The relevance assessments will be done automatically according to the base classification of the Princeton Shape Benchmark. We will employ a ternary relevance scale: highly relevant (2), marginally relevant (1), not relevant (0). Together with the ranked retrieval results, the participants must submit executables of their programs for verification (the executables will not be further distributed and are used for verification of the contest results only.)

*Procedure*
The following list is a step-by-step description of the activities:
1) Distribution of the collection, plus example queries, via
http://www.aimatshape.net/event/SHREC/
2) The participants must register by sending a message to shrec@cs.uu.nl.
Early registration is encouraged, so that we get an impression of the
number of participants at an early stage. Each participant must submit
5 candidate queries in the form of 3D polygonal models.
3) The organization will make a selection of objects, which will form
the query set (for example 30 queries), against which the evaluation will
be run.
4) The query set is released at http://www.aimatshape.net/event/SHREC/.
Within 48 hours after the release of the query set, each participant has to submit the retrieval results of all queries as a ranked list per query. Up to 5 ranked lists may be submitted, resulting from different runs. Each run may be a different algorithm, or a different parameter setting. In addition, an executable of the program must be submitted for verification by the organization.
5) The relevance assessments will be done automatically according to the base classification of the Princeton Shape Benchmark on a ternary relevance scale: highly relevant (2), marginally relevant (1), not relevant (0).
6) The organization will release the relevance judgments and evaluation scores of all the runs.
7) The participants write a short paper describing their method and commenting the evaluation results.
8) The description of the methods and the evaluation scores are presented at SMI06, and made available in the contest proceedings.

*Schedule*
March 10 - Call for participation.
March 24 - Distribution of the test collection, plus example queries.
April 7 - Deadline for registration and candidate query submission.
May 2 - Distribution of query set.
May 4 - Result submission and draft short paper about the methods.
May 26 - Distribution of relevance judgments and evaluation scores.
June 2 - Submission of final short papers for the contest proceedings.
June 14-16 - Results presented at a session of SMI06, proceedings available.

*Organization*
The SHREC contest is an initiative of the Network of Excellence AIM@SHAPE, http://www.aimatshape.net/. For information about the contest, the results, etc. see http://www.aimatshape.net/event/SHREC/. To contact the organizers, send a message to shrec@cs.uu.nl. The 'local' organizers are Frank ter Haar, Remco Ruijsenaars, Roelof van Zwol, Remco Veltkamp.
The evaluation collection is based on the Princeton Shape Benchmark, http://shape.cs.princeton.edu/benchmark/, extended with models from the AIM@SHAPE Shape Repository, http://shapes.aim-at-shape.net/.
The results are presented at a special session of SMI06, Shape Modeling International, http://www.ifs.tohoku.ac.jp/SMI06/.