Past Events: SHREC'06

SHREC'07 - Shape Retrieval Contest 2007

[pdf file]

Introduction

After the success of SHREC - 3D Shape Retrieval Contest 2006, the Network of Excellence AIM@SHAPE is taking the initiative to organize a sequel. The general objective is to evaluate the effectiveness of 3D-shape retrieval algorithms. The contest is organized in conjunction with the SMI'07 conference (Shape Modeling International), where the evaluation results will be presented. In contrast with the previous contest, this time the contest will involve multiple tracks. For the organization of the various tracks, the participants are requested to have an active role in the organization. This includes deciding upon track themes, building or acquiring a test collection and deciding upon the queries, relevance assessment, and performance measures.

Tracks

In the previous 3D Shape Retrieval Contest the task was to retrieve a ranked list of similar 3D objects, given a query 3D object, based on the shape characteristics of the 3D objects (no other aspects such as color). The Princeton Shape Benchmark was used to retrieve objects from. This collection consists 3D polygonal models that are 'polygon soups', i.e., without (pre)defined topology. This year, we solicit proposals for tracks. Possible tracks include (but are not limited to) the matching or retrieval involving:

• Scan data
• Polygon soups
• Water tight models
• Partial matching
• Molecules
• 3D faces
• CAD models
• Free form surfaces

Track organizers should organize the following aspects:

• The task.
  One might ask for a complete or limited ranking, a classification, etc.
• The collection.
  Is the test set collected or generated. Are there copyright issues? Will it be made public? Is there a classification of the models? The queries.
  How are the queries determined? Are the query models from the collection or new models, or is it just a verbal description, etc.
• The ground truth.
  Is there a ground truth, is there a relevance scale (highly relevant, marginally relevant, ...), how and when is it determined (based on classification, visual inspection, ...), etc.
• The evaluation method.
  Which performance measure will be used for the evaluation (precision, recall, nearest neighbor, k-th tier, average dynamic recall, normalized cumulated gain, etc.).
• The procedural aspects.
  Does every participant perform the queries, or is that done in a central place? Who does the performance assessment? When are test set and queries made available?

The following list is a step-by-step description of the activities:

1. Potential track organizers send their proposal to shrec@cs.uu.nl, describing the envisioned task, collection, queries, ground truth, evaluation method, and expected number of participants.
2. Promising tracks and their organizers are listed at the SHREC web page, the track organizers start working out the details.
3. Participants register to the tracks they want to participate in.
4. Each track is performed according its own schedule.
5. The track organizers collect the results.
6. The track results are combined into the SHREC'07 proceedings.
7. The description of the tracks and their results are presented in a SHREC session at SMI'07.

Schedule

• December 23 - Submission deadline for track proposals.
• May 1 - Each track is finished, and results are ready for inclusion into the proceedings.
• June 13-15 - Results presented at a session of SMI'07, proceedings available.

The individual tracks will have their own time schedule for registration of participants, release of queries or submission of executables, submission of results, etc.

Organization

The SHREC contest is an initiative of the Network of Excellence AIM@SHAPE. The SHREC'07 organizers are Frank ter Haar, Remco Veltkamp, and Tristan Whitmarsh. The results of SHREC'07 will be presented at a special session of SMI'07, Shape Modeling International, http://smi07.liris.cnrs.fr/

For registration, questions, etc., please contact shrec@cs.uu.nl