Past Events: SHREC'09, SHREC'08, SHREC'07, SHREC'06

SHREC'10 - Shape Retrieval Contest 2010

Introduction

The general objective of the 3D Shape Retrieval Contest is to evaluate the effectiveness of 3D-shape retrieval algorithms. SHREC'10 is the fourth edition of the contest. Like last year, it  is organized in conjunction with the Eurographics Workshop on  3D Object Retrieval, where the evaluation results will be presented and printed in the corresponding workshop proceedings published by the Eurographics Book Series.

Thanks to the effort of previous track organizers, SHREC already provides many resources to compare and evaluate 3D retrieval methods. For this year's contest, we aim at new and updated tracks. Therefore, the participants are requested to have an active role in the organization of the event. This includes proposing track themes, building or acquiring a test collection, and deciding upon the queries, relevance assessment, and performance measures.

The participants of each track will collectively write a paper, which will be published in the proceedings of the Eurographics Workshop 3D Object Retrieval. At least one author per track must register for the workshop, and present the results.

Tracks

In the previous 3D Shape Retrieval Contests the task was to retrieve ranked lists of similar 3D objects, given a query 3D object, based on the shape characteristics of the 3D objects. The contest tracks organized in the past years covered different aspects and tasks of 3D shape retrieval, for example:

• Stability on watertight models
• Classification of watertight models
• CAD models
• Generic 3D models
• 3D face models
• Protein models
• Partial models
• Structural models
• Machine learning
  

Now, we solicit again proposals for tracks. You may opt for one of the above themes, or propose new ones. Track organizers should organize the following aspects:

• The task.
  One might ask for a complete or limited ranking, a classification, etc.
• The collection.
  Is the test set collected or generated. Are there copyright issues? Will it be made public? Is there a classification of the models?
• The queries.
  How are the queries determined? Are the query models from the collection or new models, or is it just a verbal description, etc.
  
• The ground truth.
  Is there a ground truth, is there a relevance scale (highly relevant, marginally relevant, ...), how and when is it determined (based on classification, visual inspection, ...), etc.
  
• The evaluation method.
  Which performance measure will be used for the evaluation (precision, recall, nearest neighbor, k-th tier, average dynamic recall, normalized cumulated gain, ...).
• The procedural aspects.
  Does every participant perform the queries, or is that done in a central place? Who does the performance assessment? When are test set and queries made available?
  

See http://www.aimatshape.net/event/SHREC/ for examples of organizational aspects of previous tracks.

Procedure

The following list is a step-by-step description of the activities:

1. Potential track organizers send their proposal to shrec@cs.uu.nl, describing the envisioned task, collection, queries, ground truth, evaluation method, and expected number of participants.
2. Promising tracks and their organizers are listed at the SHREC web page, the track organizers start working out the details.
   
3. Participants register to the tracks they want to participate in, by contacting the track organizers. 
4. Each track is performed according its own schedule.
5. The track organizers collect the results.
6. The track results are combined into a joint paper, published in the proceedings of the Eurographics Workshop on 3D Object Retrieval.
7. The description of the tracks and their results are presented at the Eurographics Workshop on 3D Object Retrieval (May 2, 2009).
   

Overall schedule

• January 15      - Submission deadline for track proposals
• Januray 17      - Notification of acceptence of track proposals
• Jan, Feb          - Each track has its own timeline
• February 21    - Each track is finished, and results are ready for inclusion in a track report
• March 7           - Camera ready track papers submitted for printing
• May 2              - EUROGRAPHICS Workshop on 3D Object Retrieval including SHREC'10

The individual tracks will have their own time schedule for registration of participants, release of queries or submission of executables, submission of results, etc.

Organization

The SHREC contest is supported by the FOCUS-K3D project. For information about the contest, the results, etc. see the SHREC home page. The results of SHREC'10 are presented at the Eurographics Workshop on 3D Object Retrieval, May 2, 2010, http://www-rech.telecom-lille1.eu/3dor/ and http://www.eurographics2010.se/coevents/


The SHREC organizers are Remco Veltkamp and Geert-Jan Giezeman. To contact the organizers, send a message to shrec"@"cs.uu.nl.