InfoVis 2003 Contest
Visualization and Pair Wise Comparison of Trees

News:   
The contest continues in 2004, see the InfoVis 2004 contest
All the entries prepared by the contest authors are now in the Information Visualization Repository

IEEE TVCG paper summarizes results and lessons learned from the 2003-2005 contest

September 5th 2003: We announced the results. We awarded three "1st place" and three "second place". All the entries prepared by the contest authors will remain available online after the conference in the Information Visualization Repository , and the community will then be invited to submit additional entries.

We thank , , and Stephen North for providing the financial support necessary to purchase the contest prizes.

Contest Chairs

Jean-Daniel Fekete, INRIA, France
Catherine Plaisant, HCIL, University of Maryland

Send email to: Jean-Daniel.Fekete@inria.fr; plaisant@cs.umd.edu

Goals of the contest

Our goal in organizing the InfoVis 2003 contest is to:

- Initiate the development of benchmarks for information visualization,
- Establish a forum to promote evaluation methods and focus the evaluation community, and
- Create a new interesting event at the conference

For InfoVis 2003 we are inviting submissions of case studies of the use of information visualization for the analysis of tree structured data, in particular to look at differences between pairs of similar trees.

Several pairs of datasets will be provided in a simple XML format. The application domain of the information will be described, and some open-ended tasks will be provided to help you start the analysis process.

Schedule and deadlines

January: [done] Announcement in InfoVis 2003 Call for Participation, Rules of contest and final schedule available

February 1st: [done] Publication of datasets and tasks, followed by a question period (1 month). Any clarifications and updates will be posted on this website.

May 10th: End of the question period (was extended from March 1st)

August 1st: Deadline for submission of a 2-page PDF summary + video (mandatory) + accompanying information webpage.

September 5th: Results and team notification

At the conference (October 19-21 2003 in Seattle)

First place entries will receive a prize and will have a speaking slot to present their work during the contest session at the conference. The length of the oral presentation is still to be determined but is likely to be similar to a short paper.

Second place entries will be presented as contest interactive posters at the conference. All accepted entries will be posted after the conference on a permanent evaluation benchmark website. The archival site will include the submitted materials (2-page PDF summary, video, and accompanying information webpage ) as well as annotations and commentary by the judges.

We will award prizes for best entries in several categories (value of a few hundred dollars.) The judges will decide on the exact categories based on the material received - possibilities include "Best Overall", "Best Interaction", and "Most Original".

Rules

Everybody can participate (except the contest organizers and judges): Individual or teams, students (e.g. class project teams) or corporate teams

You can use an existing commercial product or a research prototype, and of course you can combine tools.

"Partial answers" are entirely acceptable but we encourage you to address all three datasets/problems. In other words, if your tool or approach only partially addresses the problem or handles only some of the datasets or tasks, it is still useful to hear about your work and what your tool can do! If you have decided to focus on one of the datasets, we strongly encourage you to show how your tool would display the other datasets. The quality of the written analysis of the strengths and weaknesses of your tool for given datasets and tasks will be a judging criteria. So don't be shy... and participate!

At least one author of accepted submissions has to attend the Symposium

Contest materials

Data format, datasets, background information and tasks (released Feb. 1st)

Submission Information

What to submit?

A 2-page PDF summary, a video and a webpage of accompanying information (plus a submission index page with team information)

The 2-page PDF summary should provide an overview of the tool and the analysis process, summarize the strengths and weaknesses of the approach, and provide at least one example screen shot. We recommend that you include the name of the tool(s) you used in the title of the summary. The 2-page summary should be formatted in two-column SIGGRAPH format as described the InfoVis 2003 Call for Participation . First and second place entries will have their 2-page summaries included as-is in the printed poster compendium.

The video should focus on the interactive features of the tool and how they help analyze the data. Videos should not exceed 15 minutes in length, 5 minutes is the recommended length. NOTE: Because interactivity is an important component and it is difficult to evaluate on paper we mandate the submission of video. On the other hand we do not wish this requirement to stop you from submitting a contest entry so if you anticipate having difficulties generating a digital video, please contact the contest chairs - at least a month from the deadline - to arrange alternative submission methods.

The webpage of additional information should be a single webpage providing information to complement the 2-page summary and video.
It should include detailed explanations and analysis results, additional annotated screen shots, information about the team and how you worked together (when appropriate), background on the origin of the tool, references and any other information you think is relevant to make your study more interesting in the context of the evaluation of information visualization. This webpage should be printable and the equivalent of about no more than 6 printed A4 pages (it can be less of course). Note that this single webpage can contain links to additional information (e.g. authors' personal webpages, pages about the tools used or high resolution screen shots) but judges will not be required to follow those links to other information while evaluating the case study submission.

How to submit?

All submissions should be sent by email to both chairs: Jean-Daniel.Fekete@inria.fr and plaisant@cs.umd.edu

1- Send an email indicating your intent to participate. This is not mandatory, but it will help us plan the judging period of this new event, so try to let us know by the end of June.

2- Prepare a submission index page with the following information:

- Title: this should be the same as the 2-page summary title, with a subtitle "Submission Index"
- Team members: for each member provide: Name, Affiliation, Address, Email, Phone (and indicate who is the main contact,)
- Tools used: indicate the name of the tools and the source of the tools (i.e. is it an "in-house" tool, a "third party" tool developed somewhere else than the lab or company the team members represent, or a "hybrid" (please describe.)

- a link to the 2-page summary,
- a link to the video,
- a link to the additional information webpage.

3- Package all your materials (the Index page, the 2-page Summary, video and additional webpage) in a compressed .zip or tar .gz file. We recommend that you try to unpack this file and check that the links still work at the different location, i.e. that the links are appropriately made, relative or absolute depending on what you wanted.

4- When you are ready, and by August 1st, upload your packaged materials (i.e. one zip or gz file) to ftp://ftp.cs.umd.edu/incoming/hcil/iv03contest/ .

You can simply open that ftp page with a web browser and drag and drop your zip file in that directory,
OR with a command line ftp you can say:

ftp ftp.cs.umd.edu
use name= anomynous and use your email address as password
cd incoming/hcil/iv03contest
and upload your compressed files (e.g. "bin" then "put myfile.zip")

Note that you will only be able to write in this directory but not read other files, e.g. you won't be able to see what files are there. You won't be able to overwrite the file you just uploaded, therefore you will need to add a version number if you upload a second version.

Please use descriptive filename in the following format : 1stauthor name, toolname, version (if needed), e.g.:

jones-treemap.zip
or
jones-treemap2.zip (if you decide to upload a second updated version before the deadline.

(and if everything fails: put the zip file on the web somewhere so we can download it!)

5- Finally, send email to us at Jean-Daniel.Fekete@inria.fr; plaisant@cs.umd.edu to tell us that you are done (just in case the ftp upload didn't quite work perfectly). Within a day or two of the deadline we will send a confirmation email that we received your submission.

Judging

Submissions will be reviewed by external judges with information visualization expertise, and application domain experts. Criteria will include: quality of the data analysis (what interesting insights you found in the data), appropriateness of the visual representation for the data and the tasks, usefulness of the interactivity, flexibility of the tool (how well the tool accommodated the different datasets), quality of the written case study (description of the strengths AND weaknesses of the tool used), novelty (whether the tool was off-the-shelf versus a new approach developed from scratch.)

Questions?

Contact: Jean-Daniel.Fekete@inria.fr; plaisant@cs.umd.edu

 

Return to the InfoVis 2003

Last updated: 8/1/2003