Here is a collection of texts centered on the evaluation of image retrieval systems. Highlights issues and challenges in evaluating image retrieval systems and describes various initiatives that provide researchers with the necessary evaluation resources.

The pervasive creation and consumption of content, especially visual content, is ingrained into our modern world. We’re constantly consuming visual media content, in printed form and in digital form, in work and in leisure pursuits. Like our cave– man forefathers, we use pictures to record things which are of importance to us as memory cues for the future, but nowadays we also use pictures and images to document processes; we use them in engineering, in art, in science, in medicine, in entertainment and we also use images in advertising. Moreover, when images are in digital format, either scanned from an analogue format or more often than not born digital, we can use the power of our computing and networking to exploit images to great effect. Most of the technical problems associated with creating, compressing, storing, transmitting, rendering and protecting image data are already solved. We use - cepted standards and have tremendous infrastructure and the only outstanding ch- lenges, apart from managing the scale issues associated with growth, are to do with locating images. That involves analysing them to determine their content, clas- fying them into related groupings, and searching for images. To overcome these challenges we currently rely on image metadata, the description of the images, - ther captured automatically at creation time or manually added afterwards.

Seven Years of Image Retrieval Evaluation.- Data Sets Created in ImageCLEF.- Creating Realistic Topics for Image Retrieval Evaluation.- Relevance Judgments for Image Retrieval Evaluation.- Performance Measures Used in Image Information Retrieval.- Fusion Techniques for Combining Textual and Visual Information Retrieval.- Track Reports.- Interactive Image Retrieval.- Photographic Image Retrieval.- The Wikipedia Image Retrieval Task.- The Robot Vision Task.- Object and Concept Recognition for Image Retrieval.- The Medical Image Classification Task.- The Medical Image Retrieval Task.- Participant reports.- Expansion and Re–ranking Approaches for Multimodal Image Retrieval using Text–based Methods.- Revisiting Sub–topic Retrieval in the ImageCLEF 2009 Photo Retrieval Task.- Knowledge Integration using Textual Information for Improving ImageCLEF Collections.- Leveraging Image, Text and Cross–media Similarities for Diversity–focused Multimedia Retrieval.- University of Amsterdam at the Visual Concept Detection and Annotation Tasks.- Intermedia Conceptual Indexing.- Conceptual Indexing Contribution to ImageCLEF Medical Retrieval Tasks.- Improving Early Precision in the ImageCLEF Medical Retrieval Task.- Lung Nodule Detection.- Medical Image Classification at Tel Aviv and Bar Ilan Universities.- Idiap on Medical Image Classification.- External views.- Press Association Images — Image Retrieval Challenges.- Image Retrieval in a Commercial Setting.- An Overview of Evaluation Campaigns in Multimedia Retrieval.

The pervasive creation and consumption of content, especially visual content, is ingrained into our modern world. We’re constantly consuming visual media content, in printed form and in digital form, in work and in leisure pursuits. Like our cave– man forefathers, we use pictures to record things which are of importance to us as memory cues for the future, but nowadays we also use pictures and images to document processes; we use them in engineering, in art, in science, in medicine, in entertainment and we also use images in advertising. Moreover, when images are in digital format, either scanned from an analogue format or more often than not born digital, we can use the power of our computing and networking to exploit images to great effect. Most of the technical problems associated with creating, compressing, storing, transmitting, rendering and protecting image data are already solved. We use - cepted standards and have tremendous infrastructure and the only outstanding ch- lenges, apart from managing the scale issues associated with growth, are to do with locating images. That involves analysing them to determine their content, clas- fying them into related groupings, and searching for images. To overcome these challenges we currently rely on image metadata, the description of the images, - ther captured automatically at creation time or manually added afterwards.