Metadata, the data associated with information resources, is expensive. How much metadata is useful and for what purpose? What is the role of metada in building better query engines and making it easier for people to find information? Is human effort essential to make metadata useful or is it possible to produce serviceable metadata algorithmically?

We need to look closely at our assumptions about metadata. Are the assumptions about the utility, or even the necessity, of metadata a legacy of years of library science and practice? Is our thinking about metadata colored by that theory (Svenonius, 2000) and practice (Gorman, 1989)?

To address these questions, Cornell University and Syracuse University have merged their expertise to be able to conduct a life-cycle model of evaluation. We have been studying metadata from its initial generation through to its use in accessing desired educational resources. With a group of real users of the STEM Digital Library, we are attempting to empirically address and answer the following questions:

1. How is metadata 'best generated'? Manually, automatically, or semi-automatically? Is the same method best for both browsing and querying?

2. How will metadata be utilized by other users to access the resources in the Digital Library, both in querying and browsing?

3. What is the relative contribution of individual metadata elements to users in accessing resources either by querying or by browsing?

We plan to develop notions of appropriate metadata that account for the nature of content, audience, cost, and use. We have designed studies that test these notions in a variety of contexts in order to establish the role that automatic methods can play in producing usable metadata. Since the principal costs of metadata come from the efforts of skilled professionals, our efforts include comparing human-generated metadata against that produced by machines.

This research project will provide empirical evidence on the significance of metadata, its relationship to the user and how it can best be created. The results will impact the core NSDL (National Science Digital Library) system and the organizations that provide collections for the library and ultimately define by searching experience of the user. Based on the collaboration between researchers who bring different but complementary perspectives to the topic of metadata, the results will make a substantive contribution to the digital library community's understanding of metadata.

Publications

Liddy, L., Allen, E., Finneran, T., Gay, G., Hembrooke, H., and Granka, L. (2003). MetaTest: Evaluation of Metadata from Generation to Use. Proceedings of the Joint Conference on Digital Libraries (JCDL), Houston, TX.

Liddy, L., Gay, G., Allen, E., Hembrooke, H., Finneran, T., and Granka, L. (2003, October). MetaTest: Experimental Results on Utilities of Metadata for Information Access. Poster session presented at the NSDL All Projects Meeting, Washington, DC.

Liddy, L., Gay, G., Allen, E., Hembrooke, H., Finneran, T., and Granka, L. (2002, December). MetaTest: Evaluating the Quality and Utility of Metadata. Poster session presented at the NSDL All Projects Meeting, Washington, DC.