Some Thoughts on Failure
Analysis for Noisy Data
Donna Harman
NIST, USA
A
critical piece of evaluation is doing a failure analysis of the experimental
results. This is necessary to improve results, but equally important, it is
necessary to understand where the problems REALLY are, as opposed to just
following "conventional" wisdom.
The talk will present some current failure analysis techniques and
examine how these techniques could be extended to retrieval from noisy data.
About Donna Harman
Donna Harman
graduated from Cornell University
with a degree in electrical engineering, but has been involved with research in
new search engine techniques for many years.
She retired from the National Institute of Standards and Technology in
2005 after leading a group that worked in the area of natural language access
to full text, both in search and browsing modes. In 1992 she started the Text Retrieval
Conference (TREC), an ongoing forum that brings together researchers from industry
and academia to test their search engines against a common corpora involving
over a million documents, with appropriate topics and relevance judgments. She received the 1999 Strix Award from the
U.K Institute of Information Scientists for this effort. Starting in 2000 she
worked with Paul Over (also at NIST) and the text summarization community to
form a new effort (DUC) to evaluate text summarization. She is currently a scientist emeritus at
NIST.
