1. Some Personal Observations Donna Harman NIST

2. Language issues I see learning about accessing information both within and across different languages as the major contribution of CLEF Note that this includes language independent technologies such as image retrieval because the ability to access images globally is of major importance Additionally CLEF is the premier forum for addressing information access issues “unique” to Europe

3. So what has changed? Image retrieval moving more away from text Speech retrieval once again shown to be robust, but with a different source of speech (interviews) The new QAST subtask exciting because we are working on different sources of speech again; but we need to move beyond English Geography and maps growing more important; but maybe we need to think more about how people want to access this type of information; maybe this is something for an interactive task???

4. My list of language issues What IR or QA techniques are language independent and what ones are not? What are the special characteristics of a language that need to be dealt with as special cases? For congratulations here: U. Neuchâtel and Microsoft Research India Why is this important--->the emphasis is MULTILINGUAL HERE!!!!

5. Analysis; more analysis You have spent months of time on software and runs; now make the effort pay off Analysis is more than statistical tables –Failure analysis: look at topics or questions where your new method failed –Dig REALLY deep to understand WHY something worked or didn’t work –Think about generalization of what you have learned; this particularly applies to components Then PUBLISH, including outside of CLEF