A comparison of novices' initial performance, learnability and memorability with a web based IR interface

Abstract

This paper reports on an empirical study of novices’ searching with a web-based IR interface. Two different novice groups took part in this study. The NP group took part in the initial performance experiment. The NLM group took part in two different tests. The first test measured their learnability with the interface after a brief “hands-on” training. The second test was run four weeks after the learning in order to assess their memorability of search skills. Novices’ performance across all experiments was measured in terms of time taken to complete search tasks, search terms used, error rates, and success of the tasks performed. Their subjective satisfaction with the interface was also measured at the end of all tests. The results of the study showed that novices could readily perform simple search tasks. Novices’ performance improved significantly when a brief “hands-on” training was provided. Their subjective satisfaction with the interface also improved after the training. However, novices’ memorability of search skill was poor. Their satisfaction with the interface also diminished during memorability test. The paper concludes with some principles of effective IR interface design for naive searchers.