Metrics, user models, and satisfaction

Alfan Farizki Wicaksono, Alistair Moffat

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Citations (Scopus)

Abstract

User satisfaction is an important factor when evaluating search systems, and hence a good metric should give rise to scores that have a strong positive correlation with user satisfaction ratings. A metric should also correspond to a plausible user model, and hence provide a tangible manifestation of how users interact with search rankings. Recent work has focused on metrics whose user models accurately portray the behavior of search engine users. Here we investigate whether those same metrics then also correlate with user satisfaction. We carry out experiments using various classes of metrics, and confirm through the lens of the C/W/L framework that the metrics with user models that reflect typical behavior also tend to be the metrics that correlate well with user satisfaction ratings.

Original languageEnglish
Title of host publicationWSDM 2020 - Proceedings of the 13th International Conference on Web Search and Data Mining
PublisherAssociation for Computing Machinery, Inc
Pages654-662
Number of pages9
ISBN (Electronic)9781450368223
DOIs
Publication statusPublished - 20 Jan 2020
Event13th ACM International Conference on Web Search and Data Mining, WSDM 2020 - Houston, United States
Duration: 3 Feb 20207 Feb 2020

Publication series

NameWSDM 2020 - Proceedings of the 13th International Conference on Web Search and Data Mining

Conference

Conference13th ACM International Conference on Web Search and Data Mining, WSDM 2020
Country/TerritoryUnited States
CityHouston
Period3/02/207/02/20

Keywords

  • Effectiveness metric
  • Evaluation
  • Session
  • User model
  • Web search

Fingerprint

Dive into the research topics of 'Metrics, user models, and satisfaction'. Together they form a unique fingerprint.

Cite this