TY - JOUR
T1 - Modeling search and session effectiveness
AU - Wicaksono, Alfan Farizki
AU - Moffat, Alistair
N1 - Funding Information:
This work was in part supported by the Australian Research Council (grant LP150100252, held in collaboration with Seek.com; and grant DP190101113). We gratefully acknowledge the assistance of Bahar Salehi and Justin Zobel (The University of Melbourne); Damiano Spina (RMIT University); and Sargol Sadeghi and Vincent Li (Seek). We also thank the anonymous referees, who provided helpful comments that improved the paper.
Funding Information:
This work was in part supported by the Australian Research Council (grant LP150100252 , held in collaboration with Seek.com; and grant DP190101113 ). We gratefully acknowledge the assistance of Bahar Salehi and Justin Zobel (The University of Melbourne); Damiano Spina (RMIT University); and Sargol Sadeghi and Vincent Li (Seek). We also thank the anonymous referees, who provided helpful comments that improved the paper.
Publisher Copyright:
© 2021 Elsevier Ltd
PY - 2021/7
Y1 - 2021/7
N2 - Many information needs cannot be resolved with a single query, and instead lead naturally to a sequence of queries, issued as a search session. In a session test collection, each topic has an associated query sequence, with users assumed to follow that sequence when reformulating their queries. Here we propose a session-based offline evaluation framework as an extension to the existing query-based C/W/L framework, and use that framework to devise an adaptive session-based effectiveness metric, as a way of measuring the overall usefulness of a search session. To realize that goal, data from two commercial search engines is employed to model the two required behaviors: the user conditional continuation probability, and the user conditional reformulation probability. We show that the session-extended C/W/L framework allows the development of new metrics with associated user models that give rise to greater correlation with observed user behavior during search sessions than do previous session metrics, and hence provide a richer context in which to compare retrieval systems at a session level.
AB - Many information needs cannot be resolved with a single query, and instead lead naturally to a sequence of queries, issued as a search session. In a session test collection, each topic has an associated query sequence, with users assumed to follow that sequence when reformulating their queries. Here we propose a session-based offline evaluation framework as an extension to the existing query-based C/W/L framework, and use that framework to devise an adaptive session-based effectiveness metric, as a way of measuring the overall usefulness of a search session. To realize that goal, data from two commercial search engines is employed to model the two required behaviors: the user conditional continuation probability, and the user conditional reformulation probability. We show that the session-extended C/W/L framework allows the development of new metrics with associated user models that give rise to greater correlation with observed user behavior during search sessions than do previous session metrics, and hence provide a richer context in which to compare retrieval systems at a session level.
KW - Information retrieval
KW - Search effectiveness
KW - Session effectiveness
KW - User behavior
KW - User model
KW - Web search
UR - http://www.scopus.com/inward/record.url?scp=85104317559&partnerID=8YFLogxK
U2 - 10.1016/j.ipm.2021.102601
DO - 10.1016/j.ipm.2021.102601
M3 - Article
AN - SCOPUS:85104317559
SN - 0306-4573
VL - 58
JO - Information Processing and Management
JF - Information Processing and Management
IS - 4
M1 - 102601
ER -