university of washington,

No.23-13 Task-aware Retrieval with Instructions

Yadan Luo Follow Aug 02, 2023 · 1 min read
No.23-13 Task-aware Retrieval with Instructions
Share this

We study the problem of retrieval with instructions, where users of a retrieval system explicitly describe their intent along with their queries. We aim to develop a general-purpose task-aware retrieval system using multi-task instruction tuning, which can follow human-written instructions to find the best documents for a given query. We introduce the first large-scale collection of approximately 40 retrieval datasets with instructions, BERRI, and present TART, a multi-task retrieval system trained on BERRI with instructions. TART shows strong capabilities to adapt to a new retrieval task via instructions and advances the state of the art on two zero-shot retrieval benchmarks, BEIR and LOTTE, outperforming models up to three times larger. We further introduce a new evaluation setup, X^2-Retrieval to better reflect real-world scenarios, where diverse domains and tasks are pooled and a system needs to find documents aligning users’ intents. In this setup, TART significantly outperforms competitive baselines, further demonstrating the effectiveness of guiding retrieval with instructions.

Speaker Bio

Akari Asai is a Ph.D. student in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, advised by Prof. Hannaneh Hajishirzi. Her research lies in natural language processing and machine learning. Her recent research focuses on question answering, multilingual NLP, and NLP efficiency. She received the IBM Fellowship in 2022 and the Nakajima Foundation Fellowship in 2019. Prior to UW, she obtained a B.E. degree in Electrical Engineering and Computer Science from the University of Tokyo.

More Details

Zoom Recording

Zoom link

Join Newsletter
Get the latest news right in your inbox. We never spam!