Saron D. Samuel

I’m a PhD student in Computer Science at Johns Hopkins University, advised by Eugene Yang and Benjamin Van Durme, and part of the Center for Language and Speech Processing and the Human Language Technology Center of Excellence (HLTCOE). I work on information retrieval: building systems that find and rank information across languages and modalities — text, video, and audio — and asking whether they do so fairly and completely, not just accurately.

A few recent projects: CoverageBench, a benchmark suite for measuring how completely a retrieval system covers the relevant information for a query rather than just its precision or recall (SIGIR 2026); a systematic study of whether reasoning-based rerankers are any fairer than standard ones (ECIR 2026); and MMMORRF, a multimodal, multilingual video retrieval system that fuses visual, audio, and text signals (SIGIR 2025 demo). You can read more about all of these on my projects page.

Before JHU, I completed my BS and MS in Computer Science at Stanford University, advised by Mehran Sahami, where I worked with Omar Khattab and Chris Potts on building accurate neural retrieval and reranking models from very little labeled data. I’m a Percy Pierre Doctoral Fellow and a recipient of the Jun Wu and Yan Zhang Graduate Student Fellowship.

I’m always glad to talk about information retrieval, fairness in ML systems, or mentoring in STEM — my CV has the full history, and my publications page has the papers. Feel free to reach out.

news

Apr 02, 2026	Paper accepted to SIGIR 2026! Paper, HuggingFace
Dec 16, 2025	Paper accepted to ECIR 2026! Paper, Tweet
Sep 23, 2025	Awarded Jun Wu and Yan Zhang Graduate Student Fellowship.
Aug 25, 2025	Started PhD at JHU!
Jun 15, 2025	Graduated from Stanford! Honored to be a finalist of the Baccalaureate Speech Competition.

selected publications

CoverageBench: Evaluating Information Coverage across Tasks and Domains

Saron Samuel, Andrew Yates, Dawn Lawrie, and 4 more authors

In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2026

Abs arXiv Bib HTML Website

Precision and recall are computed per-document, so a retrieval system can score well on both while still missing whole facets of a query’s information need. CoverageBench is a benchmark suite for measuring how completely a set of retrieved documents covers the information relevant to a query, built from 334 topics across 7 existing collections (including CAsT, Fair Ranking, NeuCLIR, RAG, RAGTIME, and CRUX) with nugget-level relevance judgments and baseline results from BM25 and Qwen3-8B-based rerankers.
@inproceedings{samuel2026coverage, title = {CoverageBench: Evaluating Information Coverage across Tasks and Domains}, author = {Samuel, Saron and Yates, Andrew and Lawrie, Dawn and Soboroff, Ian and Adriaanse, Trevor and {Van Durme}, Benjamin and Yang, Eugene}, booktitle = {Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval}, year = {2026}, }
Does Reasoning Make Search More Fair? Comparing Fairness in Reasoning and Non-Reasoning Rerankers

Saron Samuel, Benjamin Van Durme, and Eugene Yang

In Proceedings of the 48th European Conference on Information Retrieval, 2026

Abs arXiv Bib HTML

Reasoning rerankers are increasingly used in search, but it was unknown whether their extra reasoning steps help or hurt fairness. We ran the first systematic comparison of fairness in reasoning versus non-reasoning rerankers, evaluating six models on the TREC 2022 Fair Ranking Track data. Reasoning neither reliably improves nor harms fairness: Attention-Weighted Rank Fairness stayed within a narrow 0.33-0.35 band across all models despite large swings in relevance quality, and geographic-attribute disparities persisted regardless of architecture, suggesting fairness has to be specifically trained for rather than assumed to emerge from better reasoning.
@inproceedings{samuel2026reasoning, title = {Does Reasoning Make Search More Fair? Comparing Fairness in Reasoning and Non-Reasoning Rerankers}, author = {Samuel, Saron and {Van Durme}, Benjamin and Yang, Eugene}, booktitle = {Proceedings of the 48th European Conference on Information Retrieval}, year = {2026}, }
MMMORRF: Multimodal Multilingual Modularized Reciprocal Rank Fusion

Saron Samuel, Dan DeGenaro, Jimena Guallar-Blasco, and 13 more authors

In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (Demo Track), 2025

Abs DOI arXiv Bib HTML

Videos carry information across several channels at once: visuals, on-screen text, audio, and speech. Leading multimodal retrieval systems like VAST and LanguageBind lean almost entirely on the visual channel because they’re built on vision-language models. MMMORRF instead extracts text and features from both the visual and audio modalities and fuses them with a modality-aware weighted reciprocal rank fusion, improving nDCG@20 by 81% over leading multimodal encoders and 37% over single-modality retrieval on the MultiVENT 2.0 and TVR benchmarks.
@inproceedings{samuel2025mmmorrf, title = {MMMORRF: Multimodal Multilingual Modularized Reciprocal Rank Fusion}, author = {Samuel, Saron and DeGenaro, Dan and Guallar-Blasco, Jimena and Sanders, Kate and Eisape, Oluwaseun and Spendlove, Tanner and Reddy, Arun and Martin, Alexander and Yates, Andrew and Yang, Eugene and Carpenter, Cameron and Etter, David and Kayi, Efsun and Wiesner, Matthew and Murray, Kenton and Kriz, Reno}, booktitle = {Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (Demo Track)}, year = {2025}, doi = {10.1145/3726302.3730157}, }