Medical Image Retrieval Needs a New Benchmark

This paper establishes a benchmark for 3D content-based image retrieval (CBIR) in medical imaging using the TotalSegmentator dataset. It evaluates supervised embeddings trained on medical images against self-supervised embeddings from non-medical datasets, testing retrieval at both organ and region levels. By introducing a late interaction re-ranking method inspired by text retrieval, the study achieves near-perfect recall across diverse anatomical structures. The results provide a much-needed benchmark and roadmap for future development of AI-powered medical image retrieval systems, enabling more reliable, precise, and efficient radiology workflows.