A Multimodal Dataset for Synthesizing Rap Vocals and 3D Motion

RapVerse is a large-scale multimodal dataset designed to train AI models on rap music. It includes two main subsets: Rap-Vocal (108+ hours of rap vocals paired with clean, timestamped lyrics) and Rap-Motion (26+ hours of studio performance videos annotated with 3D body mesh data using SMPL-X). The dataset supports research into singing voice synthesis, motion generation, and multimodal learning. Collected through a meticulous pipeline involving audio separation, transcription, human pose estimation, and manual filtering, RapVerse is a unique resource at the intersection of music, language, and embodied AI.