Visual Prompt Generators (VPGs): Encoding Images to LLM Tokens

Explains how MLLMs use VPGs and cross-attention with learnable query embeddings to extract essential visual tokens from image patches for LLM input