Redirecting to original paper in 30 seconds...
Click below to go immediately or wait for automatic redirect
📄 Abstract
Abstract: Multimodal Large Language Models (MLLMs) often struggle with fine-grained
perception, such as identifying small objects in high-resolution images or
finding key moments in long videos. Existing works typically rely on
complicated, task-specific fine-tuning, which limits their generalizability and
increases model complexity. In this work, we propose an effective,
training-free framework that uses an MLLM's intrinsic uncertainty as a
proactive guidance signal. Our core insight is that a model's output entropy
decreases when presented with relevant visual information. We introduce a
unified mechanism that scores candidate visual inputs by response uncertainty,
enabling the model to autonomously focus on the most salient data. We apply
this simple principle to three complex visual tasks: Visual Search, Long Video
Understanding, and Temporal Grounding, allowing off-the-shelf MLLMs to achieve
performance competitive with specialized, fine-tuned methods. Our work
validates that harnessing intrinsic uncertainty is a powerful, general strategy
for enhancing fine-grained multimodal performance.
Key Contributions
This paper proposes an effective, training-free framework that leverages the intrinsic uncertainty of Multimodal Large Language Models (MLLMs) to guide complex visual tasks. The core insight is that output entropy decreases with relevant visual information, enabling a unified mechanism to score visual inputs based on response uncertainty, allowing MLLMs to autonomously focus on salient data without task-specific fine-tuning.
Business Value
Enables rapid deployment and adaptation of MLLMs for various visual tasks without costly retraining, making advanced AI capabilities more accessible and efficient for businesses.