arxiv_cv 93% Match Research Paper Medical Researchers,Radiologists,Neurologists,AI Researchers in Healthcare,Biomedical Engineers 20 hours ago

Language-Enhanced Generative Modeling for PET Synthesis from MRI and Blood Biomarkers

large-language-models › multimodal-llms

📄 Abstract

Abstract: Background: Alzheimer's disease (AD) diagnosis heavily relies on amyloid-beta positron emission tomography (Abeta-PET), which is limited by high cost and limited accessibility. This study explores whether Abeta-PET spatial patterns can be predicted from blood-based biomarkers (BBMs) and MRI scans. Methods: We collected Abeta-PET images, T1-weighted MRI scans, and BBMs from 566 participants. A language-enhanced generative model, driven by a large language model (LLM) and multimodal information fusion, was developed to synthesize PET images. Synthesized images were evaluated for image quality, diagnostic consistency, and clinical applicability within a fully automated diagnostic pipeline. Findings: The synthetic PET images closely resemble real PET scans in both structural details (SSIM = 0.920 +/- 0.003) and regional patterns (Pearson's r = 0.955 +/- 0.007). Diagnostic outcomes using synthetic PET show high agreement with real PET-based diagnoses (accuracy = 0.80). Using synthetic PET, we developed a fully automatic AD diagnostic pipeline integrating PET synthesis and classification. The synthetic PET-based model (AUC = 0.78) outperforms T1-based (AUC = 0.68) and BBM-based (AUC = 0.73) models, while combining synthetic PET and BBMs further improved performance (AUC = 0.79). Ablation analysis supports the advantages of LLM integration and prompt engineering. Interpretation: Our language-enhanced generative model synthesizes realistic PET images, enhancing the utility of MRI and BBMs for Abeta spatial pattern assessment and improving the diagnostic workflow for Alzheimer's disease.

Key Contributions

Developed a language-enhanced generative model that synthesizes Abeta-PET images from MRI and blood biomarkers using LLMs and multimodal fusion. This approach shows high agreement with real PET scans and diagnostic outcomes, offering a potential solution for improving PET accessibility.

Business Value

Could significantly reduce the cost and increase the accessibility of Alzheimer's disease diagnosis, enabling earlier intervention and better patient management.

Paper Metadata

Innovation Type

Algorithmic Development

Deployment Feasibility

Moderate. Requires integration with existing medical imaging workflows and validation across diverse patient populations. LLM inference can be computationally intensive.

Limitations Addressed

High cost and limited accessibility of Abeta-PET scans for Alzheimer's disease diagnosis.

Performance Gains

High agreement between synthetic and real PET diagnoses (accuracy = 0.80).

Technical Tags

PET SynthesisMRIBlood BiomarkersLarge Language Models (LLM)Generative ModelingMultimodal FusionAlzheimer's Disease DiagnosisMedical ImagingSSIMPearson's r

Research Topics

Medical ImagingGenerative AIAlzheimer's DiseaseMachine LearningMultimodal LearningBiomarkers

Methods & Architectures

Language-enhanced generative modelLLMMultimodal Information FusionGenerative Adversarial Networks (GANs) - implied by synthesis Generative ModelLarge Language Model (LLM)

Applications & Tasks

Healthcare Medical Diagnostics Neuroscience Biotechnology Image SynthesisDisease DiagnosisBiomarker PredictionData Augmentation Synthesizing PET images from MRI and blood biomarkersPredicting Alzheimer's disease diagnosis

Datasets & Benchmarks

Benchmarks

SSIM = 0.920 +/- 0.003 • Pearson's r = 0.955 +/- 0.007 • accuracy = 0.80

SSIMPearson's rAccuracyImage QualityDiagnostic ConsistencyClinical Applicability

Related Fields

Medical ImagingMachine LearningNeuroscienceArtificial IntelligenceBiotechnologyNatural Language Processing

Keywords

PET SynthesisMRIBlood BiomarkersAlzheimer's DiseaseLLMGenerative ModelMultimodal FusionMedical DiagnosticsImage SynthesisNeuroimagingSSIMPearson CorrelationClinical Applicability

Academic Context

#Medical Imaging#Generative AI#Alzheimer's Disease#Machine Learning#Multimodal Learning#Biomarkers

Technology Stack

Frameworks & Libraries

Large Language Model (LLM)

Commercial Potential

Potential Products

AI-powered diagnostic tools for Alzheimer'sSynthetic medical image generation platformsCost-effective diagnostic aids

Target Industries

HealthcarePharmaceuticalsBiotechnologyMedical Device Manufacturing

Use Case Examples

Diagnosing Alzheimer's disease using non-PET dataGenerating synthetic PET scans for training AI modelsReducing the need for expensive PET scans

Competitive Edge

Offers a novel approach to PET synthesis by leveraging LLMs and multimodal data, potentially outperforming methods relying solely on imaging or simpler fusion techniques.

Market Opportunity

Significant market for AD diagnostics and neurodegenerative disease research tools.

Revenue Models

Software licensingservice fees for diagnostic analysisdata licensing.

Resource Requirements

Compute Needs

High (for LLM inference and generative model training)

Data Requirements

Paired Abeta-PET images, MRI scans, and blood-based biomarkers.

Deployment Constraints

Requires access to multimodal patient data and robust validation in clinical settings.

Scalability

Scalability depends on the LLM size and the efficiency of the multimodal fusion mechanism.

Regulatory Considerations

Medical device regulations (FDACEetc.) would apply for clinical deployment.

Production Readiness

Maturity Level

Research

Time to Market

3-5 years

Patent Potential

Moderate (novel generative approach)

View Full Paper Back to Papers