arxiv_ml 95% Match Research Paper Computational Biologists,ML Researchers,Bioinformaticians,Drug Discovery Scientists 2 weeks ago

g-DPO: Scalable Preference Optimization for Protein Language Models

large-language-models › alignment

📄 Abstract

Abstract: Direct Preference Optimization (DPO) is an effective approach for aligning protein language models with experimental design goals. However, DPO faces a scalability bottleneck: the number of possible training pairs grows quadratically with the number of labeled sequences, leading to prohibitive training times even for modestly sized datasets. We introduce g-DPO, a framework that (i) uses sequence space clustering to prune redundant pairs while preserving training signal, and (ii) amortizes likelihood computations with group-based approximations. Across three protein engineering tasks, g-DPO maintains in-silico and in-vitro performance that is statistically indistinguishable from standard DPO, while converging 1.8 to 3.7 times faster, with greater gains expected as the size of the dataset increases.

Authors (6)

Constance Ferragu

Jonathan D. Ziegler

Nicolas Deutschmann

Arthur Lindoulsi

Eli Bixby

Cradle ML Team

Submitted

October 22, 2025

arXiv Category

cs.LG

arXiv PDF

Key Contributions

g-DPO addresses the scalability bottleneck of Direct Preference Optimization (DPO) for protein language models by introducing sequence space clustering to prune redundant pairs and using group-based approximations for amortized likelihood computations. This framework achieves comparable performance to standard DPO but converges significantly faster.

Business Value

Accelerates the design and optimization of proteins for various applications (e.g., therapeutics, enzymes), reducing R&D costs and time-to-market for new biologics and biomaterials.

Paper Metadata

Innovation Type

Algorithmic

Deployment Feasibility

High. Improves an existing, valuable training technique.

Limitations Addressed

The quadratic complexity of training pairs in DPO, leading to prohibitive training times and scalability issues, especially for large datasets.

Performance Gains

1.8 to 3.7 times faster convergence than standard DPO,Statistically indistinguishable in-silico and in-vitro performance

Technical Tags

Direct Preference Optimization (DPO)Protein Language ModelsScalabilitySequence ClusteringAmortized LikelihoodGroup-based ApproximationsProtein EngineeringExperimental Design

Research Topics

Reinforcement Learning from Human Feedback (RLHF)Preference OptimizationProtein DesignLarge Language ModelsEfficient Training

Methods & Architectures

g-DPO frameworkSequence space clusteringPruning redundant pairsAmortized likelihood computationsGroup-based approximations Protein Language ModelsDirect Preference Optimization (DPO)

Applications & Tasks

Computational Biology Drug Discovery Materials Science Biotechnology Scalability bottleneck in DPOQuadratic growth of training pairsProhibitive training times Aligning protein language modelsOptimizing protein sequences for experimental designAccelerating DPO training

Datasets & Benchmarks

Benchmarks

Three protein engineering tasks

In-silico performanceIn-vitro performanceConvergence speed

Related Fields

Computational BiologyMachine LearningNatural Language ProcessingReinforcement LearningBioinformatics

Keywords

DPODirect Preference OptimizationProtein Language ModelsScalabilitySequence ClusteringProtein EngineeringAlignmentRLHFEfficient TrainingBiotechnologyg-DPO

Academic Context

#Reinforcement Learning from Human Feedback (RLHF)#Preference Optimization#Protein Design#Large Language Models#Efficient Training

Commercial Potential

Potential Products

Protein Design SoftwareAccelerated ML Training Platforms for BiologyBiologics Discovery Tools

Target Industries

BiotechnologyPharmaceuticalsAgricultureMaterials Science

Use Case Examples

Designing novel enzymes for industrial processesDeveloping therapeutic proteins with improved efficacyCreating new biomaterials with desired properties

Competitive Edge

Offers a significant improvement in training efficiency for DPO in the context of protein language models, making preference optimization more practical for large-scale biological applications.

Market Opportunity

Rapidly growing market for AI in drug discovery and protein engineering.

Revenue Models

Licensing of algorithmsSaaS platforms for protein design.

Resource Requirements

Compute Needs

High (for training protein language models)

Data Requirements

Labeled protein sequences with preference data

Deployment Constraints

Requires careful selection of clustering parameters; effectiveness depends on the quality of preference data.

Scalability

The clustering and group approximation methods are designed to improve scalability significantly as dataset size increases.

Regulatory Considerations

Ethical considerations in protein design and potential dual-use applications.

Production Readiness

Maturity Level

Research

Time to Market

2-4 years

Patent Potential

High (for the g-DPO methodology)

View Full Paper Back to Papers