arxiv_cv 90% Match Research Paper ML Engineers,Hardware Designers,Researchers in Model Compression,Computer Vision Practitioners 3 days ago

Efficiently Training A Flat Neural Network Before It has been Quantizated

computer-vision › object-detection

📄 Abstract

Abstract: Post-training quantization (PTQ) for vision transformers (ViTs) has garnered significant attention due to its efficiency in compressing models. However, existing methods typically overlook the relationship between a well-trained NN and the quantized model, leading to considerable quantization error for PTQ. However, it is unclear how to efficiently train a model-agnostic neural network which is tailored for a predefined precision low-bit model. In this paper, we firstly discover that a flat full precision neural network is crucial for low-bit quantization. To achieve this, we propose a framework that proactively pre-conditions the model by measuring and disentangling the error sources. Specifically, both the Activation Quantization Error (AQE) and the Weight Quantization Error (WQE) are statistically modeled as independent Gaussian noises. We study several noise injection optimization methods to obtain a flat minimum. Experimental results attest to the effectiveness of our approach. These results open novel pathways for obtaining low-bit PTQ models.

Authors (3)

Peng Xia

Junbiao Pang

Tianyang Cai

Submitted

November 3, 2025

arXiv Category

cs.CV

arXiv PDF

Key Contributions

Proposes a framework to efficiently train a 'flat' full-precision neural network that is crucial for low-bit quantization. It achieves this by measuring and disentangling quantization error sources (AQE and WQE) and using noise injection optimization, significantly reducing quantization error in PTQ.

Business Value

Enables deployment of powerful deep learning models on resource-constrained devices (e.g., edge AI, mobile), reducing hardware costs and power consumption.

Paper Metadata

Innovation Type

Algorithmic

Deployment Feasibility

High for enabling deployment on edge devices, as it focuses on improving the efficiency of quantized models.

Limitations Addressed

Addresses the significant quantization error in Post-Training Quantization (PTQ) methods, particularly for Vision Transformers, by proactively training a network optimized for low-bit precision.

Performance Gains

Significant reduction in quantization error compared to existing PTQ methods, leading to better performance of quantized models.

Technical Tags

post-training quantization (PTQ)vision transformers (ViTs)model compressionquantization errorlow-bit quantizationneural network trainingactivation quantization error (AQE)weight quantization error (WQE)

Research Topics

Model CompressionQuantization TechniquesEfficient Deep LearningComputer VisionHardware Acceleration

Methods & Architectures

Post-training quantization (PTQ)Flat network trainingError source disentanglementGaussian noise modeling (AQE, WQE)Noise injection optimization Vision Transformer (ViT)CNN

Applications & Tasks

Edge Computing Mobile Devices Embedded Systems Resource-constrained environments High quantization error in PTQLack of tailored training for low-bit modelsRelationship between well-trained and quantized models overlooked Efficient model compressionReducing quantization errorTraining models for low-bit quantization

Related Fields

Machine LearningComputer VisionHardware AccelerationModel CompressionDeep Learning Optimization

Keywords

quantizationPTQViTmodel compressionlow-bitefficient deep learningneural networksedge AIembedded systemsactivation quantization errorweight quantization errorflat network

Academic Context

#Model Compression#Quantization Techniques#Efficient Deep Learning#Computer Vision#Hardware Acceleration

Technology Stack

Frameworks & Libraries

PyTorchTensorFlow

Programming Languages

Python

Commercial Potential

Potential Products

Optimized deep learning libraries for edge devicesTools for efficient model deployment on embedded systems

Target Industries

Mobile TechnologyIoTAutomotiveRoboticsConsumer Electronics

Use Case Examples

Running complex vision models on smartphonesDeploying AI on low-power embedded devices for real-time processing

Competitive Edge

Offers a novel pre-quantization training strategy that directly addresses the root causes of quantization error, outperforming standard PTQ methods.

Market Opportunity

Large and growing market for edge AI and efficient deep learning deployment.

Revenue Models

Licensing of the training methodologyintegration into MLOps platforms.

Resource Requirements

Compute Needs

Standard GPU training infrastructure, potentially less demanding than training from scratch.

Data Requirements

Standard image datasets used for training vision models (e.g., ImageNet).

Deployment Constraints

The final quantized model must fit within the target device's memory and computational limits.

Scalability

Applicable to various Vision Transformer architectures and potentially other neural network types.

Production Readiness

Maturity Level

Research

Time to Market

1-2 years for integration into model optimization toolchains.

Patent Potential

Moderate to High, for the novel training framework and error disentanglement techniques.

View Full Paper Back to Papers