arxiv_ml 85% Match Research Paper Researchers in NLP and ML,Data Scientists,Information Retrieval Specialists 2 weeks ago

Stick-Breaking Embedded Topic Model with Continuous Optimal Transport for Online Analysis of Document Streams

generative-ai › flow-models

📄 Abstract

Abstract: Online topic models are unsupervised algorithms to identify latent topics in data streams that continuously evolve over time. Although these methods naturally align with real-world scenarios, they have received considerably less attention from the community compared to their offline counterparts, due to specific additional challenges. To tackle these issues, we present SB-SETM, an innovative model extending the Embedded Topic Model (ETM) to process data streams by merging models formed on successive partial document batches. To this end, SB-SETM (i) leverages a truncated stick-breaking construction for the topic-per-document distribution, enabling the model to automatically infer from the data the appropriate number of active topics at each timestep; and (ii) introduces a merging strategy for topic embeddings based on a continuous formulation of optimal transport adapted to the high dimensionality of the latent topic space. Numerical experiments show SB-SETM outperforming baselines on simulated scenarios. We extensively test it on a real-world corpus of news articles covering the Russian-Ukrainian war throughout 2022-2023.

Authors (3)

Federica Granese

Serena Villata

Charles Bouveyron

Submitted

October 21, 2025

arXiv Category

cs.LG

arXiv PDF

Key Contributions

This paper introduces SB-SETM, an online topic model that extends ETM to handle data streams by merging models from successive document batches. It uniquely employs a stick-breaking construction for topic inference and a continuous optimal transport strategy for merging topic embeddings, enabling automatic topic number inference and effective handling of high-dimensional latent spaces.

Business Value

Enables real-time understanding of evolving trends and topics in large volumes of text data, such as news feeds, social media, or customer feedback, allowing for more agile decision-making and trend analysis.

Paper Metadata

Innovation Type

Algorithmic Innovation

Deployment Feasibility

Moderate. Requires efficient implementation of optimal transport and online learning algorithms. Scalability to extremely large streams needs careful consideration.

Limitations Addressed

Challenges of online topic modeling,Automatic inference of topic number,Merging topic embeddings in high-dimensional spaces

Technical Tags

topic modelingonline learningdocument streamsoptimal transportstick-breaking processembeddingunsupervised learningtime series analysislatent variable modelsdimensionality reduction

Research Topics

Unsupervised LearningTime Series AnalysisNatural Language ProcessingMachine Learning TheoryData Stream Mining

Methods & Architectures

Stick-Breaking ConstructionContinuous Optimal TransportTopic Merging StrategyEmbedded Topic Model (ETM) Embedded Topic Model (ETM)

Applications & Tasks

Text Analysis Information Retrieval Document Analysis Topic DiscoveryConcept Drift DetectionData Stream Analysis Online Topic ModelingDocument Stream AnalysisTopic Evolution Tracking

Related Fields

Machine LearningNatural Language ProcessingStatisticsData Mining

Keywords

online topic modelingdata streamsembedded topic modeloptimal transportstick-breakingunsupervised learningdocument analysistime serieslatent topicscontinuous learningtext miningdynamic topic models

Academic Context

#Unsupervised Learning#Time Series Analysis#Natural Language Processing#Machine Learning Theory#Data Stream Mining

Commercial Potential

Potential Products

Real-time trend analysis toolsDynamic content recommendation systemsEvolving topic detection services

Target Industries

MediaFinanceSocial MediaMarket Research

Use Case Examples

Monitoring evolving public opinion on social mediaTracking emerging themes in financial newsAnalyzing shifts in customer feedback over time

Competitive Edge

Offers an online alternative to offline topic models, specifically addressing the challenges of continuous data streams and automatic topic number inference, which are often limitations in existing methods.

Market Opportunity

Growing market for real-time analytics and NLP solutions.

Revenue Models

SaaS for analytics platformsAPI services.

Resource Requirements

Compute Needs

Moderate to High, depending on stream size and dimensionality.

Data Requirements

Continuous streams of text documents.

Deployment Constraints

Requires robust streaming infrastructure and efficient computation of optimal transport.

Scalability

Designed for data streams, but computational cost of optimal transport needs to be managed for very high throughput.

Production Readiness

Maturity Level

Research

Time to Market

1-3 years

Patent Potential

Low

View Full Paper Back to Papers