Everything K-culture — comebacks to K-beauty, straight to your inboxGet it in your inbox›
BERTilda: Explainable Topic Lifecycle Tracking with Split/Merge Detection via Similarity-and-Flow Temporal Graphs
arXiv:2608.181012026-08-20
A method that tracks how topics split, merge, and disappear over time, and shows its reasoning
Topics in news or social media do not just drift smoothly; they can suddenly split into sub-debates or merge into broader narratives. BERTilda discovers topics independently in each time window, then links them across windows using both semantic similarity and a bidirectional document-flow signal that tracks where documents actually go and come from, labeling each transition as continuation, split, merge, disappearance, or unclear. Tested on U.S. congressional tweets, three human annotators reached majority agreement of up to 87% with BERTilda's outputs, the highest among compared methods.
METAL MEDIA explanatory visual
A method that tracks how topics split, merge, and disappear over time, and shows its reasoning
01Motivation: many dynamic topic models assume smooth topic drift, while snapshot models (fit separately per time window) leave the correspondence between windows' topics unresolved
02Method: topics are discovered independently per window using BERTopic, then linked across adjacent windows via (1) cosine similarity between topic embeddings and (2) bidirectional coverage -- outflow (where a topic's documents go next) and inflow (where a topic's documents came from)
03Transparent rules applied to the resulting graph label each link as continuation, split, merge, disappearance, or unclear, producing a temporal topic graph
04Evaluated on 357,896 tweets from 544 members of the 119th U.S. Congress plus UN General Debates and State of the Union speeches; three independent annotators reviewed 120 randomly sampled events (30 per type)
05BERTilda achieved a macro-average validation precision of 0.775, higher than similarity-only (0.571) and forward-only (0.725) baselines, with especially strong disappearance detection; under threshold perturbations, macro-average precision stayed between 0.663 and 0.718, showing it isn't tied to one hand-tuned setting
An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.
What they did
Motivation: many dynamic topic models assume smooth topic drift, while snapshot models (fit separately per time window) leave the correspondence between windows' topics unresolved
Method: topics are discovered independently per window using BERTopic, then linked across adjacent windows via (1) cosine similarity between topic embeddings and (2) bidirectional coverage -- outflow (where a topic's documents go next) and inflow (where a topic's documents came from)
Transparent rules applied to the resulting graph label each link as continuation, split, merge, disappearance, or unclear, producing a temporal topic graph
Evaluated on 357,896 tweets from 544 members of the 119th U.S. Congress plus UN General Debates and State of the Union speeches; three independent annotators reviewed 120 randomly sampled events (30 per type)
BERTilda achieved a macro-average validation precision of 0.775, higher than similarity-only (0.571) and forward-only (0.725) baselines, with especially strong disappearance detection; under threshold perturbations, macro-average precision stayed between 0.663 and 0.718, showing it isn't tied to one hand-tuned setting
Table 1: Datasets and temporal segmentation used in experiments.
Dataset
Domain
Time span
Unit
Windowing
Congress tweets
social media
2024–2025
tweet
7d window, 3d step
UN General Debates
speeches
1970–2015
paragraph/speech
3y window, 2y step
State of the Union
speeches
1790–2018
paragraph/speech
7y window, 3y step
Table 2: Default decision thresholds used in the alignment and event-labeling stages. Dataset-specific window size and step are described in the experimental setup.
Symbol
Meaning
Value
Similarity and edge validation
τdoc
min doc-to-topic similarity for attribution
0.40
τtopic
min topic-to-topic similarity for candidate edge
0.70
τcovout=τcovin
min coverage for a validated edge
0.30
Event-labeling thresholds
αcont
continuation similarity threshold
0.90
αcontout=αcontin
min inflow/outflow coverage for continuation
0.50
αsplitout
min total outflow coverage for split
0.70
αsplitin
min inflow coverage per successor
0.50
αdispout=αdispin
min inflow/outflow coverage for disappearance
0.40
αmergeout
min outflow coverage per predecessor
0.50
αmergein
min total inflow coverage for merge target
0.70
Table 3: Static topic quality metrics on the congressional tweets dataset.
Model
CV
NPMI
UMass
Diversity
TQ
BERTilda (BERTopic + temporal graph)
0.7381
0.1981
-2.8489
0.9751
0.7200
Top2Vec
0.3565
−0.3005
−11.5995
0.8937
0.3184
Table 4: Distribution of predicted event labels on the 120 annotated gold-set items.
Method
Continue
Disappear
Split
Merge
BERTilda (similarity + bidirectional coverage)
30
30
30
30
Similarity-only (normalized topic similarity)
20
48
21
20
Lexical-only (normalized c-TF-IDF similarity)
4
106
0
0
Forward-only (no backward attribution)
20
48
20
21
Table 5: Annotator-confirmed validation rate (precision) on predicted events, using majority vote across three annotators.
Method
Continue
Disappear
Split
Merge
BERTilda (similarity + bidirectional coverage)
0.867
0.800
0.767
0.667
Similarity-only (normalized topic similarity)
0.750
0.417
0.667
0.450
Lexical-only (normalized c-TF-IDF similarity)
0.750
0.226
n/a
n/a
Forward-only (no backward attribution)
0.800
0.558
0.826
0.714
Table 6: Overall topic quality metrics on UN General Debates (mean ± std across windows).
Model
CV
NPMI
UMass
Diversity
TQ
BERTilda
0.593 ± 0.066
0.084 ± 0.044
-1.105 ± 0.776
0.921 ± 0.074
0.551 ± 0.095
Tomotopy DTM
0.361 ± 0.037
-0.030 ± 0.013
-0.697 ± 0.614
0.589 ± 0.067
0.211 ± 0.019
DETM
0.542 ± 0.012
–
–
0.998 ± 0.000
0.541 ± 0.012
Table 7: Temporal drift on UN General Debates (drift = β1).
Model
Metric
Drift
p-value
BERTilda
CV
0.0007
0.5635
Diversity
0.0051
0.0062
TQ
0.0031
0.1062
Tomotopy DTM
CV
0.0009
0.0204
Diversity
0.0016
0.0310
TQ
0.0010
2.05e-09
DETM
CV
2.13e-04
0.1215
Diversity
-1.53e-07
0.9717
TQ
2.13e-04
0.1219
Why it matters
For analyzing political speech, news, or social media where narratives constantly fragment and recombine over time, having an auditable basis (document flow) for why an event was labeled a split or merge matters. Unlike black-box deep learning approaches, analysts can inspect the graph directly to check reasoning and correct errors.
Terms in this paper
snapshot topic model · a topic model fit separately on documents from one time window at a time, rather than jointly across time
embedding · a numeric vector representation capturing the meaning of text
cosine similarity · a measure of how similar two vectors are based on the angle between them
bidirectional coverage (outflow/inflow) · a signal counting how many documents from a topic flow into the next window's topics (outflow) and how many a topic receives from the previous window (inflow)
BERTopic · an open-source topic modeling tool combining sentence embeddings, clustering, and keyword extraction
Figures we cannot republish
Figure 1: Representative local threshold-sensitivity analyses on the congressional gold set. Left: varying τdoc. Right: varying αdispout. Full curves for all five thresholds are reported in the supplementary material.
Figure 2: Example temporal evolution of a topic (Holocaust/Israel–Antisemitism), illustrating split, continuations, and disappearance across consecutive windows.
Longitudinal text streams exhibit topic birth and death, but also discrete structural reorganizations in which themes split into subtopics or merge into broader narratives. Many dynamic topic models emphasize smooth drift, while snapshot topic models (fit independently per time window) leave temporal correspondence underspecified. We present BERTilda, an explainable framework that discovers topics independently in each window (using an embedding-based topic model) and then constructs a temporal topic graph linking topics across adjacent windows. Links are supported by two complementary signals: (i) semantic similarity between topic representations and (ii) a bidirectional coverage signal that estimates document outflow (where a topic goes) and inflow (where a topic comes from) via cross-window tweet-to-topic attribution. Graph-based rules label continuations, splits, merges, disappearances, and unclear transitions. We evaluate BERTilda on political corpora, including U.S. congressional tweets and historical speech datasets, report topic-quality and temporal-stability diagnostics, and validate lifecycle labels on a gold-standard subset annotated by three independent annotators. On the annotated subset, BERTilda reaches majority agreement rates up to 87% and attains the highest macro-average agreement across the compared methods, with particularly strong disappearance detection relative to similarity-only and forward-only baselines.
