Ahmed JouiniAI Research Engineer · Computer Vision
All work06 / 07
The Story MarketMunich, Germany (remote)
2021Dec 2021 – Feb 2022

NLP · topic modelling

German News Topic Discovery

A month of German news, organised into topics.

Multilingual sentence embeddings, density-based clustering and class-based TF-IDF turn thousands of unlabelled German articles into readable topics.

  • 3,283German articles from one month of news (December 2021)
  • 40topics found without fixing their number in advance
  • 3document representations benchmarked: TF-IDF, Doc2Vec, BERT

Overview

The problem

A media-intelligence company needs to know what the press is writing about, in German, every month. Thousands of articles arrive without labels, new stories appear constantly, and keyword lists miss them.

What was missing

Bag-of-words representations count words, not meaning, and the number of topics in a month of news is not known in advance. The approach had to work in German, find the number of topics by itself and keep uncertain articles apart rather than forcing them into a topic.

Architecture

INPUTGerman presstargeted scraping ·article text + 19metadata fieldsPROCESSINGCleaningmissing text removed ·DE + EN stop wordsDOCUMENT REPRESENTATION · BENCHMARKEDTF-IDFDoc2VecSELECTEDMultilingual Sentence-BERTdistiluse-base-multilingual-cased ·512-dPROCESSINGUMAP5-D · cosine · 15 neighboursPROCESSINGHDBSCANmin cluster size 15 · outlierskept apartPROCESSINGc-TF-IDF topic wordsbi- and tri-grams per clusterPROCESSINGTopic mergingsmallest topic into its mostsimilar, 20 roundsPROCESSINGOutlier re-clusteringnew topics joined if cosine ≥0.4OUTPUTTopicsword clouds · topic sizes ·document mapsLEARNEDNamed-entityrecognitionpeople, organisations,placesOUTPUTClient-facing analyticsentities and themes from unstructured articlesThe selected path is the one used for the topic results shown below; TF-IDF and Doc2Vec were the compared baselines.

Swipe to see the whole diagram →

From scraped articles to topics and entities. The highlighted path produced the topic results below.
INPUTArticle textGerman, first 128tokensPROCESSINGWordPiece tokensmultilingual casedvocabularyDISTILBERT ENCODERTransformer layer 1 · 768-dTransformer layer 2 · 768-dTransformer layer 3 · 768-dTransformer layer 4 · 768-dTransformer layer 5 · 768-dTransformer layer 6 · 768-dPROCESSINGMean poolingover tokenembeddingsLAYERDense 768 → 512tanhOUTPUTDocument vector512-d · compared bycosinec-TF-IDF for topic c and n-gram t: (count of t in c ÷ words in c) × log(articles ÷ count of t in all topics)

Swipe to see the whole diagram →

Inside the document encoder (distiluse-base-multilingual-cased): a DistilBERT encoder, mean pooling and a dense layer give one 512-d vector per article. Topic words come from class-based TF-IDF.
Technical details
  • Embeddings: sentence-transformers distiluse-base-multilingual-cased (DistilBERT, mean pooling, dense 768 → 512 with tanh; inputs truncated to 128 tokens).
  • UMAP to 5 dimensions (cosine, 15 neighbours) before clustering; a separate 2-D UMAP for the maps.
  • HDBSCAN with minimum cluster size 15 and excess-of-mass selection; unassigned articles keep the label −1.
  • Class-based TF-IDF: all articles of a topic are joined into one document, n-grams of length 2–3, German and English stop words plus a custom list.
  • Topic merging: the smallest topic is merged into its most similar topic (cosine on c-TF-IDF), 20 rounds. Outliers are re-embedded and re-clustered; a new topic joins an existing one when the cosine similarity of their top words exceeds 0.4.

What I built

  1. 01

    Data acquisition

    Targeted scraping of major German press sources, with article text and 19 metadata fields (title, author, agency, dates, category, keywords, paywall status and more).

  2. 02

    Representation benchmark

    TF-IDF, Doc2Vec and pretrained BERT-based document embeddings compared for German news.

  3. 03

    Topic pipeline

    Multilingual Sentence-BERT embeddings, UMAP reduction, HDBSCAN clustering and class-based TF-IDF on bi- and tri-grams, with topic merging and re-clustering of outliers.

  4. 04

    Entities

    A named-entity recognition module that pulls people, organisations and places out of unstructured articles for client-facing analytics.

Main results

40topics clustered from 3,283 articlesafter merging
418articles in the largest topic: Omicron variant and COVID-19December 2021
1,028articles left unassigned by the first pass, then re-clusteredoutliers are kept apart, not forced
Largest topics after merging, with their highest-scoring n-grams (topic names added for reading)
TopicArticlesTop n-grams
COVID-19418omikron variante · covid 19 · bund länder
German federal politics211olaf scholz · angela merkel · friedrich merz
Football177champions league · fc bayern · premier league
UK politics166boris johnson · downing street
Document maps (2-D UMAP): the first clustering pass (left) and the re-clustering of outlier articles (right). Colours are topics; −1 is unassigned.
Document maps (2-D UMAP): the first clustering pass (left) and the re-clustering of outlier articles (right). Colours are topics; −1 is unassigned.

Qualitative results

Topic: German federal politics.
Topic: German federal politics.
Topic: European football.
Topic: European football.
Topic: Russia–Ukraine border tensions and Nord Stream.
Topic: Russia–Ukraine border tensions and Nord Stream.

Limits and next steps

UMAP is stochastic, so topics vary between runs; about a third of the articles were unassigned after the first pass; the encoder reads only the first 128 tokens of each article; evaluation was qualitative.

Beyond this project

Unsupervised organisation of any large, unlabelled text stream: support tickets, reviews, reports, scientific literature.

Contact

Interested in the work?

I'm looking for AI research engineer roles in computer vision and video understanding, ideally in sports analytics, though not only. Ready to start a new journey.

Paris, France

[email protected] Email me
Download CV LinkedIn HAL