CS Paper Radar

Agreement Between Reconstructed and Native GDELT GKG Records: A Field-Level Baseline

Nikita A. Klimin

cs.CL cs.IR ·submitted 2026-10-26 12:48 UTC · v1
code/dataunusual

Abstract

The Global Knowledge Graph (GKG) of the Global Database of Events, Language, and Tone (GDELT) characterizes news articles using named entities, topics, quantitative and temporal features, quotations, tone, and dictionary-based content measures. Its codebook defines the record format, but the software that produces the records is not public, so researchers cannot generate compatible records for articles they collect themselves. We built an open-source reconstruction, gdelt-regkg, and measured how closely it agrees with GDELT's archived (native) records for 193,550 English articles from 8,729 publishers (2020-2025) and 2,500 articles in five other languages. Agreement depends strongly on the field. English tone, the percentage of positive minus negative words, has a mean absolute error (MAE) of 1.79 points and a concordance correlation coefficient of 0.75. Identity agreement (F1) is 0.65 for persons, 0.42 for organizations, and 0.65 for locations. It is lower for date components (0.42), complete counts with their objects and locations (0.07), and quotations (0.07), while GCAM category presence reaches micro F1 0.84 within the implemented dictionaries. In subsets where retrieved and native titles and lengths match, tone error falls to 1.47 and organization F1 rises to 0.48, yet most disagreement remains. A larger gazetteer raises coverage of native places from 89.5% to 97.1% with little change in location F1. Tone computed on English translations of non-English articles is less accurate (MAE 2.10-2.64) than for English articles. The results are conditional on article pages that could still be retrieved, and the correspondence subsets do not isolate the effect of input differences. Reconstructed records should therefore be assessed field by field and not treated as completely interchangeable with native ones.

Comment
15 pages, 10 tables, 5 figures

Closest papers

Nearest neighbours by meaning across the whole index.

ReGKG-2025: A GKG-Compatible Reconstruction of the GDELT Global Knowledge Graph for the Missing Period of 14 June to 2 July 2025

cs.DL cs.CL ·2026-11-08 ·Nikita A. Klimin
similarity 0.79
Abstract · 17 pages, 14 figures, 19 tables, including appendices. Companion methods paper: arXiv:2610.35358. Dataset: https://doi.org/10.5281/zenodo.23…

The June 2025 outage in the Global Database of Events, Language, and Tone (GDELT) collection pipeline left a gap in the Global Knowledge Graph (GKG), interrupting article-level news metadata used in downstream research. ReGKG-2025 is an archive-derived sample of 281,620 reconstructed article records covering all 1,665 missing 15-minute batches. Articles were recovered from Common Crawl and the Wayback Machine using source quotas estimated from surrounding native GKG data. Text was screened and normalized, non-English bodies were translated, and flagged records were manually reviewed. Eligible articles were converted to the 27-column GKG 2.1 layout. The versioned package includes record-level provenance, archive replay and WARC locators, source summaries, source-by-day calibration weights, and the complete manual decision journal. The sample reaches about 14% of modeled native volume and represents sources accounting for 84% of selected reference record mass. In an intact-week test, 41% of recovered articles matched native GKG records; field agreement on matched pairs is close to the processor baseline (tone MAE 1.69). The dataset supports retrospective content analysis and source-level aggregates; its tone and theme distributions differ from native GKG, and it does not support exhaustive counts. Data: https://doi.org/10.5281/zenodo.23964265.

0.20

Comparison of Common Crawl News & GDELT

cs.IR ·2026-09-30 ·Ameir El Ouadi, David Beskow
publishedhighly cited similarity 0.57
Abstract

The corpus of worldwide news is important for natural language processing, knowledge graphs, large language models, and other technical efforts. Additionally, this corpus is important for understanding the people, places, organizations, and events that interact in real-time every day. This paper compares two news datasets used for these tasks today, namely the Global Database of Events, Language, and Tone (GDELT) and Common Crawl News. Our research highlights the strengths and limitations of each dataset, analyzing their content and coverage. Notably, while GDELT relies on broadcasts, prints, and web news from across the globe, Common Crawl focuses on news sites from around the world gathered through web crawling. Our analysis revealed considerable differences in where the two datasets gather their news sources.

0.64

From UNDRR Reports to Event Records: Schema-Constrained LLM Extraction of Georeferenced Disasters

cs.CL cs.AIcs.IR ·2026-09-20 ·Camilla Andreozzi, Phuong-Anh Nguyen-Le, Zhijing Jin, Revati Mani
similarity 0.56
Abstract · 17 pages, 3 figures

Disaster-risk-reduction archives describe hazard events in prose that databases such as EM-DAT (Delforge et al., 2025) cannot ingest directly. We present an LLM pipeline that generates candidate georeferenced event records using a controlled hazard vocabulary and fixed schema, retaining evidence for review. Applied to 10,000 documents from PreventionWeb, the knowledge hub managed by UNDRR, it produced 3,572 records from 1,913 documents across 24 hazard types and resolved 81% of location mentions to OpenStreetMap geometries. On 171 human-positive document windows from a stratified 217-document reference set, GPT-5 achieved 86.0% pooled attribute $F_1$, versus 44.2% for the spaCy-gazetteer baseline. Evaluation pools hazard families, location strings, and event years within documents, without assessing their assignment to individual events. GPT-5.4 ranked highest among ten LLMs (86.6% $F_1$). Verbatim evidence occurrence was 72.0% for GPT-5 and 47.2% for GPT-5.4, measuring textual traceability without establishing attribute support. We report production failure modes and automated label and location-rule compliance checks. Prompts, schema, and outputs will be released for adaptation to national reporting archives.

0.34

When the Knowledge Base Becomes the Gold Standard: Measuring Resource-Shared Evaluation Loops in Entity-Level Machine Translation

cs.CL cs.DL ·2026-08-12 ·Jinhyung Bae, Dain Kil, Seongmin Oh, Seungmin Lee
code/data similarity 0.55
Abstract · 21 pages, 3 figures. Code and model outputs: https://github.com/nepersoned/malmoi-sjw-eval

The Seungjeongwon Ilgi, a UNESCO Memory of the World record, is only 37.4% translated, and the most conspicuous failure mode in automatic translation is the person name -- a misread name corrupts the historical fact rather than merely the surface. Low-resource historical domains have no expert gold standard for entity translation, so practitioners substitute a knowledge base (KB) for the gold. That KB is the same resource injected into the system: scoring becomes self-referential and the metric measures instruction compliance rather than translation quality. We measure this loop. Using expert person-name annotations from the National Institute of Korean History as a gold independent of the injection pipeline, we hold the entity set fixed and vary only the provenance of the correct reading. Of 527 expert-annotated mentions, only 31.1% lie outside the injection pipeline, and the residual loop is not uniform -- in the overlapping segment the injected reading agrees with the human translation 97.8% of the time against 70.1% in the independent one, so the segment that looks healthiest is the one the loop is holding up. Across four models, a difference-in-differences analysis shows the gain from KB injection is confined to the segment whose gold shares the injected resource; in the independent segment it is at or below zero. Post-injection preservation clusters in a narrow 0.910-0.996 band even though baseline capability differs fivefold, so the reported gain is the complement of prior performance and weaker models appear to improve more dramatically. On an independent sample built by removing the construction filter, the measure replicates within model (overlapping intervals) while discriminating between models (non-overlapping intervals) -- it reflects a property of the model, not of the sample.

0.30

Forecasting Corpus-Sampling Disagreement in Masked Text Reconstruction

cs.CL cs.LGstat.ML ·2026-10-09 ·Hare Krishna
similarity 0.54
Abstract · 12 pages + Appendices

If a second text corpus were collected from the same population, how much would a predictor's outputs change? We study this question for masked-text reconstruction using a smoothed predictor built from nearby token-pair counts and evaluated over the full 50,257-token GPT-2 vocabulary. Treating each source document as one observation, we derive a first-order formula for the expected disagreement between predictors fitted to two independent corpora. The formula accounts for variable document length, random context counts, and correlations across token offsets, and it can be evaluated from a separate reference corpus without access to either target corpus. On TinyStories, WikiText-103, and CNN/DailyMail, growing each corpus from 512 to 2,048 chunks of 64 tokens lowers disagreement by only 5--40\%, far short of the 75\% implied by $1/n$ scaling. The forecast tracks this slow decline, which we trace to smoothing of rarely seen contexts. Forecast accuracy depends less on how these correlations are modeled than on the size of the reference corpus: small references systematically underpredict disagreement. A source-split jackknife that uses only reference data reduces this calibration error by 57--59\%. After this correction, a simpler smoothed-multinomial approximation is most accurate when the reference corpus is smaller than the target corpus, and a nonlinear source bootstrap is most accurate when it is larger. Finally, small masked diffusion language models trained on the same corpora show that the count-based forecasts partly identify which inputs are most sensitive to the corpus, but miss the neural models' much faster decline in disagreement.

0.23

Linking Speakers of the German Parliament to Wikidata: Scope and Coverage of Metadata

cs.DB cs.SI ·2026-09-16 ·Thomas Haider, Arne Cypionka, Maximilian Teich
published similarity 0.54
Abstract

This paper links all individuals who spoke in the German Bundestag between 1949 and 2021 to Wikidata, creating a longitudinal dataset that connects parliamentary speech transcripts with structured biographical metadata. We evaluate the coverage, composition, and potential biases of the retrieved properties and statements, with particular attention to gender, professional background, historical legacies, and transnational dimensions such as place of birth, languages, and foreign awards. The results demonstrate both the analytical potential of combining GermaParl with Wikidata and the importance of critically assessing uneven metadata coverage in open knowledge graphs.

0.47

WPTokens: A Dataset for Studying the Regulation of Wikipedia

cs.SI cs.CY ·2026-10-05 ·Léo Joubert
similarity 0.54
Abstract

Studying how Wikipedia is regulated requires knowing who adds, rewrites, moves and deletes which piece of content, when and where. WPTokens is a dataset that answers these questions over the full revision history of the French-language Wikipedia. It relies on a positional diff over two layers: a text layer made of sentences, and a node layer made of the structural elements of wikitext (templates, links, references, headings, tables, lists, etc.). Each piece of content keeps a stable identifier for its whole life, and every change to it is recorded as an addition, an update, a move, a move with update, or a deletion. Built from the April 2026 dump, the dataset covers 2.75 million articles, 153.6 million revisions, 342.6 million tokens and 573.7 million token events, stored in an indexed SQLite database. A companion R package turns these records into temporal contributor--token multigraphs, completed by a layer of revert ties between contributors. We describe the tokenization, the diff algorithm, the data model and the graph construction, and discuss the limitations of the approach.

0.04

AI in the Clinical Documentation Loop: Effects of LLM-Induced Note Bloat and Diagnostic Overcall on Downstream LLMs

cs.CL cs.AI ·2026-11-07 ·Jinghui Liu, Anthony Nguyen, Taylan Gurgenci
similarity 0.53
Abstract · ML4H 2026 - Findings

Large language models (LLMs) are increasingly used to process medical records for healthcare tasks such as clinical documentation. Because these tools also generate documentation, the relationship is now bidirectional: models both read from and write to the record. Understanding how LLMs behave when reading records that already contain LLM-authored content is therefore critical for their safe deployment. In this study, we examine how LLM-authored content in the medical record affects downstream LLMs that read the same record. We consider two types of LLM-induced artifacts: bloat, in which LLM-generated summaries accumulate in the record and change the length and provenance of its content over time; and distortion, in which LLMs subtly alter clinical findings rather than fabricating facts outright. As a case of distortion, we study diagnostic overcall (documenting an unconfirmed differential diagnosis in the record) and test whether it propagates to the reader LLM's output. Across seven reader models from four families, we find evidence of both effects. First, LLM-generated summary notes modestly but consistently degraded downstream hospital-course generation compared with copy-forward and templated text of matched length. Second, injected overcalls reappeared in downstream outputs at rates that varied by reader model, without detectable changes in standard quality metrics. These findings show the need to monitor AI-generated content in medical records and to account for it when those records are later processed by AI.

0.13