D·1
Media
The media agenda since 1946 — front pages, index samples, and television news, all coded with the same topic system. The domain this site's own media data set will extend with continuous, AI-classified coverage.
One codebook, applied consistently
Data & methods
One category scheme, applied to everything. That codebook is what makes a bill, a hearing, a front page and a budget line comparable — and it is what the project has been applying, document by document, for decades. Below: the scheme, the data that already exists, and how the coding gets done — first by people, then by machines trained on their work.
New to the scheme? For students and scholars defines the terms. Reusing any of this? For researchers carries the license and the citation.
01
The codebook
A single hierarchy of policy topics — 21 major topics and some 220 subtopics beneath them. Every document in every venue gets exactly one major topic, chosen by the same rules whether it is a Senate hearing or a newspaper front page. That constraint is the whole point: it is what lets attention be compared across institutions and across decades.
01Macroeconomics02Civil Rights03Health04Agriculture05Labor06Education07Environment08Energy09Immigration10Transportation12Law and Crime13Social Welfare14Housing15Banking, Finance, and Domestic Commerce16Defense17Technology18Foreign Trade19International Affairs20Government Operations21Public Lands23Culture24State and Local Government Administration
21 major topics, each opening into finer subtopics. The same list codes a Senate hearing, a party platform and a front page — which is what makes them comparable.
A newspaper also prints obituaries, sport, weather and advertising, and none of those is a policy topic. For media sources the coding therefore adds a media extension — 18 further codes, 24 to 41, plus a “no policy content” outcome — so such items can be classified rather than pushed into a policy category they do not belong in. The race-state news collections on this site use a simpler rule: the classifiers have a single “no policy content” outcome (code 998) in place of the separate extension codes. Those articles are deliberately left out of every policy figure, which is why the topic shares on this site are shares of policy coverage.
02
The data
Most of it already exists, and every file below downloads straight from this page. 10 domains carry two dozen downloadable series between them — Gallup's Most Important Problem back to the 1940s, congressional bills, hearings, public laws and roll calls, State of the Union addresses and executive orders, both party platforms, Supreme Court cases, and federal budget authority with outlays reconstructed to 1791.
D·2
New York Times
The paper of record as an agenda measure — the front page, the index sample, and the weights that make them representative.
D·3
Social groups
Attention to, and representation of, social groups in US coverage — this site's second data build.
D·4
Public attention (Trends)
Google Trends search interest as a public-attention measure across policy topics — this site's third data build.
D·5
Most Important Problem
Gallup has asked Americans the same question since the 1940s: what is the most important problem facing the country? The MIP series codes every answer into the Policy Agendas topic system — the longest-running measure of the public's agenda.
D·6
Parties
What the parties promise: both platforms coded statement by statement since 1948.
D·7
Legislative
Congress at work: every bill, hearing, public law, and roll-call vote, topic-coded across seven decades.
D·8
Executive
The presidential agenda: State of the Union addresses coded quasi-sentence by quasi-sentence, executive orders, and veto rhetoric.
D·9
Judiciary
The Supreme Court's agenda, case by case, across the twentieth century and beyond.
D·10
Budget
Where the money actually goes: budget authority by function back to 1947 (with outlays reconstructed to 1791), adjusted for inflation and crosswalked to the Policy Agendas topic system.
All 10 domains and their series →
02b
What this site adds
Three new builds extend the same scheme to sources the project has not carried continuously. Analytics only — no article text is republished.
D·1
In preparationMedia agendas
Policy-agendas coding of US media coverage — the lead data build. Topic shares, attention over time, and outlet comparisons across the coverage.
D·2
Design stageSocial groups
Attention to, and representation of, social groups in US coverage — built on a dedicated dictionary and category scheme.
D·3
Design stagePublic attention
Google Trends search interest as a public-attention measure across policy topics, alongside the media agenda.
03
Human coding — the standard
The prevailing approach in this field, and the foundation of every Policy Agendas series: people trained on a codebook, reading documents and assigning topics.
The codebook
Every document is read and assigned one topic from the Comparative Agendas master codebook — 21 major topics and 220 subtopics, the same scheme used across the project family since the early 1990s.
Trained coders
Coders work from written coding rules rather than intuition, with difficult and boundary cases adjudicated against the codebook so that decisions are consistent across a corpus and across the decades it spans.
Double coding
A share of each corpus is coded independently by two coders, and intercoder agreement is measured and reported. Reliability is a property of the data set, not an afterthought.
The benchmark
Human-coded data are what everything else is measured against. They are the reference standard for this project, and the training and test material for anything automated.
04
Automated coding — the alternative at scale
Where a corpus outruns what a coding team can read, the same codebook can be applied by classifiers trained on human-coded examples — checked against human coding, never instead of it.
Where it applies
Some corpora are simply too large to code by hand — a continuous news feed produces more articles in a week than a coding team can read in a year. Automated classification extends the same codebook to that scale.
Trained on human codes
The classifiers learn from human-coded examples: the codebook and the coders define the categories, and the model reproduces those judgments on new documents.
Validated, not assumed
Classifiers are tested against human-coded examples before they are used on a new corpus. The race-state news on this site is coded by three equal votes: two fine-tuned XLM-RoBERTa classifiers (poltextlab english-cap-v5 and pooled-cap-v4, each giving a probability for every topic) and one open-weight language model, which names a single topic and is consulted only where the two classifiers’ averaged probabilities leave the leading topics close. Each topic’s score is the sum of the votes, and the highest score is the code. On the project’s CAPBENCH test, where the label is what four trained coders agreed on, this panel matches the coders on 88.2% of the easiest tier and 72.9% of the intermediate tier, against 84.6% and 69.2% for the two classifiers alone and 81.8% and 63.7% for the language model alone. That test uses congressional speeches, bills and New York Times articles, not state newspapers, and the intermediate tier is being re-scored on its final item set. Figures for this site’s own corpus have not been published; until they are, treat the automated coding as indicative and the human-coded archival series as the standard.
A complement
Automated coding does not replace human coding. It extends its reach, and human coding remains the standard that judges it.