CS Paper Radar

Who Counts as Exposed? Measurement Choices in an Open, Re-estimable Index of Occupational AI Exposure

Luca Mouchel, Pierre Bouquet, Yossi Sheffi

cs.IR ·submitted 2026-11-13 13:41 UTC · v1
benchmark

Abstract

Occupational AI exposure indices are used as inputs to economic analysis, workforce planning, and policy, and the choices built into them decide which workers are described as exposed. We build an open, re-estimable task-level index and vary its two main design decisions, context and calibration, to show how much the answer depends on them. For each of the 18,796 occupation--task pairs in \onet, seven open-weight reasoning models rate exposure under a common rubric. Each model rates with or without a context of retrieved news and research passages, workplace descriptors, and a bottleneck-based reasoning protocol, and the ensemble is combined with or without simplex-constrained calibration to observed Claude usage. The largest difference from the dominant 2023 benchmark is not a disagreement about what AI can do but about how much credit to give exposure that depends on software integrations not yet built, a choice that no data can settle. At the benchmark's own weight, the calibrated index counts fewer exposed workers than the benchmark, and when only direct exposure counts it counts more. At full score resolution, the ensemble ranks tasks better than the benchmark against observed usage and an independent physical-task classification. Adding context mainly shifts the level of exposure rather than improving the ranking. Context and calibration both relocate exposure toward higher-wage occupations, and calibration, which anchors the index to one platform's users, does more of it, so we also release the uncalibrated arm. Exposure indices used as policy inputs should state such choices, report their numbers under the alternatives, and be re-estimated as systems change.

Closest papers

Nearest neighbours by meaning across the whole index.

Who Delegates to AI? Evidence from Agent Configurations in Github

cs.AI cs.CY ·2026-08-19 ·Hyeongjae Lee, Jihyang Cheon, Lanu Kim
similarity 0.67
Abstract

A growing body of literature measures the extent to which occupations are exposed to AI, yet existing measures capture where AI could perform tasks rather than whether workers have actually adopted it. We introduce a distinct tier of exposure, delegated exposure, which records whether a worker has committed a task to AI by embedding it into a structured workflow. We operationalize this concept through the Agentic Adoption Index (AAI), measuring how closely an occupation's tasks align with the agentic routines that practitioners have built and shared. Using semantic embeddings of roughly 888,000 agent skill specifications from public GitHub repositories, we compute their similarity to nearly 18,000 O*NET task statements and aggregate these scores to the occupational level. We present three main findings. First, the occupations where task delegation concentrates differ sharply from those identified as most vulnerable by pre-AI automation frameworks. Second, the AAI aligns more closely with measures of technical capability than with measures of current conversational LLM use. Third, for occupations requiring a bachelor's degree or less, the AAI increases alongside average wage levels; however, this relationship reverses for occupations requiring a master's degree or higher, where adoption declines among higher earners. These patterns replicate on an independently collected corpus of agent skills from the Manus Skills Marketplace. This lower adoption among highly educated, high-earning workers may reflect tasks that inherently resist advance specification or professional discretion over the pacing of workflow codification. Distinguishing these mechanisms will require longitudinal measurement.

0.38

Dull, Dirty, Dangerous? Working Conditions and the Limits of Occupational AI Exposure

cs.AI cs.HCecon.GN ·2026-11-04 ·Siyi Yu
similarity 0.62
Abstract

Automation is often justified as relief from dull, dirty, and dangerous work. I examine how these conditions relate to occupational AI exposure using 21 O*NET items across 754 US occupations. Higher exposure is associated with lower condition scores under a large language model (LLM) measure and AI Occupational Exposure (AIOE), with equal occupation weights and employment weights. Adjustment for physical demands, required education, training and experience, and wages reduces the composite association. The adjusted confidence intervals include zero for LLM exposure and remain below zero for AIOE. Separating repetition from restricted discretion also changes what the combined Dull score conveys. These comparisons show how item groupings and occupational characteristics shape the interpretation of exposure estimates. Occupational profiles offer a starting point for selecting contrasting research settings. Establishing desired automation or realized relief requires evidence about workers' preferences and specific interventions.

0.27

GDP.xlsx: Benchmarking Frontier Models on Professional Workbooks

cs.CL cs.AI ·2026-11-13 ·Suhaas Garre, Sushant Mehta, Edwin Chen
publishedbenchmarkcode/data similarity 0.62
Abstract · 14 pages, 4 figures, 6 tables. Accepted to the Agent Evaluation Science Symposium (New York City, November 2026). Leaderboard: https://surge…

We introduce GDP_xlsx, a benchmark of 70 realistic workplace requests written by professionals in 13 domains, each paired with an expert rubric of binary criteria and with the complete workbook or CSV file it refers to. Together the files hold 2.18 million populated cells, and the rubrics 791 criteria. Agents work in a container with spreadsheet tools, and an agentic judge that can open the file applies the rubric. Completeness and exclusivity criteria require set-valued answers to be found in full and with nothing extra. We rank models by strict pass@1, the share of attempts that meet every criterion. Across 20 model configurations, including frontier models from Google, Anthropic, and OpenAI, the top-scoring one, Gemini 4 Argon, passes only 38.3% of attempts, and the median score is 19.3%, leaving wide headroom. We release the evaluation harness and three public tasks; the other 67 are held out to protect the leaderboard from contamination.

0.50

AI Exposure and the Changing Responsibilities in Job Postings

cs.CY ·2026-11-06 ·Dora Zhao, Lin Ai, Pranav Khadpe, Siddharth Suri, Scott Counts
similarity 0.62
Abstract

The growing adoption of artificial intelligence (AI) in the workplace has raised questions about how employer demand for different work activities is changing. We examine how the prevalence and content of work activities in online job postings change in association with activity-level measures of AI exposure, derived from Copilot and Claude usage. To enable systematic comparisons, we develop a pipeline that extracts the responsibilities from job advertisements and maps them onto O*NET's standardized taxonomy of Intermediate Work Activities (IWAs). We then apply this approach to 50,000 job advertisements from 33 firms between 2023 and 2025. From this analysis, we first find that explicit mentions of AI increased by 66%, although much of this growth occurs outside descriptions of the actual work activities associated with a job. Looking specifically at the advertised job responsibilities, activities with greater AI exposure tend to decline in advertised prevalence, but only mildly as a share of the total number of responsibilities. At the same time, more AI-exposed work activities increasingly are associated with explicitly AI-related responsibilities, suggesting that employers are looking to hire people who can perform these responsibilities with AI.

0.21

How exposed is work to robotics? Occupational, industry, and geographic measures

cs.RO cs.CYecon.GN ·2026-10-16 ·Sebastian Sartor, Robert Seamans, Neil Thompson
theory similarity 0.60
Abstract

Exposure indices that map technical capabilities onto occupational abilities have become a standard tool for anticipating how AI may reshape work, but existing measures either cover software AI or proxy robotics through patents and installation data that reflect the industrial robots of past decades. We construct the first abilities-based exposure measures for robotics and physical AI, and the first to span occupations, industries, and geographies. Following the framework of Felten et al. (2018, 2021), we build a robotics capability taxonomy of 16 capabilities in eight domains from 78 academic, industry, and standards sources, use a large language model to score the relatedness of each capability to the 52 ONET work abilities, and aggregate to Robotics Occupational Exposure (ROE) for 774 occupations, Robotics Industry Exposure (RIE) for 250 NAICS industries, and county-level Robotics Geographic Exposure (RGE). We find that robotics exposure concentrates in physically intensive work such as construction, farming, transportation, and production, and in industries such as logging, mining, trucking, and warehousing. Unlike generative-AI exposure, ROE is higher in occupations with lower wages, lower education requirements, and less creative-ability content. Physical collaboration and assistance, terrain-based locomotion, grasping and handling, and task and functional reasoning move the exposure ranking most, yet no single capability carries the index. Because the measures capture technical relatedness rather than readiness, cost, or adoption, they should be read as an upper bound on where physical automation could affect work.

0.17

Knowledge or Calculator? Decomposing the Skill Premium in Verifiable Financial Agent Workflows

cs.AI ·2026-10-02 ·Jermyn Zhen Yong Bek, Zhuang Qiang Bok, Zhongtian Sun
similarity 0.58
Abstract · 10 pages, 1 figure, 9 tables

Financial AI agents must do more than retrieve facts: investment workflows require correct quantitative execution, reliable use of procedural resources, and auditable structured outputs. We introduce FinSkillBench, an evaluation suite of 2,603 point in time episodes across 12 subtasks in portfolio construction, risk management, and fundamental analysis, with hidden regenerable ground truth and task specific deterministic verifiers. Executing 17,820 episodes across 9 models and 3 resource conditions, the paired analysis across 8 models shows that curated skill packages raise mean scores by +16.2 points (0.366 to 0.528), whereas skills generated within a single episode add only +0.5 points while consuming more tokens and turns. We then decompose the curated premium by granting human authored procedural documents and executable domain tools separately: documents alone add +5.6 points, tools alone add +19.5 points, and their combination is subadditive. The premium is strongly workflow dependent: executable tools dominate numerically intensive workflows, documentation matters more when procedural or output schema guidance is the bottleneck, and interpretive tasks benefit from both. The effects are sign stable across 10 scoring variants and cluster bootstrap analyses, and an independently implemented second harness reproduces the directional pattern while showing that effect magnitudes depend on how tools and data are exposed. Overall, a measured "skill premium" is a property of the full model, resource, and harness system rather than of the underlying model alone.

0.51

What Effort Level Should I Choose? On the Economics of Reasoning Effort in LLM Agents

cs.AI cs.CL ·2026-10-25 ·Kaiser Sun, Yongchan Kwon, Zain Hasan, Shang Zhu, Kaitlyn Zhou +1 more
similarity 0.58
Abstract

Generative models increasingly expose a reasoning effort setting that lets users scale inference-time compute, trading higher cost for potentially improved performance. Users must choose an effort level before knowing whether extra reasoning will help, and even a successful run does not reveal whether the effort purchased was necessary. Reasoning effort is therefore a credence good: buyers observe the outcome but cannot assess the marginal value of the computation they paid for. In this work, we measure the counterfactual effect of effort levels on both proprietary and open-weight models on four benchmarks, spanning single-call reasoning and long-horizon agentic tasks. Effort, it turns out, does not consistently preserve tasks that lower effort solves, can overspend, and can change trajectory behavior. First, we find that although higher effort can raise accuracy in aggregate, it frequently fails on tasks that lower effort solved. Second, on the subset of easy tasks that every effort level solves, higher effort is generally more costly, with the highest effort level costing up to 42x as much as the lowest. Lastly, effort changes not only how many tokens are used, but also how agents solve tasks: at higher effort, agents verify their own work more heavily and more often search for solutions than work through problems directly, both of which lengthen trajectories without necessarily altering the outcome. These findings show that naively choosing the highest level is not only cost-inefficient but also offers no guarantee of the best instance-level performance. A simple alternative, starting at the lowest effort and escalating only after a verified failure, matches or improves on fixed top effort across all models and benchmarks at lower cost for most proprietary models.

0.35

Balance of Benchmarks: Semantic Density Reweighting for Task-Conditioned Model Comparison

cs.AI cs.CLcs.LG ·2026-08-30 ·Jhen-Ke Lin, Hong-Yun Lin
benchmark similarity 0.58
Abstract · 65 pages including references and appendices. Expanded evaluation with WildScores, a collection of 148 developer-reported benchmarks

Model comparison increasingly relies on large collections of publicly reported benchmark scores, yet common aggregation strategies trade off evidence coverage against control over capability weighting. Manually curated suites leave potentially informative evaluations unused, while uniform averaging retains them but gives greater influence to capabilities that happen to be benchmarked more densely. We introduce Balance of Benchmarks (BoB), a framework that retains eligible benchmark evidence while adapting its influence for task-conditioned model comparison using only public aggregate scores. BoB combines semantic density weighting, score equating across benchmarks of different difficulty, and task-relevant residual pooling. We evaluate it on 605 configurations across 14 Artificial Analysis benchmarks and on WildScores, a collection of 148 developer-reported benchmarks evaluated with held-out source-lineage families. On WildScores, BoB-Support raises family-mean Spearman correlation from 0.764 under uniform standardized averaging to 0.823, reduces MAE from 6.19 to 5.10 normalized score points, and increases three-model shortlist hit rate from 65.3% to 72.6%. BoB-Constant reaches a Spearman correlation of 0.831 and a hit rate of 74.6%. Separately, density weighting reduces average ranking changes when benchmarks are repeated, including as paraphrased copies. BoB-Support also reduces retrospective three-model shortlist regret from 2.08 to 1.67 normalized score points. BoB makes benchmark inclusion, redundancy, and task relevance explicit and testable measurement choices, allowing existing benchmark evidence to be used more fully while moderating the influence of benchmark proliferation.

0.24