CS Paper Radar

ExeBench-FHD: Addressing Fidelity Issues in Decompiler Evaluation

Akarsh Ashok, Dinil Mon Divakaran, Vivek Balachandran

cs.SE cs.LGcs.PL ·submitted 2026-10-20 13:49 UTC · v1

Abstract

ExeBench is one of the most widely used benchmarks for evaluating LLM-based decompilers. We analyze ExeBench and identify nine issues that affect how faithfully a test measures a decompiler. Three of them prevent LLM decompilers from recovering C code from ExeBench's function. These are ambiguous parameter order when integer and floating-point types are mixed, unused parameters that optimized assembly may drop, and constants stored outside the function's assembly code. Six more affect test validity: non-void functions without a return, placeholder types, static or inline functions removed by the optimizer, functions named main, trivial bodies and truncated definitions. We present ExeBench-FHD, a backward-compatible revision of ExeBench that adds 16 new fields: call-site assembly, data sections, test wrappers, linked-binary disassembly and five warning flags. In ExeBench, 38.9% of functions have at least one of the five flags; dropping records with any of the four warning flags (all except unused_param, which marks functions that the new wrappers make evaluable) keeps 63.9% of the corpus. In an evaluation at -O2 on four architectures, the new fields raise GPT 6 Sol's re-execution rate by up to 26.5%, 19.1% and 21.0% on functions that read data sections, mix parameter types and have unused parameters, respectively.

Closest papers

Nearest neighbours by meaning across the whole index.

When LLM Decompilers Recompile More and Preserve Less

cs.CR cs.AI ·2026-09-04 ·Chang Liu, Edward Raff, Kristopher Micinski
similarity 0.72
Abstract

Decompilation recovers high-level source from compiled machine code and serves as a foundation for security tasks such as vulnerability detection and malware analysis. Traditional decompilers like Ghidra and Hex-Rays expose whatever they cannot resolve as visible placeholders and often emit pseudocode that will not compile or execute; LLM-based decompilers produce clean, idiomatic C and are now judged almost entirely by recompilability and re-executability: whether the output builds and passes its shipped input/output tests. We show that these metrics can reward the wrong path: a function may recompile and pass every shipped test yet diverge on other legitimate inputs, and a disclosed vulnerability may disappear from the recompiled code with no visible trace of the crash. Neither failure is caught by existing suites. To address this gap, we propose Decompile-Diverge, a behavioral comparison oracle not relying on fixed or hand-crafted tests: for each function it synthesizes a driver, grows a fuzzing corpus from the reference, and reruns the decompiled code on the same inputs to detect changes in the function's behavior. Across eight systems in nine configurations on established LLM decompilation corpora, candidates that pass every shipped test still diverge from the original on our input corpus: 4.9% overall, and as many as 13% for a single system. On 300 real GitHub library functions and 287 CVE-grounded functions, recompilability and behavioral agreement can come apart: the strongest refinement LLM lifts Ghidra's build rate from 75% to 90%, while its Matched rate falls from 74% to 62%; on disclosed vulnerabilities, up to one tenth exhibit Crash Absence in its output. Source-level analysis traces this divergence to introduced fields, types, callees, and guards that replace the visible unknowns traditional tools leave behind.

0.38

Revisiting Re-executability: Coverage-Guided Symbolic Feedback to Improve Decompilation

cs.SE ·2026-11-13 ·Yuxin Hu, Xiaohan Wang, Kevin Leach
similarity 0.72
Abstract · 21 pages, 6 figures. Yuxin Hu and Xiaohan Wang contributed equally

When source code is unavailable, decompilation provides a source-level representation of a binary. However, code recovered by conventional or LLM-assisted decompilation may not preserve the original behavior of the binary. Recent approaches evaluate re-executability using fixed test suites, but passing these suites does not ensure behavioral preservation. We present SymDec, an LLM-assisted decompiled-code repair system that uses the original binary as a behavioral reference. SymDec uses coverage-guided symbolic execution to explore the I/O-level behavior of the original and recompiled binaries, and symbolic comparison to identify inputs that expose behavioral differences. These inputs are replayed on both binaries to confirm the differences, which are returned as structured feedback to guide subsequent LLM revisions. We evaluate SymDec on 120 functions from ExeBench and DecompileBench, compiled for x86-64 and ARM64 at optimization levels O0-O3, yielding 960 binary decompilation tasks. More than half of the tasks yield candidates that pass the supplied test suite and complete symbolic comparison without detected behavioral differences. Among candidates that pass all supplied tests, symbolic comparison exposes replay-confirmed behavioral differences in 4.5% of cases, showing that test-suite re-executability can miss behavioral discrepancies. Under the same input-generation budget, symbolic execution achieves higher mean line and branch coverage than black-box fuzzing. Symbolic feedback also improves the conventional metric, increasing the mean single-step test-pass rate from 90.1% to 93.3%. These results show that symbolic comparison provides stronger evidence of behavioral consistency than test-suite re-executability alone and supplies effective feedback for repairing decompiled code.

0.21

LOHRE: Execution-Grounded Type Recovery via LLM-Orchestrated Hybrid Analysis

cs.SE ·2026-11-11 ·Hao Jiang, Yujie Yang, Tao Li, Chenhui Cui, Liangfeng Xu +4 more
similarity 0.69
Abstract

Documenting stripped binary functions for security analysis requires recovering function interfaces and user-defined type (UDT) layouts from machine code. Existing large language model (LLM)-based methods face two limitations: they lack execution-based validation to detect inferred UDTs inconsistent with actual memory layouts, and may overlook evidence about shared UDTs across functions without direct call relationships. We present LLM-Orchestrated Hybrid Reverse Engineering (LOHRE), a framework that treats inferred UDTs as falsifiable hypotheses and tests them against program execution. LOHRE uses a hypothesize-validate-refine loop to test candidate layouts through paired dynamic slice emulation of baseline and mutated inputs. To reduce the cost of processing lengthy execution histories, the Execution-Depth Heuristic (EDH) compresses paired traces into an execution-depth difference and a summary of the tested field's observed uses. A cross-function knowledge base aggregates compatible UDT hypotheses and field-level confidence scores to guide testing and hypothesis refinement. We evaluate LOHRE on RealType at four optimization levels and a Windows remote procedure call (RPC) dataset. LOHRE achieves an average UDT Structural accuracy of 57.85% across RealType's four optimization levels and 35.6% on Windows RPC, outperforming the evaluated static baselines. Disabling dynamic validation reduces accuracy to 19.28% on RealType and 9.2% on Windows RPC. Removing the cross-function knowledge base reduces accuracy to 41.03% on RealType and 10.0% on Windows RPC. Replacing EDH's compressed feedback with full execution traces reduces recovery accuracy while increasing token use by 3.7 times and runtime by 3.0 times. These results indicate that execution-based validation, compact feedback, and cross-function evidence contribute to accurate and efficient UDT recovery.

0.30

CHISEL-ing Back Source Code with AI-enabled Iterative Recovery

cs.CR cs.SE ·2026-08-28 ·Varun Kohli, N Raghava, Biplab Sikdar, Dinil Mon Divakaran
top venue similarity 0.68
Abstract · Accepted for publication in ACM CCS SURE 2026

Decompilation aims to recover high-level, compilable, and semantically equivalent code from binaries. Traditional decompilers produce pseudo-C that is difficult to read and does not compile, while the recent LLM-assisted approaches generate readable, but semantically incorrect code. LLM-aided iterative recovery is an emerging branch of research, but prior works rely on supplied test suites for semantic recovery. In this work, we present CHISEL, a test suite-free framework to iteratively recover source code from Ghidra-derived pseudo-C. CHISEL uses simple yet effective feedback from a compiler (static analysis) and a coverage-guided fuzzer (differential analysis), augmented by rich observables for grounded divergence detection and feedback, cross-iteration divergence memory, and best candidate retention. We systematically evaluate CHISEL for compilation and semantic recovery, feedback oracle soundness, and iteration overhead on 120 ExeBench functions compiled for the x86-64 architecture, across four optimizations (O0-O3), in both stripped and unstripped variants, using the open-weight Gemma4:31b LLM. CHISEL, with all recommended features, achieves an average of 96.1% re-compilability and 79.8% re-executability rates at an average of 2.1 iterations. Significantly, CHISEL recovers 26% of first-generation execution errors. At the same time, CHISEL feedback oracle falsely accepts only 9.4% candidates. Lastly, CHISEL performs significantly better than two recent prior work on LLM-assisted decompilation.

0.54

DeCompCert: A Verified Decompiler for CompCert-Generated AArch64

cs.PL cs.CR ·2026-11-13 ·Swarn Priya, Basavesh Ammanaghatta Shivakumar, Thomas Gilray, Kristopher Micinski
theory similarity 0.66
Abstract

DeCompCert is a Rocq-verified decompiler for AArch64 assembly generated by CompCert. It reconstructs CompCert C through nine reverse passes, combining deductively verified transformations with translation validation. Untrusted OCaml and Datalog analyses propose candidate reconstructions, while verified validators accept only candidates that satisfy the required semantic correspondence. The composed correctness theorem guarantees that every defined, safe behavior of an accepted C reconstruction is matched by the input CompCert assembly; under a safety assumption, the two programs are observationally equivalent. We evaluate DeCompCert on approximately 2.16 million lines of C. With exact function and call signatures, it reconstructs and recompiles all 3,173 eligible contexts under both unoptimized and optimized CompCert pipelines. Without retained signatures, its cumulative inference portfolio validates 2,606 of 3,164 contexts (82.4%), while the platform-plus-IDA configuration validates 2,407 contexts (76.1%). Native execution and output-quality experiments further evaluate the generated C and characterize the remaining limitations in signature recovery and source resemblance.

0.25

Recompilation Is Not Enough: Test-Guided Decompiled-C Repair

cs.SE cs.AI ·2026-09-07 ·Yuhan Huang, Puzhuo Liu, Jianlei Chi
similarity 0.66
Abstract · 5 pages, 2 figures

Decompiled C often becomes recompilable only after repair, but recompilation alone does not establish test-observed behavior. A recompiled command-line binary can still parse options incorrectly, print different bytes, or return a different exit status. We present a few-step workflow for repairing decompiled C using compiler feedback and related official tests. Compiler and linker diagnostics first guide build repair. Once the repaired C recompiles into a binary, smoke checks and related official tests expose behavioral discrepancies for semantic repair. In a preliminary static-enriched evaluation on 104 Coreutils 9.5 binaries with available decompiler exports and deterministic exact-output smoke comparisons, 91 binaries (87.5%) recompile and pass the test gate; 9 do not recompile within the repair budget, and 4 recompile but still fail the test gate. The result suggests that test-gate feedback can make LLM-assisted repair of decompiled C more auditable than compile-only recovery.

0.07

STILL: Recovering Lowered STL Semantics for LLM-assisted C++ Decompilation

cs.SE ·2026-08-26 ·Xiaohan Wang, Kevin Leach
top venue similarity 0.65
Abstract

LLM-assisted decompilation improves readability and re-executability, but still underperforms on stripped C++ functions that use the Standard Template Library (STL). Compilation, optimization, and symbol stripping remove or obscure source-level semantics such as container types and library-call structure, while traditional decompiler output often fails to recover them. We present STILL, a structured semantic interface that predicts function-level STL container semantics from stripped control-flow graphs and renders them as compact hints for LLM refinement. On StlBench, STILL predicts common container-level STL semantics, with the strongest cross-dataset results for stable string and vector slices. On stripped HumanEval decompilation, these hints enable DeepSeek-chat refinement to reach 28.4% executability, compared with 17.4% for no-hint refinement and 8.9% for raw Ghidra decompilation; hint utility is downstream-backbone-dependent, with decompilation-specialized models requiring lightweight adaptation to benefit from the same interface.

0.41

InSPECtor: Improving SLEIGH Processor Specification Veracity via Proxy

cs.CR cs.PL ·2026-08-13 ·Michael Chesser, Paul Quirk, Douglas Cooke, Guy Farrelly, Surya Nepal +1 more
publishedcode/data similarity 0.64
Abstract · Extended version published at USENIX Security (USENIX Security 2026). Code available at: https://github.com/Sleigh-InSPECtor/

Processor specifications underpin critical security and program- analysis tools such as disassemblers, decompilers, and emulators, yet, their correctness is rarely examined. Errors in specifications distort program behaviour, obscure vulnerabilities, and enable analysis-evasion techniques. Validating processor specifications is a non-trivial task. Our study is a significant undertaking to enable, for the first time, the systematic validation of open-source SLEIGH language specifications, predominantly used by Ghidra. We design and implement a testing framework based on an automated oracle validation strategy by proxy. Our approach leverages the structure encoded in a specification itself to enumerate decodable instruction forms and generate targeted initial states. Then differentially test the successful decoding and emulation of those instructions by comparing emulators exercising the processor specification against hardware references. Applying InSPECtor across diverse, open-source specifications---x86-64, AArch64, ARM/Thumb, RISC-V, MSP430---embedding differences in specification styles, author preferences, and instruction set architecture designs, we uncovered over 38,920 discrepancies that led to 125 unique bugs with proposed fixes, identifying decoding and semantic defects as well as cross-vendor inconsistencies. We distill our findings into 8 concrete recommendations to drive future improvements. Our work underscores the importance of specification correctness and provides a practical tool to substantially improve the fidelity of SLEIGH processor specifications, strengthening the reliability of downstream security and analysis tools.

0.47