CS Paper Radar

Spatial-Frequency Learning for Scanning Transmission Electron Microscopy Image Enhancement

Hesong Li, Ying Fu, Ziqi Wu, Ruiwen Shao, Tao Zhang

cs.CV ·submitted 2026-10-27 01:43 UTC · v1
top venuedataset

Abstract

Scanning Transmission Electron Microscopy (STEM) enables the observation of atomic arrangements at sub-angstrom resolution and plays a key role in the discovery and characterization of advanced materials. Raw STEM images are often noisy and blurred due to multiple degradations. Although recent deep learning methods improve STEM images, their performance remains limited under severe degradations. In this work, we propose a spatial-frequency learning framework for STEM image enhancement. Specifically, we design a spatial-frequency fusion network that exploits periodic atomic arrangements in the frequency domain and restores local atomic details in the spatial domain. Spatial- and frequency-domain features are enhanced separately and then fused. We also present a STEM noise calibration method for synthesizing more realistic STEM images. The parameters of scan noise, pointwise noise, and background noise are estimated through statistical analysis and fitting of real STEM images. Using these calibrated parameters, we construct a diverse dataset containing a wider range of atomic arrangements, imaging modes, and degradations. Random masks combine different atomic arrangements to simulate interfaces between multiple materials. Experimental results demonstrate that our method achieves state-of-the-art enhancement performance on synthetic datasets and real images of advanced materials while improving atomic localization accuracy.

Comment
Accepted by IEEE TPAMI

Closest papers

Nearest neighbours by meaning across the whole index.

Real-Time Atomic-Resolution Electron Phase Imaging without Probe Calibration via Ptychography-Supervised Learning

cs.CV cond-mat.mtrl-sci ·2026-09-22 ·H. Yue, C. -C. Chen, C. -N. Hsiao, J. Cheng, Y. Liu +2 more
similarity 0.67
Abstract · Submitted to Advanced Science on June 18, 2026

Atomic-scale phase imaging is central to resolving defects, interfaces, and weakly scattering atoms that govern the behavior of nanoscale materials. Electron ptychography delivers sub-ångström phase sensitivity but remains an offline technique, because its iterative reconstruction is computationally expensive and sensitive to experimental calibration, preventing live use during data acquisition. Here, a ptychography-supervised local inference framework is presented that converts four-dimensional scanning transmission electron microscopy (4D-STEM) into an acquisition-compatible phase-imaging workflow. Physics-constrained reference phase maps reconstructed from a single experimental AuPd dataset serve as teacher labels for a compact model that predicts local phase patches directly from diffraction measurements, without explicit probe input or online iterative optimization. Full-field images are assembled by deterministic overlap stitching. The workflow reaches an online latency of about 0.27 ms per probe position and a throughput of about 20,000 positions per second, an approximately 1,000-fold speed-up over GPU-accelerated ePIE, while preserving atomic-scale lattice contrast and reciprocal-space fidelity. Without fine-tuning, the same model transfers across materials (WS2), defocus conditions (high-entropy alloy nanoparticles), and instruments (hBN at 300 kV). The approach amortizes ptychographic redundancy into a fast, generalizable workflow that enables real-time atomic-scale phase imaging for materials microscopy.

0.14

A Geometry-Guided Feature-Space Framework for Interpretable Unsupervised 4D-STEM Analysis of Complex Multiphase Materials

cs.AI cond-mat.mtrl-sci ·2026-11-07 ·Arda Genc, Daniel A. Bolden, Albert Nguyen, Daniel S. Gianola, Ravit Silverstein
benchmark similarity 0.64
Abstract

Rapid, high-fidelity analysis of four-dimensional scanning transmission electron microscopy (4D-STEM) datasets from heterogeneous materials remains challenging because coexisting phases, orientation variants, local strain gradients, and intrinsic TEM thin-foil effects produce complex, overlapping diffraction signatures. Here, we present a machine-learning (ML) framework that combines physics-informed geometric descriptors with weighted high-dimensional embeddings to identify latent structural patterns with minimal prior knowledge. By constructing a regularized descriptor space before embedding and density-based clustering, the method improves cluster stability, suppresses noise-driven fragmentation, and reduces sensitivity to embedding hyperparameters. Spatial weighting produced the largest stability gain, increasing median cluster persistence from 0.04 to 0.55 while reducing the noise fraction from approximately 22% to 0%. Rotation weighting increased DBCV from 0.80 to 0.92, although the noise fraction rose from 0.2% to 1.0%, highlighting a trade-off between improved density separation and noise rejection. We validate the framework on two benchmark systems: Si/Si$_x$Ge$_{1-x}$ epitaxial layers with continuous lattice variation and deformation, and multiphase Ti-Nb alloys containing distinct phases and orientation variants. Comparisons with a pretrained vanilla autoencoder and model-free pattern vectorization show that the physics-informed geometric descriptor provides higher cluster fidelity, greater interpretability, and substantially lower dimensionality.

0.24

AutoRASOR: Autonomous Rapid Scanning Electron Microscope Operator

eess.IV cond-mat.mtrl-sci ·2026-09-18 ·Kevin Zhang, Mohammad Taha, Rafael Espinosa Castañeda, Yutong Liu, Yin Zhu +5 more
similarity 0.63
Abstract

Scanning Electron Microscopy (SEM) is a foundational technique for characterizing material microstructure, which dictates many fundamental physical and chemical properties. With the rise of Self-Driving Labs (SDLs), samples are now synthesized in large batches, demanding equally high-throughput, autonomous characterization. Existing automated electron microscopy pipelines are task-specific: they detect pre-defined features, optimize known properties, or require prior knowledge of the sample, restricting each pipeline to the material system it was built for. We introduce AutoRASOR, a task-agnostic autonomous SEM pipeline that captures the multi-scale morphology of an unknown sample without domain-specific pre-training, fine-tuning, or human prompting. AutoRASOR embeds micrographs in real time with a vision foundation model (VFM), DINOv3, and selects regions of interest (ROIs) through two complementary policies: Latent Farthest Point Sampling (LFPS) and active learning on morphological ambiguity, which is the conditional variance of unresolved fine-scale features given lower magnification appearance. Active learning on ambiguity consistently captures more diverse and rare morphologies than random ROI selection, while LFPS reliably recovers the specimen's morphological distribution within a limited capture budget, tested both on real SEM micrographs and synthetic phase-field images. By shifting autonomous characterization from pre-defined, task-specific targeting of features to morphological survey, AutoRASOR generates information-rich multi-scale datasets for diverse downstream tasks. Without the need for domain-specific pre-training or prior material assumptions, this framework can be deployed out-of-the-box to characterize novel materials produced by SDLs.

0.31

A Digital Simulation Toolkit for Physics-Based Generation of Realistic Experimental Scanning Tunneling Microscopy Images

cs.LG cond-mat.mtrl-sci ·2026-09-29 ·Huanhuan Zhao, Laxmi Bhurtel, Connor Vernachio, Fahmy Paiziah, Wonhee Ko +1 more
similarity 0.61
Abstract · 17 pages, 6 figures in main text and 2 supplementary figures

Scanning Tunneling Microscopy (STM) is a widely used tool for characterizing surfaces of materials at the atomic scale, playing a crucial role in discoveries across condensed matter physics and materials science. Despite its extreme spatial resolution, STM is one of the most sensitive microscopy techniques and is highly prone to noise. While existing unsupervised denoising methods are very cheap to train, these are primarily focused on removing the noise with minimal recovery of key physical information. While supervised methods can offer superior performance, the major bottleneck is that a large amount of paired clean-noisy experimental images is required which are impractical to obtain. Thus, we developed a low-cost physics-driven digital toolkit to rapidly generate large volume of realistic STM images. Firstly, we simulate clean images from a chosen material system. Then, with prior knowledge of the physical characteristics of the artifacts and noise present in STM experiments, we formulate several artifact-noise functions such as Gaussian electronic noise, 1/f flicker noise, scan-line noise, background tilt and mechanical drift. These physically informed noise components are then added to the simulated clean images to generate realistic STM images. We demonstrated the capability of the proposed digital toolkit to generate AI-ready data for denoising images of the (111) surfaces of copper and lead, while preserving atoms, defects, and electron waves. We also validated the quality of the downstream image analysis of learning electron wave patterns induced by quantum interference from Cu(111) images. Results show that the supervised models trained on digitally generated AI-ready data can more effectively denoise and learn electron wave patterns on Cu(111) images than benchmarked unsupervised approaches, indicating that the proposed toolkit facilitate scientific discovery.

0.26

A benchmark dataset and baseline methods for four-dimensional STEM diffraction patterns

cs.CV ·2026-09-18 ·Yuyan Guan, Haoran Zhang, Zian Mao, Antong Yang, Caifei Li +5 more
benchmarkcode/data similarity 0.61
Abstract · 16 pages, 5 figures. Data and trained model weights: https://doi.org/10.57760/sciencedb.nbsdc.00281. Code: https://github.com/Gaiya69-rgb/4D…

Four-dimensional scanning transmission electron microscopy (4D-STEM) records a two-dimensional diffraction pattern at each electron-probe position, yielding spatially resolved reciprocal-space information but large, heterogeneous data volumes. Here we describe 4D-ImageNet, a collection of 174,000 diffraction patterns comprising 145,000 experimental patterns selected from 29 acquisitions and 29,000 multislice simulations. The experimental data cover acquisition-level labels for Ag, Au, mixed Au-Ag, CoO, Pd and ZnO specimens across multiple fields of view, scan dimensions, camera lengths and exposure times. Each acquisition contributes 5,000 quality-ranked patterns with source scan coordinates and acquisition metadata. A set-prediction detector provides model-derived pseudo-labels for the direct-beam position and Bragg-disk centres, with a confidence score for each disk. The simulation data cover 13 crystal structures and include Euler rotations, reciprocal-space sampling and approximate low-index beam directions. A grouped mixed-domain masked-reconstruction benchmark is provided to assess leakage-resistant loading and evaluation across experimental and simulated data. The dataset is intended for representation learning, disk detection, diffraction-pattern retrieval, orientation analysis and simulation-to-experiment studies.

0.41

ZoomSEM: accelerating large-scale brain connectomics with intelligent SEM super-resolution

cs.CV ·2026-10-24 ·Jiateng Shou, Zhiwei Xiong, Ruikang Xu, Zhicheng Wang, Peiyao Shi +8 more
similarity 0.60
Abstract

Serial electron microscopy (EM) is essential for large-volume biological imaging and connectomics, yet its throughput remains a major bottleneck, with cubic-millimeter-scale brain imaging requiring months or years. Existing deep learning-based EM super-resolution methods are limited to small upscaling factors or rely on synthetic data, offering modest acceleration and limited practical impact. We introduce ZoomSEM, a deep learning-based super-resolution method to dramatically accelerate SEM imaging. Trained on real low- and high-resolution scanning electron microscopy (SEM) pairs, ZoomSEM enables reliable recovery of 4$\times$, 8$\times$, and up to 16$\times$ higher-resolution images from fast low-resolution scans while preserving fine neuronal structures critical for connectomic analysis, achieving up to 30-fold effective acceleration. Extensive experiments across datasets of diverse biological samples demonstrate robust generalization under realistic low-resolution acquisition. This advance enables high-resolution, large-volume serial EM imaging at unprecedented speed, poised to transform connectomics and related fields.

0.24

Scalable Bayesian Optimization of Composite Functions for Image-Based Inverse Problems in Materials Characterization

cs.LG cond-mat.mtrl-sci ·2026-09-02 ·Dasol Yoon, Poompol Buathong, Chia-Hao Lee, Yujia Zhang, David A. Muller +1 more
similarity 0.60
Abstract · 28 pages. Dasol Yoon and Poompol Buathong contributed equally

Estimating physical parameters from scientific images is a common inverse problem in materials characterization that often relies on expensive physics-based simulations. In electron microscopy, specimen thickness and crystal mistilt are critical parameters that govern how electrons scatter through the sample, and therefore the accuracy of any atomic-scale structure recovered from it. They are commonly inferred by matching experimental position-averaged convergent-beam electron diffraction (PACBED) patterns to simulated ones, but grid searches scale poorly and neural-network methods require extensive pretraining that may not transfer to new conditions. Here, we propose scalable Bayesian optimization of composite functions (SBOCF), a simulation-efficient method that exploits the known composite structure of the image-matching objective and the intermediate information contained in simulated images. By representing PACBED images with patch-level summaries and two correction terms, SBOCF preserves the original pixel-wise objective while reducing the number of modeled outputs from 24,649 to 11. Under a budget of 50 simulator evaluations, SBOCF outperformed standard Bayesian optimization with expected improvement on synthetic SrTiO3 benchmarks with thick and thin specimens, reducing the median final SSE by up to 290 times in the thick-sample case. On experimental data, SBOCF produced parameter estimates consistent with previously reported values without task-specific pretraining. For a simulated mistilted specimen, using the SBOCF estimates in a downstream ptychographic reconstruction recovered sharp atoms that were otherwise blurred. These results establish SBOCF as a promising approach for inverse problems involving expensive simulators and high-dimensional structured outputs.

0.27

Patch-based Querying Identifies Structures of Interest in Electron Microscopy

cs.CV ·2026-10-05 ·Niels Vyncke, Nicolas Nadisic, Yvan Saeys, Aleksandra Pižurica
published similarity 0.59
Abstract · 41 pages, 20 figures, 5 tables. Accepted for publication in Computers in Biology and Medicine

Volume electron microscopy (vEM) has emerged as an essential sensing technique in biomedical research, allowing the three-dimensional imaging of biological cells and tissues at nanometer-scale resolution. The ability to generate extensive datasets has reached the limitations of downstream analysis processes, which depend significantly on the intervention of human experts for preprocessing and annotation. We propose an efficient and reliable patch-based retrieval framework based on self-supervised learning of local image descriptors to locate self-similar structures in vEM datasets. Given a few manual annotations of a given cellular structure, our method can retrieve similar structures across the EM volume. Our framework is interactive, allowing the human expert to refine the search queries and retrieve relevant image patches quickly and using little labeled data. Experiments on real-world vEM images of biological tissues demonstrate that our framework can reliably identify relevant cellular structures, generalize across different organelles and acquisition modalities, and substantially reduce the search space for downstream analysis.

0.38