CS Paper Radar

Versatile Photonic Encoding Platform with Selectable Back Ends for Automotive Edge Processing

Isamu Takai, Makoto Nakai, Ayana Mizuno, Atsutaka Miyamichi

cs.ET ·submitted 2026-10-29 04:30 UTC · v1

Abstract

The increasing diversity and computational demands of automotive edge functions require hardware architectures that support heterogeneous processing while limiting implementation complexity and system overhead. We investigate a system-level approach in which a common, fixed photonic front end provides a shared physical transformation for multiple processing functions, while application adaptation is handled by lightweight peripheral processing. We propose the Versatile Photonic Encoding Platform (VPEP), implemented using a fabricated silicon-photonic chip with 32 phase-modulated inputs, a passive fixed encoding core, and 17 on-chip photodetector outputs. Without reconfiguring the photonic core, the same chip is evaluated across four representative functions relevant to automotive edge systems: classification and regression for machine learning, analog-to-digital conversion for mixed-signal processing, and photonic information encoding and recovery for communication-related processing. Compared with linear baselines without photonic encoding, VPEP improves the mean macro-F1 score on 12 of 13 classification datasets and the mean coefficient of determination on 9 of 12 regression benchmarks. For analog-to-digital conversion, the best configuration achieves an in-sample normalized root-mean-square error of approximately 0.75% of full scale. For information encoding and recovery, a lightweight nonlinear decoder reconstructs four-character (32-bit) inputs from 17-channel encoded outputs with approximately 94% character accuracy for a 26-symbol set. These results demonstrate the feasibility of cross-application processing with a single fixed silicon-photonic encoding core and suggest a system-level design approach, complementary to reconfigurable photonic architectures, that combines common photonic hardware with lightweight application-specific processing for future automotive edge systems.

Comment
21 pages, 10 figures, 3 tables, 4 supplementary tables

Closest papers

Nearest neighbours by meaning across the whole index.

A Time-Encoded Analog Photonic Interposer for Energy-EfficientIntegration of Analog Vision Sensors and Analog Accelerators

cs.AR ·2026-09-02 ·Subhradip Chakraborty, Zihan Yin, Xuming Chen, Chengwei Zhou, Gourav Datta +1 more
similarity 0.57
Abstract

This work introduces a time-encoded analog photonic interposer that enables long-distance, high-fidelity transport of analog signals between spatially separated chiplets. Unlike prior silicon-photonic links limited to digital data, the interposer preserves analog information by converting amplitudes into timing intervals using an analog-to-time converter (ATC), transmitting them over a wavelength-division-multiplexed photonic link, and reconstructing values at the receiver without an explicit high-precision ADC/DAC data-converter pipeline. The link instead embeds an implicit 6-bit time-domain quantization and uses a single wavelength per processing element independent of bit precision. Evaluated in a fully analog vision pipeline with an in-pixel computing (IPC) sensor, it achieves a 2.04x energy--delay product (EDP) improvement over an 8-bit digital electrical baseline on the 560x560 Visual Wake Words dataset, with the advantage widening with link length even against a precision-matched 6-bit baseline. The pipeline holds 89.87% and 86.15% accuracy on ResNet18 and MobileNetV2 for VWW and generalizes across CIFAR-10 and ModelNet40 within 2% of the digital baseline.

0.26

LCAP: Population-Informed Latent Chip Adaptation from Few Output Probes for Photonic Neural Networks

cs.LG ·2026-09-15 ·Tianyu Gao, Guantian Zheng
similarity 0.56
Abstract · 5 pages, 3 figures, 2 tables

Photonic neural networks (PNNs) offer efficient analog inference, but parameters optimized under ideal device models can degrade after fabrication, creating a persistent simulation-to-hardware (sim-to-real) gap. When many identically designed chips are deployed, calibrating each device from scratch compounds this cost. We propose Latent Chip Adaptation from Probes (LCAP), a population-informed framework that decomposes hardware adaptation into a transferable population correction and probe-inferred latent personalization. LCAP first learns a shared correction from 80 historical chips, then extracts a low-dimensional correction space from device-specific refinements. At deployment, 32 fixed unlabeled output probes infer an unseen chip's latent correction coordinates, enabling feed-forward personalization without target-device optimization. On a three-layer 64-mode MZI simulator with phase variation, beam-splitter errors, quantization, and crosstalk, accuracy improves from 80.4147% under direct deployment to 92.6860% after shared calibration and 93.3617% with LCAP. LCAP improves 27/30 unseen chips and raises worst-device accuracy from 89.18% to 90.54%.

0.31

SPARC: Accurate Photonic Matrix Multiplication via Representative Vectors

cs.AI ·2026-11-12 ·Yasuhiro Fujiwara, Tatsuhiro Nakamori, Hideyuki Kawashima
similarity 0.55
Abstract · 11 pages, 5 figures, 3 tables

Photonic neural network accelerators have emerged as a promising solution for reducing the power consumption of deep neural network (DNN) inference by executing matrix multiplication on analog photonic hardware. However, their inference accuracy is limited by low-bit quantization required for photonic processing and by analog photonic noise. This paper proposes an approach for improving inference accuracy by exploiting the high similarity among activation vectors in DNNs. Specifically, our approach constructs a representative vector for each group of similar activation vectors, computes its inner product digitally with high precision, and offloads the residual inner products to the photonic accelerator. Since the activation vectors within each group are highly similar, the residual vectors have much smaller norms, thereby reducing the effects of quantization errors and analog photonic noise. Furthermore, our approach preserves the photonic inner-product computation pipeline, allowing seamless integration into existing and future photonic accelerators without hardware modification. Experimental results on six representative DNN models demonstrate that the proposed approach can improve inference accuracy across diverse network architectures and application domains.

0.33

Joint Quantization and Dynamic Partitioning for Context-Aware Edge Vision Transformers

cs.AI ·2026-11-10 ·Zhu Jiaqi, Jain Prakriti, Katare Dewant, Kumar Saurabh
published similarity 0.54
Abstract

Deploying vision transformer models across connected and automated vehicles (CAVs) requires joint management of computation and Vehicle-to-Vehicle (V2V) communication. Static split-inference methods use fixed partition points and cannot adapt to time-varying link bandwidth, processing load, and concurrent multi-stream traffic. This paper proposes an edge orchestration framework that jointly selects model partition points, per-layer quantization bitwidths, and execution placement across heterogeneous vehicular compute nodes. The joint partitioning and mixed-precision allocation problem is formulated as a Mixed-Integer Linear Program (MILP) that minimizes compute and communication energy subject to end-to-end latency and task-quality constraints. The framework is evaluated on Jetson Orin Nano (8~GB), Jetson Orin NX (16~GB), and Jetson AGX Orin (32~GB) nodes using OPV2V-driven multi-stream workloads with $K \in \{2,4,6,8\}$. Across the evaluated configurations, the proposed formulation reduces V2V communication energy by $33.3$--$36.0\%$ relative to a Uniform INT8 split. At $K=8$, end-to-end task completion latency is reduced by $34.0\%$ on the Orin Nano and $33.1\%$ on the Orin NX relative to the same baseline, while limiting the reduction in 3D detection accuracy to $3.1$--$4.1$ percentage points in $\mathrm{mAP}_{50}$ relative to FP16 execution.

0.38

Quantized Edge Inference of a Self-Supervised SAR-Optical Encoder on Commodity x86 Hardware

eess.IV cs.CVcs.LG ·2026-10-27 ·Christian Sanchez-Leon, Dongming Chen
similarity 0.54
Abstract · 3 pages, 1 figure, 3 tables

Self-supervised vision transformers are increasingly used as reusable encoders for Earth observation, but their inference cost on the hardware available in the field is rarely characterized. We deploy a pretrained SAR-optical masked-autoencoder encoder on three commodity x86 nodes, from a low-power CPU without AVX2 to an AVX2+VNNI reference, at FP32, FP16 and INT8 on the CPU and, where present, the integrated GPU, and report latency, energy (RAPL and wall plug) and linear-probe accuracy. Quantization is accuracy-neutral: INT8 changes micro-mAP by only +0.0006, within the seed spread, so precision becomes a pure cost decision. INT8 lowers CPU latency on every node, by 1.45x even on the CPU without AVX2, and CPU energy wherever it is measured, and the VNNI reference (1.53x) suggests a further gain beyond AVX2. The Pareto-optimal device shifts from the integrated GPU, which is 4.38x faster than the CPU on the weakest node, to the CPU as the CPU class improves.

0.24

STEMPix: A Phase-Transition-Material-Based Pixel Sensor for Resolving Edge-Movement Direction

cs.ET cs.CV ·2026-09-03 ·Md Rahatul Islam Udoy, Sumeet Kumar Gupta, Deep Jariwala, Ahmedullah Aziz
similarity 0.54
Abstract

This paper proposes a spatio-temporal edge-movement direction pixel (STEMPix) for generating compact direction-aware edge movement information inside a CMOS-compatible image sensor array. The proposed design targets specialized sensing applications where local boundary movement is more important than full-frame intensity reconstruction. Instead of transferring full multi-bit frames for external processing, STEMPix generates a 3-bit local edge direction code (LEDC) by combining pixel-level temporal change information with neighboring-pixel spatial edge information. We design the architecture using a two-tier organization, where the photodiode layer is separated from the computation layer to preserve light-collection area while accommodating the additional in-array processing circuitry. The proposed circuit is evaluated through HSPICE transient simulations. The estimated implementation achieves a horizontal pitch of 1.73 μm, a vertical pitch of 2.36 μm, and a geometric fill factor of 95.47%. The average active switching energy is 0.465 fJ per LEDC operation across representative edge-movement cases. The proposed STEMPix operation also supports global-shutter capture and dynamic thresholding. These results indicate that STEMPix can provide a compact and scalable front-end representation for edge-movement-aware sensing systems.

0.26

Hardware-Aware Learned Representation Compression for Distributed In-Sensor Vision

cs.LG cs.AIcs.AR ·2026-09-12 ·Chengwei Zhou, Abu Masum, Xuming Chen, Mehran Moghadam, Sreetama Sarkar +5 more
published similarity 0.53
Abstract · Under submission at IEEE Transactions on Emerging Topics in Computing

In-sensor computing reduces the cost of transmitting high-resolution image data by performing early-stage processing near the sensor. However, the logic chip integrated with a CMOS image sensor (CIS) is tightly constrained in compute and memory, limiting conventional deep neural network partitioning. We present OASIS, a distributed in-sensor vision framework that uses a lightweight encoder to generate compact, task-relevant representations before off-chip transmission. The encoder is trained end-to-end using task, entropy, and reconstruction objectives, while the decoder is used only during training. OASIS supports two complementary deployment paths. The first applies 4-bit quantization and Huffman coding while preserving the spatial structure required by classification and dense-prediction tasks. The second uses Sobol-based hyperdimensional computing (HDC) to transform the encoder latent into a fixed-dimensional binary hypervector for associative-memory classification. For the SwinViT-based VWW model, mapping a $3\times3\times8$ latent to a 64-dimensional hypervector provides an additional $1.77\times$ communication reduction with less than one percentage point of accuracy loss relative to the 128-dimensional configuration, yielding an overall $18{,}816\times$ reduction compared with raw 8-bit image transmission. We implement the digital near-sensor pipeline on an AMD Xilinx Zynq UltraScale+ FPGA and characterize it using direct board-level power measurements and Vivado post-implementation analysis, together with circuit-simulated CIS models and a 7-nm ASIC projection. Across visual wake-word classification, hand tracking, and eye tracking, OASIS reduces total system energy by approximately $2\times$-$4.5\times$ while maintaining competitive accuracy, demonstrating a practical hardware-algorithm co-design path for communication-efficient in-sensor vision.

0.21

EMMI: Edge Multi-Modal Intelligence for Communication-Efficient MLLM Inference via Fused Representation Compression

cs.LG cs.DC ·2026-09-10 ·Motahare Mounesan, Irfan Khan
benchmark similarity 0.53
Abstract

Recent advances in multimodal large language mod- els (MLLMs) have opened new opportunities for edge intelligence by enabling reasoning across heterogeneous sensor modalities, such as vision, text, and telemetry data. However, deploying these capabilities on resource-constrained edge platforms remains challenging due to the substantial computational, memory, and communication demands of modern MLLMs. Rather than transmitting raw sensor observations or partitioning neural networks at intermediate layers, Edge Multi-Modal Intelligence (EMMI) communicates a compact representation between edge devices and server resources, enabling communication-efficient edge MLLM inference. To achieve this, EMMI performs modality-specific encoding, cross-modal representation fusion, and learned compression at the edge, transmitting only a compact latent representation to server-side resources for high-capacity MLLM reasoning. This representation-centric design reduces communication overhead, preserves local data privacy, and provides a fixed-size interface between heterogeneous edge devices and server-side MLLMs. Evaluation on a representative multimodal benchmark demonstrates that EMMI can reduce the communication payload by 32x while maintaining comparable downstream accuracy, resulting in up to a 3.4x reduction in estimated end-to-end inference latency under bandwidth-constrained edge conditions.

0.11