CS Paper Radar

Network-Native DNN Architecture Synthesis for Distributed Inference

Fan Yang, Xiaolong Wu, Qi Cao, Fen Hou, Yulin Shao

cs.IT cs.DC ·submitted 2026-11-16 10:06 UTC · v1
theory

Abstract

Distributed deep neural network (DNN) inference typically follows a model-first paradigm, in which a predefined DNN is partitioned and deployed over a physical network. On irregular communication-computation topologies, however, this approach can create transmission bottlenecks and leave distributed resources underutilized. This paper reverses this perspective and asks: given a physical network, what kind of DNN can it naturally support? To answer this question, we propose a network-native DNN architecture synthesis paradigm in which the physical network topology directly defines inter-node neural dependencies. Neural computation is allocated across nodes under communication, computation, storage, synchronization, and latency constraints. We establish structural properties of the synthesis problem and develop linear programming (LP)-based Architecture Synthesis (LPAS), an efficient topology-guided algorithm that produces a feasible architecture with a computable bound on suboptimality. Experiments on physical networks of different sizes show that LPAS achieves relative gaps below 0.6% to the LP-derived upper bound, with substantially lower runtime than direct mixed-integer LP optimization on larger networks. Under matched network deadlines, the network-native architectures activate more compute nodes and utilize a larger proportion of available model-storage capacity than the model-first baseline. After training, the network-native models achieve higher classification accuracy across all evaluated dataset-precision settings, with the average gain of up to 10.69%. This network-native design paradigm thus turns the physical network from a deployment constraint into an architecture design primitive.

Closest papers

Nearest neighbours by meaning across the whole index.

From Partitioning to Provisioning: Co-Designing DNN Deployment and Infrastructure Capacity

cs.NI ·2026-10-20 ·Zongyao Huang, Fan Yang, Qi Cao, Yulin Shao
similarity 0.78
Abstract

Deep neural network (DNN) partitioning enables distributed inference by dividing model computation across interconnected devices. Existing approaches have largely focused on optimizing DNN deployment over infrastructure with fixed computing and communication capacities. Yet when these resources cannot adequately support inference demands, optimizing deployment alone is insufficient: the supporting infrastructure must also be provisioned. Crucially, deployment and provisioning are inherently coupled. How a DNN is partitioned determines where capacity is needed, while capacity provisioning can fundamentally change the preferred deployment. Motivated by this interdependence, we investigate the co-design of DNN deployment and infrastructure capacity for pipelined inference. Despite the coupling between these decisions, we reveal an efficiently solvable structure: on acyclic networks, the problem admits an exact shortest-path reformulation, enabling globally optimal joint design in polynomial time. Numerical evaluations demonstrate the benefits of co-design over treating deployment and provisioning separately, highlighting the potential of inference-driven infrastructure planning.

0.39

Joint Optimization of Memory and Computing Frequency for Energy-Efficient DNN Inference

cs.AI ·2026-08-14 ·Yunchu Han, Zhaojun Nan, Sheng Zhou, Zhisheng Niu
published similarity 0.63
Abstract

Deep neural network (DNN) inference on mobile devices often incurs high latency and energy consumption due to limited computing and memory resources. To enable energy-efficient DNN inference, most existing studies focus on dynamic voltage and frequency scaling (DVFS) for adjusting the computing frequency, while the impact of memory frequency on the inference performance has been greatly overlooked. In this paper, we consider the impact of memory frequency and computing frequency on DNN inference time, and jointly optimize these two frequencies together with communication resources for energy-efficient DNN inference. Based on a realistic inference time model, we formulate an optimization problem to minimize the energy consumption of all mobile devices under the deadline constraint. For local inference, we derive a near-optimal closed-form solution via convex optimization, while an optimal closed-form solution for transmission power is obtained for edge inference with the given bandwidth. Furthermore, we propose a low-complexity heuristic algorithm to effectively solve the overall problem with polynomial time complexity. Simulation results based on measured data show that the proposed near-optimal solution for local inference can achieve optimal performance under strict deadline constraints, with a performance gap of up to 2.5% compared with the optimal solution. Meanwhile, our proposed algorithm significantly reduces the energy consumption of devices by up to 10.4% compared to other methods.

0.17

HyperCut: Fast Inter-Layer Scheduling via Directed Hypergraph and Early Filtering

cs.AR ·2026-08-19 ·Ziang Wei, Zirui Xu, Sufeng Guo, Chuanchao Gao, Yiyang Gao +2 more
code/data similarity 0.62
Abstract · 8 pages, 10 figures, 1 table

As deep neural networks (DNNs) continue to scale, inter-layer scheduling, which orchestrates the spatial allocation of compute resources and the temporal execution order across layers, has become a decisive factor in sustaining high utilization and energy efficiency on tiled accelerators. However, existing inter-layer schedulers defer cost feedback until a complete fine-grained intra-layer scheduling has been resolved. The resulting decoupled flow repeatedly explores sub-optimal or even infeasible inter-layer schedules, and the absence of early pruning during the inter-layer phase remains a critical bottleneck for design-space exploration (DSE) in DNN compilers. Our key observation is that the cost of an intra-layer scheduling can be tightly upper-bounded once the inter-layer cut fixes the sub-mesh shape, which lets us cost every inter-layer candidate without solving the intra-layer problem. Hence, we propose a hierarchical partitioning-and-mapping framework, HyperCut, that enables early filtering of inter-layer schedules based on hypergraph partitioning. Based on the directed hypergraph (DHG) abstraction of DNN, we introduce a unified representation, State, that jointly encodes the DHG partition, tile mesh allocation and tensor batch splitting. Thereby, partitioning and mapping are coupled into a union optimization object. For a DNN with N layers, the resulting theoretical design space is bounded by O(N), compared with O(9.899^N) for the state-of-the-art open-source scheduler SET. Across 10 evaluated cases, HyperCut achieves 2.0x performance improvement and 80.47% exploration time reduction over the SET baseline, measured by geometric mean.

0.41

Jointly Optimizing Memory, Computing and Offloading for Energy-Efficient DNN Inference

cs.LG cs.AI ·2026-11-08 ·Yunchu Han, Zhaojun Nan, Sheng Zhou, Zhisheng Niu
top venue similarity 0.62
Abstract

In edge intelligence systems, computation offloading and dynamic voltage and frequency scaling (DVFS) are widely used to improve energy efficiency. However, as applications like deep neural network (DNN) inference evolve in complexity, their demands for memory resources have progressively increased, establishing memory as a critical bottleneck that impacts both latency and energy consumption. In this paper, we propose a joint optimization scheme integrating memory, computing and offloading for energy-efficient DNN inference. We formulate the energy minimization problem under deadline constraints as a mixed-integer nonlinear programming (MINLP) problem, where the main challenge arises from the coupling between offloading and resource optimization. To address this issue, we decompose the original problem into two subproblems under a fixed offloading policy. The local subproblem is transformed into an equivalent geometric programming (GP) problem to derive the optimal memory frequency and computing frequency, while the edge subproblem is solved via convex optimization. An efficient coordinate descent (CD) algorithm is further developed to jointly optimize all variables. Simulation results demonstrate that our proposed algorithm achieves near-optimal performance with the gap from the optimal policy being at most 9.3%, and reduces the energy consumption of devices by up to 16.1% compared with benchmarks without memory frequency optimization.

0.18

Para-Pipe: Exploiting Hierarchical Operator Parallelism of ML Computational Graphs on SoCs

cs.DC cs.LGcs.PF ·2026-09-03 ·Yujie Zhang, Huiying Lan, Ehsan Aghapour, Zhiyuan Ning, Peng Zan +3 more
top venue similarity 0.62
Abstract · Accepted to IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems

As edge-based deep learning applications become more complex, optimizing performance on heterogeneous System-on-Chips (SoCs) presents unique challenges. Traditional pipelining techniques distributing the computation across different on-chip processing units, while effective for throughput, do not address the latency demands posed by modern neural networks with complex interdependencies and extensive operator parallelism. There is a potential in leveraging operator parallelism to enable concurrent execution across multiple processing units, thereby reducing inference latency. However, prioritizing pipelining or parallel execution often necessitates a compromise, where optimizing one performance metric adversely impacts the other. This paper introduces Para-Pipe, a hierarchical mapping framework that integrates intra- and inter-stage operator parallelism within a pipelined architecture. Para-Pipe navigates the trade-off between throughput and latency by selectively fine-tuning parallelism levels within and across pipeline stages. This strategy can significantly reduce inter-processor communication overhead, significantly improving energy efficiency. Our evaluation demonstrates that Para-Pipe generates multiple Pareto-optimal configurations, achieving a balance between throughput and latency on an Amlogic SoC equipped with ARM big.LITTLE CPUs and GPU, as well as the Black Sesame Technology SoC featuring a deep learning accelerator and two DSPs. More importantly, throughput-optimized configurations under Para-Pipe on Amlogic SoC show an average energy efficiency improvement of 11.0% over purely pipelined strategies and 23.3% relative to non-pipelined parallel execution.

0.39

Linear Reusable Neural Bases Architecture for Network Compression

cs.LG cs.AI ·2026-09-01 ·Binshuai Wang, Peng Wei, Mahyar Ghazanfari
similarity 0.61
Abstract

Memory constraints remain a critical bottleneck in the deployment of large-scale AI models. Parameter sharing across network depth reduces model storage, but repeatedly applying an identical transformation limits flexibility across layers. Inspired by time--memory trade-offs in classical algorithms, we introduce the Linear Reusable Neural Bases (LRNB) architecture, an RNN-based framework that improves parameter efficiency through parameter reuse at the \textit{neuron level}. Each feedforward residual module is represented as a linear combination of shared neural bases, with depth-specific learnable coefficients and optional shifts providing flexibility across layers. This formulation reduces parameter redundancy across depth and enables the construction of wider and deeper networks within a fixed parameter budget. We further provide a geometric interpretation of the neural bases from a vector-field perspective and extend the framework to linear projection modules. Experiments demonstrate that the LRNB architecture achieves comparable or lower final training loss than independently parameterized baselines while using fewer parameters and maintaining stable training dynamics. These findings support neuron-level reuse as a practical approach to parameter-efficient network design.

0.26

MANE: A Multi-Path Adaptive Network for Edge Onloading of Deep Neural Networks

cs.DC cs.AI ·2026-09-13 ·Sokratis Nikolaidis, Stylianos I. Venieris, Leonidas Malachias, Iakovos S. Venieris
similarity 0.61
Abstract · 6 pages, 5 figures, submitted to IEEE UIC 2026

Split computing constitutes a widely used distributed inference approach, where a lightweight head model is onloaded onto the device and a heavier tail model resides on an edge server, leveraging the growing computational capabilities of modern System-on-Chips while alleviating server load. As intelligent indoor environments such as smart offices grow increasingly populated with diverse IoT devices, a single edge server must simultaneously assist multiple devices, each competing for the same shared inference resources. Without a principled mechanism to manage this shared load, the server is quickly overwhelmed, causing latency SLO violations and rendering server-assisted inference ineffective. In this work, we present MANE, a distributed inference framework that equips the server with a multi-path tail architecture, enabling a dynamic accuracy--throughput trade-off at runtime. By introducing a novel multi-path model architecture, a three-stage training scheme featuring a Joint Head Network Distillation loss and a hysteresis-based scheduler with an equitable device-fallback policy, MANE maintains over 80% SLO satisfaction rate where state-of-the-art onloading methods fail completely, while preserving accuracy 6pp higher than on-device alternatives, across up to 40 concurrent devices.

0.13

Elastoformer: Enabling Dynamic Adaptivity via Elastic Model Transformation

cs.CV cs.AIcs.PFeess.SY ·2026-09-09 ·Sudaksh Kalra, Dolly Sapra
publishedcode/data similarity 0.61
Abstract · Published at SEC'25

EdgeAI systems are increasingly employing computer vision applications to enable intelligent, on-device decision-making in real-time. However, these deployments face highly dynamic operational conditions, with fluctuating constraints on latency, power availability, and memory resources. Deep Neural Networks (DNN), which follow fixed computational execution flows, lack the flexibility to adapt to such variability, resulting in inefficient and suboptimal performance in edge scenarios. This underscores the need for architectures that are not only efficient but also dynamically scalable at runtime. In this paper, we propose Elastoformer: A framework that transforms conventional neural networks (NN) into Elastic NN capable of real-time elastic inference. Unlike the conventional bag-of-models approach, which requires maintaining multiple independent models for different operating conditions, Elastoformer offers a single, modular solution that dynamically switches between multiple modes of operation at runtime, adapting efficiently to the changing computational budgets of edge devices without the overhead of managing separate models. Experiments reveal that our framework achieves up to 85% reduction in computation FLOPs, 50% reduction in latency and 76% reduction in memory overhead, while showcasing the architecture agnostic nature of the framework across both Vision Transformers and CNNs. Our code is available at https://github.com/sudaksh14/Elastoformer.

0.71