CS Paper Radar

A Self-Supervised Pose Estimation Approach for Construction Machines

Alaa Alshubbak, Daniel Görges

cs.CV ·submitted 2026-10-30 12:41 UTC · v1
published

Abstract

Pose estimation is a computer vision task used to estimate a skeleton of dynamic systems to predict future movements. Most of the research in this direction is based on a supervised learning approach which requires a massive amount of labeled datasets. In this paper, a self-supervised three-stage model based on a contrastive learning approach is introduced for estimating a skeleton of dynamic construction machines; such as excavators without using any labeled images for the first stage. The whole model structure is divided into three stages: the pre-train stage using the SimCLR contrastive approach, and two fine-tuning stages for the transfer learning and the downstream task. The model can leverage the features and learn from a huge unlabeled dataset called ACID to two small datasets generated from NVIDIA Isaac and MATLAB Simscape simulators as well as transfer the knowledge to a smaller dataset with a ratio of 3.5% from the original ACID dataset. The results show that the proposed approach can improve the accuracy of pose estimation for heavy construction machines in real images by 11% and 13% in comparison to the normal self-supervised approach with two backbones ResNet-50 and HRNet-W32, respectively.

Comment
12 Pages, 12 figures, submitted for the International Symposium on Visual Computing conference
Journal ref
Advances in Visual Computing, 2023
DOI
10.1007/978-3-031-47966-3_31

Closest papers

Nearest neighbours by meaning across the whole index.

Synthetic-to-Real ViT-Based Pose Estimation of a Noncooperative UAV

cs.CV ·2026-10-03 ·Krishnanujam Srinivas, Hanish Acharla, Brij Agrawal, Leonardo Herrera
similarity 0.62
Abstract · 12 pages, 8 figures, 5 tables

Remote pose estimation of noncooperative Unmanned Aerial Vehicles (UAVs) from imagery is critical, as they cannot be influenced or instrumented in advance. Deep-learning-based approaches offer a promising solution; however, their development is constrained by the cost and difficulty of acquiring large-scale real-world datasets with accurate pose labels. Synthetic imagery provides an alternative, but models trained on synthetic data must overcome the synthetic-to-real domain gap to generalize to real-world imagery. This work investigates the inherent synthetic-to-real generalization capability of a Vision Transformer (ViT)-based model for monocular UAV pose estimation. The proposed approach employs a self-supervised DINOv2 backbone and is trained exclusively on labeled synthetic imagery while being evaluated on labeled real-world imagery. Pose ambiguity-aware strategies are incorporated during training and inference to address ambiguities arising from the projection of a three-dimensional target onto a two-dimensional image plane and from target symmetries. An $α$-$β$ filter is further integrated during inference to improve pose estimations. To assess the model under operational requirements, it is evaluated in terms of Mean Angular Error (MAE) and inference time, both before and after filtering, using a real-world dataset containing 77,077 labeled UAV images. Before filtering, the model achieves an MAE of $19.18^{\circ}$ and an inference time of $13.25$ ms, whereas after filtering, these values are $8.74^{\circ}$ and $13.42$ ms, respectively.

0.21

Residential Construction Image Dataset Using CSI Divisions for Deep Learning-based Object Detection

cs.CV cs.AI ·2026-10-18 ·Fahad Iqbal, Shayan Mirzabeigi
benchmarkdataset similarity 0.61
Abstract

Accurate recognition of construction site elements is essential for the application of deep learning (DL) in the construction industry. Despite rapid advances in computer vision, the adoption of DL in construction has been limited by the lack of structured, publicly available datasets aligned with established industry standards. To address this gap, we introduce the Residential Construction Dataset Using CSI Divisions (ResCD-CSI) image dataset, a manually labeled image benchmark organized according to the divisional structure defined by the Construction Specifications Institute (CSI). The dataset comprises high-resolution images collected from active residential construction sites across the United States, capturing diverse building components, materials, and equipment under real-world conditions. In total, 16,365 objects spanning 14 major CSI divisions were precisely annotated using bounding boxes, ensuring consistency and reliability for object detection tasks. By aligning with the CSI divisional framework, ResCD-CSI establishes a standardized resource that bridges construction practice with computer vision research and supports the development of Artificial Intelligence-driven solutions for residential construction monitoring.

0.17

AutoCompass: Accurate Visual Localization on Public Maps by Learning from Weak Labels

cs.CV ·2026-09-02 ·Javier Tirado-Garín, Alan Savio Paul, Shuai Chen, Axel Barroso-Laguna, Tommaso Cavallari +3 more
top venue similarity 0.60
Abstract · ECCV 2026

Neural map matchers estimate an image's 3-DoF pose relative to a 2D map. These models are trained on large-scale datasets of geo-referenced images, whose position and heading labels often contain noise that affects the trained models. To address this, we present AutoCompass, a supervision approach for training neural map matchers from inaccurate absolute pose labels. First, we show that heading labels are unnecessary: trained from raw GPS labels, models learn to predict accurate headings, automatically. Second, defining a tolerance region around raw GPS improves positional accuracy. Third, if available, our supervision uses relative poses between training images, obtained via SLAM or SfM, which provide a more accurate training signal. Across driving and egocentric benchmarks, AutoCompass consistently outperforms counterparts trained with the usual strong reliance on absolute pose labels.

0.47

Joint-State Inference for Autonomous Excavators Using Cab-View Vision-Control Fusion

cs.RO ·2026-11-10 ·Junwei Li, Jian Pan, Mengchi Cai, Shuai Zhao, Xizhuo Jiang +4 more
similarity 0.60
Abstract

Reliable state feedback of excavator front-end mechanisms is essential for autonomous excavation, yet retrofitting existing machines with dedicated joint sensors, external cameras, or visual markers can increase deployment and maintenance burdens. This paper proposes a closed-loop autonomous excavation framework for continuous inference of excavator joint states using onboard visual observations and control-related temporal information. The framework encodes structural cues from the current cab-view image, models short-term motion trends from historical joystick commands, and introduces a rolling state buffer for the bucket joint to preserve temporal continuity under severe occlusion. Mechanical joint-range constraints are incorporated into the learning objective to suppress infeasible estimates. The current cab-view image and inferred states are supplied to an Action Chunking with Transformers (ACT)-based control policy for online action generation. Experiments in an AGX Dynamics for Unity simulation environment under diverse operating conditions evaluate joint-state inference and end-to-end task execution. Across 20 autonomous excavation trials, the framework achieved a 95% task-completion rate, with an average joint-state-inference-and-control latency of 10.11 ms per step. The bucket-joint inference achieved a mean absolute error (MAE) of 0.965 degrees. These results demonstrate a sensor-light and computationally efficient joint-state-inference approach that can support closed-loop autonomous excavation with reduced dependence on additional sensing infrastructure.

0.13

ViewStruct: View-Grounded Articulation and Connectivity Inference

cs.CV ·2026-10-12 ·Wenjing Bian, Ranran Huang, Andreas Geiger
code/data similarity 0.59
Abstract · Project page: https://bianwenjing.github.io/viewstruct/

Interacting with unfamiliar objects requires understanding how their parts connect and move. We introduce ViewStruct, a framework for jointly predicting connectivity and articulation in objects and scenes. Given an RGB image with aligned part segmentation and depth, optionally supplemented by images or video, ViewStruct predicts rigid-body groupings, attachments, and joint parameters. Its view-grounded representation links predictions to observed regions and expresses joint geometry in a gravity-aligned, view-relative frame, without requiring category-specific canonical orientations. A pretrained vision--language model is adapted to predict structure in seconds, without requiring observed articulation motion or per-instance optimisation. Experiments demonstrate improved structural prediction over evaluated baselines and transfer to unseen categories, asset sources, and real-world human-interaction videos, despite task-specific training exclusively on synthetic data. Simulated manipulation experiments further demonstrate the utility of predicted joints for generating robot motion trajectories.

0.24

Auditing Quality Filters for Long-Tail Human Data Curation

cs.CV cs.AI ·2026-09-25 ·Rishav Agarwal, Nirshal Chandra Sekar, Anirudh Vemula
similarity 0.59
Abstract · 4

Robots on construction sites must detect workers who are kneeling or bending, which we call low poses. These workers can be lost from training datasets during automatic labeling. We study a pipeline that detects people, estimates their body joints using NLF, and groups similar poses. Low poses account for only about 2 percent of the retained examples. This low share may partly reflect the pipeline's quality filter, which rejects examples with low detection confidence or uncertain joint estimates. We examine this filtering using four alternative pose clues: bounding-box shape, vertical body span, pose grouping aligned to the scene's vertical direction, and image appearance. All four suggest that low poses are rejected by the filter more often. Separately, controlled simulated scenes show that a person detector fine-tuned on a public construction dataset misses more workers in these poses even when we correct their bounding box height is matched to that of standing workers. These findings suggest that low poses are scarce and hard to find, and we cannot rely on bounding boxes or poses for long-tail human data curation.

0.22

PVRA: A Pointwise Key-point Voting Framework for Robotic Assembly

cs.RO cs.CV ·2026-08-20 ·Kulunu Samarawickrama, Roel Pieters
published similarity 0.58
Abstract · 14 pages, 3 figures. Accepted for presentation at the European Conference on Robotics (ECoR) 2026

Modern computer vision has enabled partial autonomy in robotic assembly manipulation. However, performing autonomous manipulation of a progressive assembly demands a more specific set of skills, in addition to perceiving the objects. Through a comparative analysis of research in the associated domains, we deduce that object-centric perception must advance towards learning assembly dependencies to predict meaningful actionable outputs for autonomous assembly manipulation. Subsequently, we present a 3D keypoint-based modular learning framework to learn assembly dependencies to infer actionable outputs given a RGB-D input of an assembly scene. We train and evaluate our trained network on an assembly pose estimation dataset and compare it against object-centric baselines with an augmented set of metrics for progressive assemblies.

0.36

DisParQ: Self-Supervised Part Concepts for Interpretable Vision Foundation Models

cs.CV cs.AIcs.LG ·2026-10-07 ·Adam Pardyl, Siddhartha Gairola, Sukrut Rao, Adam Wróbel, Bartosz Zieliński +2 more
similarity 0.58
Abstract · Under review

Concept-based vision models represent images through an intermediate layer of human-inspectable concepts, so what a model relies on can be traced to those concepts. However, those models are often limited to fixed categories or depend on language to define their concepts. We introduce DisParQ (Discrete Parts with Quantized attributes), a method that learns spatially grounded, discrete concept representations from a powerful frozen vision-only self-supervised backbone. It requires no class labels and no language supervision. Each image patch is assigned to exactly one concept from a learnable prototype dictionary, and only a sparse subset of concepts may activate per image. To capture how each concept varies across images (e.g., the type of a "wheel"), we learn continuous residuals alongside the concepts and then quantize them into discrete attributes. A spatial decoder reconstructs the backbone's representation from the concepts and attributes alone, so successful reconstruction means that the discrete representation preserves the backbone's information. We evaluate DisParQ across seven datasets, from general recognition (ImageNet, PartImageNet, Places) to fine-grained benchmarks (CUB, Cars, Dogs, Flowers). We show that DisParQ closely matches its frozen DINOv2 teacher on ImageNet linear probing (83.2% top-1), achieves higher concept consistency than language-aligned models, remains competitive on fine-grained recognition, and enables cross-category part-based retrieval.

0.14