CS Paper Radar

A Vision-Language Model (VLM)-based Pipeline for End-to-End Procedural Modeling of Field-Grown Maize from Point Clouds

Mozhgan Hadadi, Talukder Z. Jubery, Adarsh Krishnamurthy, Baskar Ganapathysubramanian

cs.CV ·submitted 2026-10-02 15:41 UTC · v1

Abstract

Editable 3D models of field-grown crops support high-throughput phenotyping and in silico breeding trials, but building them from scanned point clouds requires organ-level segmentation and fitting. Procedural generators can turn an organ-level parameter set into an analysis-suitable 3D model, but obtaining that set requires hours of manual tuning per plant or segmentation models trained on species-specific labels. We present an automated pipeline that reconstructs procedural maize models from raw 3D point clouds without manual tuning or species-specific training data. A multimodal vision-language model (VLM) annotates leaf midlines in rendered orthographic views. Deterministic geometric algorithms back-project the annotations onto the point cloud, merge them into 3D leaves by cross-view consensus, and grow the midlines to full blades on an orientation-weighted surface graph. Measured organ parameters populate a plant descriptor for a Non-Uniform Rational B-Spline (NURBS)-based procedural model generator. Each leaf surface is then refined against its scan points by differentiable NURBS fitting. The pipeline reached a median whole-plant Chamfer distance of 5.4 mm on 100 genotypically diverse field-grown maize plants from the MaizeField3D dataset. The reconstructions were closer to the scans than those of an earlier semi-automated pipeline based on manual annotations. The pipeline recovered 1,017 of 1,023 (99.4%) curated reference leaves at an intersection-over-union of at least 0.5 without using those labels as input. These results show that VLM annotations become usable organ-level measurements when downstream geometric stages can correct them. This makes automated generation of editable 3D plant assets feasible at the scale of modern phenotyping experiments.

Closest papers

Nearest neighbours by meaning across the whole index.

Zero-Shot 3D Plant Organ Segmentation with SAM3 and Semantic NeRFs

cs.CV ·2026-09-07 ·Andreas Gilson, Laura Hennig, Peter Pietrzyk
datasettheory similarity 0.68
Abstract · Submitted to CVPPA Workshop at ECCV 2026

Accurate 3D plant organ segmentation is fundamental to automated phenotyping. Existing approaches rely on annotated training data or species-specific model configurations. We present an annotation-free pipeline for 3D plant organ segmentation, combining text-prompted SAM3 segmentation with semantic neural radiance fields (NeRFs). Given only multi-view RGB images and a list of class names, our zero-shot pipeline produces semantically labeled 3D point clouds without manual annotation, per-species fine-tuning, or domain-specific preprocessing. Multi-view NeRF fusion acts as effective implicit consensus mechanism that lifts imperfect per-frame masks into accurate 3D labels. On a controlled Begonia maculata testbed the SAM3 pipeline achieves 92.6% mIoU, reaching 95.9% of the oracle upper bound established with perfect ground-truth masks. The pipeline was further evaluated on a new dataset spanning ten diverse plant point clouds reaching an average 0.856 mIoU, with leaf and pot IoU above 0.91 and 0.90 for every species, respectively. These results demonstrate that annotation-free 3D plant organ segmentation is now feasible and approaching the range of supervised methods.

0.15

Automated Maize Ear Phenotyping Using 3D Reconstructions

cs.CV ·2026-09-01 ·Ritwesh A. Kumar, Som Tripathi, Peja Matthews, Srikar Reddy, Talukder Zaki Jubery +3 more
similarity 0.65
Abstract

Maize kernel traits such as row number, kernels per row, and kernel size vary largely for genetic reasons and are consistently associated with regions of the genome that influence yield. Manual measurement of these traits, however, cannot keep pace with the volume of maize generated in a breeding program. To address this, we developed and validated a fully automated pipeline for extracting these traits from 3D point clouds of corn ears, built on a recently developed video-to-point-cloud platform. Raw video frames are processed through COLMAP and NeRF, the ear is isolated via density-based separation, and the point cloud is distance-calibrated to physical units. The calibrated ear point cloud was Z-axis aligned via PCA and cylindrically unwrapped to a 2D image. We enhanced contrast and performed zero-fine-tuning instance segmentation using Cellpose-SAM. A triple-juxtaposed unwrap strategy was used to prevent double-counting at the seam. The pipeline achieved kernel count R^2 = 0.921 (MAPE = 10.33%) and kernel row number within +-2 rows for 95.2% of ears (MAE = 0.75 rows) on a 168-ear held-out set from the 268-ear labeled dataset. The resulting multi-trait dataset has known genotype identity for each ear, positioning it for phenotype-to-genotype association analyses.

0.34

Efficient 3D plant segmentation through multiscale graph learning and optimized geometric processing

cs.CV ·2026-11-07 ·Jesus Ivan Ruiz-Martinez, Andres Mendez-Vazquez, Eduardo Rodriguez-Tello
similarity 0.65
Abstract

Plant organ segmentation on compact computing platforms requires accurate predictions and affordable geometric preparation. We develop and evaluate a multiscale edge based graph attention model that combines local shape descriptors, two neighborhood scales and a residual stem and leaf correction in approximately 78 thousand parameters. The engineering application is plant point cloud segmentation for precision agriculture, assessed through grouped accuracy experiments and physical Raspberry Pi 5 inference. Broad comparisons with hierarchical, graph and transformer baselines establish PointNeXt as the strongest external accuracy reference among the tested local recipes. A nested four variant ablation on a local maize collection separates geometric information from the combined multiscale configuration across seven outer groups and three seeds. The complete model achieves the highest mean intersection over union, 86.36%, although uncertainty and boundary errors qualify the contribution of each component. We then optimize full cloud principal component analysis, sampling and neighbor queries without changing learned weights or descriptor definitions. All 1,180 paired desktop comparisons on PlantNet and TomatoWUR preserve exact logits and predictions. Multiscale pipeline latency falls from 330.33 to 96.82 milliseconds on PlantNet and from 1,054.52 to 463.10 milliseconds on TomatoWUR. PointNeXt also benefits from optimized sampling and remains faster over the complete pipeline. These results support compact graph learning with reduced geometric overhead, while physical board measurements establish inference feasibility rather than energy efficiency or readiness for autonomous flight.

0.15

EgoMaize: A First-Person Maize Instance Segmentation Benchmark under Severe Field Occlusion

cs.CV ·2026-09-11 ·Jiayi Li, Zihan Zhang, Erhankang Yan, Yitian Chen, Yuze Li +2 more
publishedbenchmarkcode/data similarity 0.63
Abstract · Accepted to BMVC 2026

Close-range first-person field images are important for mobile maize phenotyping because many plant-level traits depend on in-canopy structures that are difficult to ob serve from overhead views. However, post-seedling maize fields create a difficult in stance segmentation setting: stems, leaves, tassels, and neighboring plants are elon gated, repetitive, and strongly occluded. We introduce EgoMaize, a compact benchmark for first-person maize instance segmentation, where the task is to predict ownership consistent plant masks and plant-owned stem/tassel cues from close-range field images with severe same-class overlap. Existing visible-only labels can fragment one physi cal plant into disconnected supervision, while full-amodal labels may require unverifi able completion behind neighboring plants or field objects. EgoMaize therefore uses an evidence-closed annotation workflow for occluded maize regions and assigns unreli able maize regions to ignore rather than background. Baseline results show that pre trained query-based grouping, boundary refinement, and high-resolution crop refine ment help different aspects of the task, but no architecture solves the coupled chal lenges of fine structure recovery, same-class instance ownership, and occlusion reason ing; occlusion-level analysis further shows that performance decreases as plant visi bility becomes more limited. The dataset and code are publicly available at https: //github.com/JaaaaaaaD/EgoMaize.

0.45

Scale Matters: Adaptive Granularity Selection for Cross-Species 3D Plant Organ Segmentation

cs.CV ·2026-08-18 ·Carla Salazar, Lazaros Nalpantidis
publishedcode/data similarity 0.63
Abstract · Accepted at the Computer Vision in Plant Phenotyping and Agriculture (CVPPA) Workshop at the European Conference on Computer Vision (ECCV) 2…

Recent 3D foundation models provide powerful feature representations for point cloud learning by controlling spatial granularity. However, relying on a fixed spatial granularity severely limits generalization in applications like plant phenotyping, where organ morphology and size vary substantially across species and growth stages. To address this, we propose AGS-PlantSeg, a few-shot 3D plant organ segmentation method that leverages the frozen Utonia (arXiv:2603.03283) foundation model combined with Adaptive Granularity Selection. By dynamically selecting the best granularity levels for each specific plant model, our method extracts optimized geometric features for a lightweight MLP segmentation head. Extensive experiments across PLANesT-3D (arXiv:2407.21150), Pheno4D , and Crops3D demonstrate that AGS-PlantSeg significantly improves cross-species generalization, achieving 88.9% average mIoU performance and outperforming fixed-granularity baselines by 2.5 mIoU points. Despite requiring minimal annotated data, our approach is highly competitive with fully supervised, plant-specific architectures.

0.39

Denoising-Aware Temporal Point Cloud Completion for 3D Crop Architecture Recovery and Phenotypic Trait Extraction

cs.CV ·2026-08-28 ·Mrudul Mittal, Soumyashree Kar
benchmark similarity 0.62
Abstract · 25 pages of pdf for manuscript , total 12 files including bbl and tex

High-throughput phenotyping depends on accurate 3D reconstruction of plants across growth stages, yet the development and evaluation of temporal completion methods are limited by the lack of datasets with complete geometric ground truth. To address this challenge, we introduce SynthCrop4D, a procedurally generated synthetic dataset of temporally evolving plant point clouds that provides controllable noise, occlusion, and complete plant geometry for benchmarking reconstruction methods. Using this dataset, we evaluate a two-stage pipeline that combines spatial denoising and temporal point cloud completion. First, a denoising module removes structural artifacts from raw laser-scanned point clouds. The resulting data are then processed by an Adaptive Temporal PoinTr model that reconstructs the current growth stage (t) using information from the previous stage (t-1), enabling recovery of regions missing due to self-occlusion. We evaluate the proposed framework on both SynthCrop4D and the real-world Pheno4D dataset (tomato and maize) under settings with and without denoising. Results show that denoising substantially improves reconstruction quality, with the best configuration achieving a Chamfer Distance of 0.0061 on SynthCrop4D (Temporal PoinTr + Mamba-DG) and an F-Score of 0.2080 on Pheno4D (Vanilla PoinTr + Mamba-DG). We further demonstrate the use of completed point clouds for phenotypic trait extraction, including plant height, canopy width, and convex hull volume, obtaining hull-volume MAEs of 0.021 on synthetic data and 0.343 on real data. Together, SynthCrop4D and the proposed pipeline provide a benchmark and methodology for temporal plant reconstruction and high-throughput crop phenotyping.

0.24

Evaluating Mesh Reconstruction Methods for Crop Phenotyping

cs.CV ·2026-09-15 ·Karanvir Singh, Theo Morales, Binh-Son Hua, Mukesh Saini
similarity 0.62
Abstract · 12 pages, 17 Figures

Phenotyping an agricultural crop is crucial for studying its entire life cycle, as it provides vital insights to improve yield and, ultimately, food production. Doing the same for crops grown on remote sites is a challenge for the specialists who cannot be available on-site. 3D reconstruction techniques offer a promising solution to this problem by enabling crop digitization, allowing specialists to access the resulting 3D crop models from anywhere at any time. In this work, we evaluate recent 3D reconstruction pipelines for crop phenotyping. We focus on 7 mesh reconstruction pipelines and measure the fidelity and consistency of their outputs qualitatively and quantitatively. Our results suggest that the meshes produced by the GGGS, PGSR, and 2DGS are preferable to the other pipelines, owing to their quantitative metrics and visually pleasing outputs. The GGGS pipeline is better than the second-best pipeline (2DGS) by about 27% on the radar chart with 5 dimensions, namely, User ratings, Chamfer distance, LPIPS, PSNR, and SSIM.

0.29

Plant Orbiter: 3D Plant Reconstruction and Generation with Video Diffusion Orbit Cameras and 3D Gaussian Splatting

cs.CV ·2026-11-09 ·Zane K. J. Hartley, Yong En Kok, Lewis A. G. Stuart, Andrew P. French, Michael P. Pound
similarity 0.62
Abstract · Submitted to Plant Phenomics. 16 pages, 6 figures, 5 tables

Modern digital phenotyping pipelines require the precise measurement of a range of 3D morphological plant traits. The accurate, high throughput acquisition of these traits helps breeders and researchers accelerate the development of new varieties, techniques and products. Recent advances in 3D technologies have enabled a rapid transition from 2D to 3D pipelines, but high throughput acquisition of digital 3D twins of plants remains a significant challenge due to complex morphology, thin structures and occlusions. In this paper we present Plant Orbiter, a new approach to generating 3D plants that leverages recent advances in video diffusion to produce high quality 3D Gaussian Splats of plants from a single image. We show that our method more accurately recreates plant morphology than other state-of-the-art methods, achieving the highest 3D F-score against all baselines, while being more practical for high throughput pipelines due to easier alignment and reduced cost of camera placement. Combined with a 2D diffusion model, we show that our method also achieves state-of-the-art results in the generation of 3D plant models from a text prompt.

0.24