CS Paper Radar

A Lightweight Multimodal Vision-Language Framework for Early-Stage Anatomical Green Fruit Classification in Commercial Orchards

Ranjan Sapkota, William Bu, Chen Chen, Yunjun Xu, Manoj Karkee

cs.CV cs.AI ·submitted 2026-08-23 20:26 UTC · v1
code/datahot direction

Abstract

Accurate identification of early-stage apple fruitlet anatomical structures, including the calyx, fruitlet body, and peduncle, is essential for robotic thinning, crop-load management, and other precision orchard operations. This study presents a lightweight multimodal vision-language framework that adapts TinyCLIP for fine-grained fruitlet anatomy classification in complex orchard environments. A dataset of 600 high-resolution RGB images collected from Scilate and Scifresh apple orchards was converted into 224 x 224 image patches and annotated for three anatomical classes. Domain-specific language prompts, such as ``a photo of a class,'' were used to guide multimodal alignment between orchard imagery and horticultural structures. A sliding-window inference strategy with a stride of 112 pixels aggregates patch-level predictions into spatial heatmaps, enabling interpretable whole-image localization of fruitlet components relevant to robotic thinning. Patch-level evaluation on an NVIDIA T4 GPU achieved F1-scores of 0.95 for calyx, 0.98 for fruitlet, and 0.85 for peduncle, with a macro-F1 score of 0.93. Deployment-oriented optimization using ONNX and TensorRT enabled efficient inference on NVIDIA Jetson hardware, preserved accuracy under INT8 quantization, and supported model sizes of approximately 127-137 MB with millisecond-level patch inference. These results demonstrate that lightweight vision-language models can provide interpretable and edge-deployable perception for automated fruitlet analysis and future robotic thinning systems. The source code and implementation details are publicly available at https://github.com/WilliamBu1/A-Lightweight-Vision-Language-Model-for-Early-Stage-Fruitlet-Classification-in-Apple-Orchards.

Closest papers

Nearest neighbours by meaning across the whole index.

Optimizing YOLO27, YOLO26, YOLO11, and YOLOv8 for Fine-Grained Small-Object Detection and Segmentation in Complex Orchard Environments

cs.CV ·2026-08-23 ·Ranjan Sapkota, Manoj Karkee
benchmarkcode/data similarity 0.72
Abstract

This study presents an architectural and experimental cross-generation analysis of Ultralytics YOLO27 (YOLOv27), YOLO26 (YOLOv26), YOLO11 (YOLOv11), and YOLOv8 for fine-grained robotic perception in complex orchard environments. Fine-grained detection and instance segmentation of early-stage fruit anatomy remain challenging in complex orchard environments because of limited pixel footprints, green-on-green similarity, occlusion, and substantial scale variation. Because YOLO27 has been announced but its public implementation and trainable segmentation models are not yet available, the present study provides an architectural analysis of YOLO27, while controlled experiments benchmark YOLOv8, YOLO11, and YOLO26; YOLO27 experiments will be incorporated following public model availability. Five model scales-nano (n), small (s), medium (m), large (l), and extra-large (x)-were evaluated for fruitlet, calyx, and peduncle detection and segmentation using conventional 640 x 640 and small-object-focused 960 x 960 configurations, yielding 30 experiments. YOLO11s-960 achieved the highest observed mask mAP@50:95(0.402) and box mAP@50:95(0.426), whereas YOLO26s-960 achieved comparable values of 0.397 and 0.425 with only 10.37~M parameters and 34.1~GFLOPs. Peduncle remained the most challenging class, and increasing model capacity did not consistently improve accuracy. Overall, compact-to-moderate YOLO models combined with small-object-focused training provided favorable accuracy-efficiency trade-offs, establishing a reproducible benchmark for fine-grained agricultural robotic perception. Code, trained models, and experimental configurations are publicly available, and will be updated through our Github Link: https://github.com/rnjnspkt/Optimizing-and-Comparing-Ultralytics-YOLOv26-YOLOv11-and-YOLOv8-for-Small-Object-Detection-and-Seg

0.64

CAppleNet: A Coordinate Attention-Based YOLO11 for Real-Time Apple Stem Cavity and Calyx Detection

cs.CV cs.AI ·2026-11-10 ·Alican Güneş, Robert Selim Hill, Gökhan Hasanoğlu, Murat Kurt
similarity 0.69
Abstract · 19 pages, 6 figures, 15 tables

Automated fruit grading, sorting, and orientation systems require real-time, accurate detection of fruit-level morphological landmarks rather than the fruit boundary alone. This paper presents a custom, fully self-collected and self-annotated dataset of 1506 apple images (1054 training, 301 validation, 151 test) captured under controlled, fixed-camera conditions and annotated for three target classes: fruit body, stem cavity, and calyx. A YOLO11s detector trained with default hyperparameters is adopted as the baseline, and a systematic, one-factor-at-a-time ablation study is conducted across input resolution, data augmentation policy, optimizer configuration, and lightweight architectural enhancement to identify the factors that most improve detection of these fine-grained landmarks under high inter-cultivar color and texture variability. Neither increased input resolution, nor any of six tested augmentation variants, nor manual optimizer tuning improved on the untuned baseline (F1 = 0.9047); only a same-slot architectural substitution of the backbone's C2PSA block with Coordinate Attention (CA) matched the baseline's overall F1 (0.9038) while reducing parameter count by 10.2% and achieving the study's best mAP@0.5 (0.935) and best stem-cavity F1 (0.8217). A confidence-threshold sweep identified 0.15 as the deployment operating point for this model, and a confirmatory experiment combining the CA-swap architecture with the best-performing augmentation variant showed the two effects do not combine additively, reversing the architectural gain rather than compounding it. The resulting design recommendations --- the CA-swap architecture alone, at a 0.15 confidence threshold --- target real-time apple landmark detection for orientation-aware industrial sorting and alignment systems.

0.20

Multi-Scale Fruit Capsules: Dilated Convolutions and Dynamic Routing for In-the-Wild Explainable Fruit Recognition

cs.CV ·2026-08-20 ·Subhankar Chattoraj, Sawon Pratiher, Samiran Das, Hubert Konik
published similarity 0.68
Abstract · Accepted in IEEE Industrial Electronics Conference (IECON) 2026

The same fruit appears in a bunch, unpicked, peeled, bagged in plastic, or sliced on a dish, so automated fruit classification in the wild (AFCW) must absorb wide intra- class and narrow inter-class variability in shape, size, colour and texture. Convolutional networks route information through pooling, which discards the pose and location of the region of interest and therefore generalises poorly across these presentations. We propose FruitCapsNet, a capsule network whose Fruit Capsules replace the standard convolutional front end with dilated convolutions: the receptive field grows exponentially at constant parameter cost, so each capsule encodes multi-scale context before dynamic routing resolves part whole spatial agreement. Hyper-parameters, including the dilation factor, are selected by Bayesian optimisation rather than grid search. On three public datasets (SMP, FruitsGB, Fruits-360) and a new 19-class, 10,639-image in-the-wild dataset (PD-19), FruitCapsNet exceeds ten fine-tuned transfer-learning backbones at one-third the depth, with the largest margin (+2.7% over the nearest competitor) on the hardest set. Grad-CAM saliency propagated from the DigitCaps layer shows that the improvement comes from attributing decisions to whole-fruit regions rather than to object edges, giving post-hoc evidence that the gain is not a dataset artefact.

0.22

From Vision to Harvest: Benchmarking Vision-Language Models for Multi-Arm Robotic Fruit Harvesting

cs.RO cs.CV ·2026-09-11 ·Vrishan Inukollu, Adyan Zaman, Anvi Kudaraya, Carlos Lazcano, Yuankai Zhu +2 more
benchmark similarity 0.64
Abstract

Multi-arm robotic harvesting offers a promising path to improve harvesting efficiency and reduce reliance on manual labor. However, practical deployment remains challenging because the system must generalize across diverse environments while efficiently coordinating multiple arms in a shared workspace. Existing methods often require substantial data collection in target environments or rely on simplifying assumptions that limit planning quality. In this work, we introduce the first comprehensive benchmark for evaluating pretrained Vision-Language Models (VLMs) on zero-shot multi-arm fruit harvesting planning. Our benchmark uses real-world apple and citrus orchard images and compares a VLM-based planning pipeline with a traditional perception-and-planning pipeline. The VLM pipeline directly generates harvesting sequences and waypoints for each arm, while a lightweight trajectory verifier checks for collisions. Our results show that frontier VLMs can generate effective multi-arm harvesting plans zero-shot, but a practical deployment remains limited by accurate 3D waypoint generation and collision-aware coordination. These results highlight both the promise and current limitations of pretrained VLMs for multi-arm robotic harvesting.

0.28

BananaVLM: A Domain-Adapted Vision Language Model for Banana Crop Disease Diagnosis

cs.CV q-bio.QM ·2026-08-27 ·Sangam Kumar Jena, Pandarasamy Arjunan
benchmarkcode/data similarity 0.63
Abstract · 11 pages

Banana crop diseases threaten food security across the world, yet field diagnosis remains difficult because of limited expert access and visual similarity among disease classes. We present BananaVLM, a domain-adapted vision-language model for banana disease diagnosis built on LLaVA-v1.5-7B and fine-tuned with Low-Rank Adaptation (LoRA). To address the lack of multimodal agricultural instruction data, we introduce BananaInstruct, an automated pipeline that converts raw disease images into $\approx$80,000 question--answer pairs without manual annotation, using LLaVA-1.5-13B for symptom descriptions and Mistral-7B for agricultural Q&A generation and label grounding. Evaluated against 14 open-source and 5 closed-source VLMs on a nine-class benchmark, BananaVLM achieves 92.21% in-domain and 83.28% out-of-domain (OOD) classification accuracy, outperforming the best closed-source baseline, Gemini~2.5~Pro, by 49.8 and 63.3 percentage points, respectively. Binary healthy/diseased identification reaches 98.38% OOD accuracy with 1.00 recall. A controlled LoRA--DoRA comparison shows that DoRA performance degrades across epochs, while LoRA remains more effective for fine-grained classification. Qualitative evaluation using nine LLM judges yields G-Eval win rates of 0.81--0.96, while five domain experts prefer BananaVLM in 98.48% of 990 blind pairwise comparisons. These results show that lightweight domain adaptation with automated instruction tuning is an effective and scalable approach for specialized agricultural AI. Code, datasets, and model weights are available at https://github.com/samy101/banana-vlm

0.53

Beyond Black Boxes: An Ontology Based Vision Framework for Transparent Fruit Disease Detection

cs.CV cs.AI ·2026-10-07 ·M. J. Marapperuma, B. V. K. I. Vidanage
top venue similarity 0.62
Abstract

Efficiency and reliability in the timely identification of phytopathogens would help minimize crop loss in agricultural sectors but existing deep learning based methods, although accurate, would require a large annotated dataset and are black-boxed in their operation, thus, restricting flexibility and transparency. The current paper presents a hybrid detection framework that is able to overcome these drawbacks through the unified real-time vision with a semantic knowledge base. Web Ontology Language (OWL) was used to develop a comprehensive ontology of the disease of fruits that formally codifies phytopathological knowledge, such as symptom phenotype, spectral phenotype (HSV), and morphological phenotype. Such knowledge-guided configuration guides the image processing pipeline, making it possible to diagnose using this knowledge under expert specified rules without a large training dataset. The ontology underwent a computationally verified test of its structural soundness and commonsensical practical usefulness and scored an overall of 7.4/10. On the public set of MangoFruitDDS, the diagnostic performance was an F1 Score of 85.00%, precision of 85.00%, and a recall of 85.23%. These findings are very competitive and similar to other known deep learning models like the VGG16 that prove that the knowledge-driven approach provides an accurate, transparent, and scalable solution on automated fruit disease diagnosis.

0.45

HarvestPoint-ACT: Explicit Target Selection and Harvest-Point Conditioning for Robotic Fruit Harvesting under Occlusion

cs.RO ·2026-08-19 ·Hanying Hu, Weipeng Li, Yikun Huang, Hao Chen, Zhengtao Hu +2 more
similarity 0.61
Abstract · Submit to IEEE ROBIO 2026

End-to-end imitation learning avoids hand-made robot motion for approaching and grasping, but the policy must still decide which fruit to pick and where to close the gripper. Occlusion can make the policy lose the selected fruit during harvesting, and the correct closing point is difficult to infer from pixels alone. This paper presents HarvestPoint-ACT, which makes both decisions explicit in perception and provides them to the policy. An instance segmentation front end with a keypoint branch predicts a mask and a harvest point for each visible fruit, where the harvest point specifies the location to close the gripper. A scheduler ranks detected candidates by occlusion and travel distance and selects one target. After each attempt, it redetects and reranks the candidates because the canopy may have changed. The selected fruit is encoded for an action chunking transformer as an eight-dimensional state, containing the absolute harvest point, the vector from the gripper to that point, a validity flag, and a confidence score. When the selected fruit is temporarily undetected, the system retains the last harvest point estimate in the robot base frame and marks it as stale, and aborts the attempt if the loss persists. On a canopy mock-up, HarvestPoint-ACT achieves a success rate of 88%, and of 75% under heavy occlusion.

0.30

Towards Practical Precision Agriculture: Real-Time Fruit Detection and Video Analytics on Embedded Edge Hardware

cs.CV ·2026-09-11 ·Ivica Dimitrovski, Vlatko Spasev, Ivan Kitanovski, Petre Lameski, Dane Boshev
similarity 0.61
Abstract

Static-image benchmarks do not capture the computational and temporal requirements of practical orchard video analytics. This study presents an end-to-end framework for real-time fruit detection, tracking, and counting on the NVIDIA Jetson Orin Nano Super. A lightweight YOLO26s detector is trained independently on four public datasets representing apples, mangoes, blueberries, and strawberries under a common protocol. The models are deployed on embedded platform using PyTorch and TensorRT at FP32, FP16, and INT8 precision. APPLE MOTS is then used for temporal video analytics because it provides orchard sequences with persistent fruit identities, enabling evaluation of multi-object tracking and unique-fruit counting. The selected FP16 TensorRT detector is integrated into an NVIDIA DeepStream pipeline combining hardware-accelerated decoding, ByteTrack tracking, and motion-aware line-crossing analytics. Across the four detection tasks, mean test mAP@50:95 ranges from 0.4957 to 0.8656. On the Jetson, TensorRT FP16 achieves 66.76-74.56 images/s at 13.41-14.98 ms prediction latency, while reducing mAP@50:95 by only 0.0020-0.0054 and gross energy consumption by approximately 64-66% relative to PyTorch FP32. The complete detector-tracker-analytics pipeline reaches 44.96-54.11 FPS and sustains the configured 30-FPS input rate without output-frame loss. On held-out orchard video sequences, HOTA ranges from 0.345 to 0.538, event-level counting F1 from 0.611 to 0.803, and relative count error from 6.2% to 51.6%. Performance varies across acquisition geometries: near-lateral row viewing yields the most stable tracking and counting, whereas forward traversal remains association- and recall-limited despite spatially adaptive counting geometry. These results show that practical edge-based fruit monitoring requires efficient detection and acquisition geometries that support reliable temporal association.

0.17