# Localizing Small Apples in Complex Apple Orchard Environments

**Authors:** Christian Wilms, Robert Johanson, Simone Frintrop  
**arXiv:** [2202.11372](https://arxiv.org/abs/2202.11372)  
**Date:** 2022-02-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2202.11372v1/

## Key points

- Localization of fruits is important for yield estimation and fruit picking in automated agriculture
- Small apples present a challenging problem due to their size and complex orchard environments
- Christian Wilms et al. propose two approaches based on AttentionMask, an object proposal generation system that focuses on small objects
- The first approach involves adding a new module specifically designed for very small apples, while the second integrates AttentionMask into a tiling framework
- Both methods outperform standard object proposal generation systems on the MinneApple dataset covering complex apple orchard environments
- AttentionMask works by splitting its backbone into two parts to generate pyramid levels and extract windows from relevant areas containing objects of interest
- The proposed methods build upon AttentionMask's state-of-the-art results in object proposal generation and its strong performance in complex real-world environments
- The extended feature pyramid allows for selective extraction of only relevant windows, making it more efficient than other systems
- These proposed methods offer promising solutions for localizing small apples in complex orchard environments and could potentially be applied to other types of fruits as well.

## Summary

In the field of automated agriculture, the localization of fruits is a crucial first step for yield estimation or fruit picking. One such example is the localization of small apples in images of entire apple trees, which presents a challenging problem due to their size and complex orchard environments. To address this issue, Christian Wilms et al. propose two approaches based on AttentionMask, an object proposal generation system that focuses on small objects. The first approach involves adding a new module specifically designed for very small apples, while the second integrates AttentionMask into a tiling framework. Both methods outperform standard object proposal generation systems on the MinneApple dataset covering complex apple orchard environments. AttentionMask works by splitting its backbone into two parts: one processes the input image to generate a base pyramid level S8, while further pyramid levels are created using residual necks or the second part of the backbone (orange). Per pyramid level, an attention map focuses on extracting windows from relevant areas containing objects of interest (indicated by red areas in heat maps). A segmentation and an objectness score are generated for each extracted window (green windows) to separate objects from backgrounds and provide confidence scores. The proposed methods build upon AttentionMask's state-of-the-art results in object proposal generation and its strong performance in complex real-world environments. The extended feature pyramid allows for selective extraction of only relevant windows, making it more efficient than other systems. The evaluation further analyzes improvements with respect to apple sizes and shows different characteristics between the two approaches. Overall, these proposed methods offer promising solutions for localizing small apples in complex orchard environments and could potentially be applied to other types of fruits as well.