CROPS EU-project

  • Increase font size
  • Default font size
  • Decrease font size

WP4: Intelligent Sensor Fusion and Learning Algorithms

Print PDF

WP4 is devoted to the research and development of intelligent algorithms for sensing and grasping in the cRops robotic platform. The algorithms aim to enable the cRops robotic platform to perform robustly and efficiently in a variety of operating and sensing conditions by fusing the information from multiple and different types of sensors, and to easily adapt to new varieties of crops and to crops other than those of the cRops project through continuous learning. Specifically, WP4 focuses on the development and implementation of:

  • Adaptive sensor fusion framework and algorithms (T4.1)
  • Learning algorithms for the different sensing tasks (T4.2)
  • Learning algorithms for grasping (T4.3)

and evaluation and benchmarking (T4.4) of algorithms developed.

We have developed, implemented and tested algorithms for:

  • Adaptive threshold algorithm
  • Adaptive sensor fusion algorithm
  • Learning in sensing
  • Graspability maps generation
  • Evaluation and benchmarking for grasping

grasping of apple

The WP includes four tasks:

As part of T4.1 Adaptive sensor fusion, an adaptive sensor fusion algorithm and an algorithm for determining thresholds adaptively were implemented and analyzed for two databases. The databases include several apple and pepper databases. The results were analyzed in respect to several parameters.

Adaptive sensor fusion

As part of T4.2 Learning in sensing, a database with forest data was acquired in Swedish forests. Data comprises RGB images and laser data for scenes with trees, humans, rocks, and bushes. A classifier system was constructed and algorithms for image segmentation and semi-automatic labeling were developed.

As part of T4.3 Learning in grasping theoretically grounded grasp quality measures that can be computed directly from a point cloud were developed along with efficient graspability map computation methods that facilitate map computation during run-time. The computed maps were tested in a physical environment. Dynamic motion primitives (DMP) were applied to apple harvesting a laboratory setup. The parameters of each DMP were learned based on demonstration.Gripper representations Good grasp regions pepper

Modelling a gripper for pepper fruits to determine successful grasp poses (red rectangle in right image)

As part of T4.4 Evaluation and benchmarking, a new methodology for evaluating the connection between gripper design and sensing was developed in addition to formulating measures for sensor fusion evaluation.

Processing pipeline

WP4 related publications (for a complete overview see the Dissemination page)


  1. Ohev-Zion, A., Shapiro, A. 2011. Grasping of Deformable Objects  Applied to Organic Produce. Proc. of TAROS 2011, LNAI 6856.
  2. Shemesh, M., Ben-Shahar, O. 2011. Free Boundary Conditions Active Contours with Applications for Vision. Proc. of International Symposium on Visual Computing (ISVC).
  3. Eizicovits, D., Yaacobovich, M., Berman, S. 2012. Discrete fuzzy grasp affordance for robotic manipulators. IFAC Symposium on Robot Control, SYROCO.
  4. Yaacobovich, M., Eizicovits, D., Berman, S. 2012. Grasp Affordance for Robotic Selective Harvesting based on Human Demonstrations. International Conference of Agricultural Engineering CIGR-AgEng.
  5. Eizicovits, D., Vitzrabin, E., Edan, Y., Berman, S. 2013. Integration of a Robotic Harvester System Module. Israel Agricultural Conference (ISAE).
  6. Eizicovits, D., Berman, S. 2013. Constructing successful grasps based on Graspability maps. Israeli Conference on Robotics (ICR).
  7. Vitzrabin, E., Edan, Y. 2013. Apple detection using multi-dimensional adaptive thresholding with multi-resolution windows. Israeli Conference on Robotics (ICR).
  8. Hershkovitz-Cohen, A., Berman, S. 2013. Path Planning of Manipulator for Harvesting using DMP. Israeli Conference on Robotics (ICR).
  9. Barnea, E., Ben Shahar, O. 2014. Depth Based Fruit Detection from Partial Pose Estimation using Symmetry. To be presented at AgenG International Conference (July 2014, Zurich).
  10. Meiron, R., Ben Shahar, O. 2014. Highlight detection with application to sweet pepper localization. To be presented at AgenG International Conference (July 2014, Zurich).
  11. Oberty, R., Marchi, M., Tirelli, P., Vitzrabin, E., Edan, Y. 2014. Sensor fusion of multispectral and hyperspectral imaging: preliminary analysis of disease detection in grapevine. To be presented at AgenG International Conference (July 2014, Zurich).
  12. Ohev-Zion, A., Shapiro, A. 2014. Stability of compliant planar robotics grasp, with application to fruits grasping. To be presented at AgenG International Conference (July 2014, Zurich).
  13. Vitzrabin, E., Edan, Y. 2014. Multi-dimensional adaptive thresholding for apple detection. To be presented at Israel Agricultural Conference (ISAE) 2014.
  14. Reshef, R., Eizicovits, D., Berman, S. 2014. Path Planning for Selective Harvester Robot. To be presented at Israel Agricultural Conference (ISAE) 2014.


  1. Kapach, K., Barnea, E., Mairon, R., Edan, Y., Ben-Shahar, O. 2012. Computer Vision for Fruit Harvesting Robots - State of the Art and Challenges Ahead. Int. J. of Computational Vision and Robotics 3(1/2): 4-34.
  2. Ben-Yosef, G., Ben-Shahar, O. 2012. A tangent bundle theory for visual curve completion. IEEE Transactions on Pattern Analysis and Machine Intelligence 34: 1263-1280.
  3. Eizicovits, D., Berman, S., 2014, Efficient sensory-grounded grasp pose quality mapping for gripper design and online grasp planning. Accepted for publication in Robotics and Autonomous Systems.
  4. Bac, C.W., Hemming, J., Henten, E. van., Edan, Y. 2014. Harvesting robots for high-value crops: state-of-the-art review and challenges ahead. Accepted for publication in Journal of Field Robotics.

Several additional journal papers submitted.

Last Updated on Saturday, 10 May 2014 18:16