THE TECHNICAL VISION SYSTEM FOR DIAGNOSIS OF THE HARVESTING UNIT OF THE ROBOT FOR GATHERING WILD PLANTS A.I. Kuznetsov, A.V. Tyryshkin National Research Tomsk Polytechnic University Department of Control Systems and Mechatronics Tomsk, Russia kuznteh@gmail.com 2017

Topicality Advantages of using robots in agriculture: productivity increase; cost reduction; reduction of labor intensity of service; improving the quality of products.

Fig.1. Prognosis of dynamics of the market of agricultural robots Topicality Fig.1. Prognosis of dynamics of the market of agricultural robots

Topicality Reserves of cranberry marshes of Tomsk region more 10 million tons Manually collects no more than 15% of the entire crop. The use of robots promises increased collection efficiency.

Topicality The collection unit can often fail. In the event of a malfunction, the robot will continue to move, but the cranberries will not be collected. The diagnostic system will prevent the work from being "idle".

Review of existing solutions Fig.2.AGROBOT HUELVA SW6010 Fig.3. Octinion Fig.4. Shibuya Seiki Fig.5. BoniRob All existing systems are designed to work in artificial conditions.

Goals and objectives Main idea: create diagnosis system for solving problem controlling the operation of harvesting unit. Function: diagnosis of the harvesting unit determination of the efficiency of berry picking; increase the convenience of servicing the robot; improving the robot's economy. Goals: choose of the principle of the system; choose of the realization method; development of the work algorithm; realization of the algorithm; debugging and adjustment.

Choose of the principle of the system Monitoring the details Control of the main function Benefits: simplicity of the diagnostic algorithm; completeness of information about the state of the parts of the node. Benefits: tolerance to various design options; the possibility of collecting statistical data; lack of direct contact with responsible details; ease of installation and transfer of the system.

Choose of the realization method Main function of harvesting unit is – decrease in the number of berries

Figure of the proposed solution Computer Harvesting unit Autonomous platform Camera №1 Camera №2 Fig.6. Harvesting robot

Development of the work algorithm Main functions: determination of the malfunction of the harvesting unit; determining the efficiency of the harvesting unit; control zones without berries; setting the parameters of the recognized berries; setting options for displaying error messages.

Algorithm Fig.7. Block diagram

Development of the function of recognition and counting of berries One of the main distinctive features of cranberries is the color.

Development of the function of recognition and counting of berries Fig.8. Dependence of the color of the berries on different lighting conditions

Setting cranberry options Select a sample of the berries from the photo. Select a color sample from the tone ruler.

Development of the function of recognition and counting of berries Fig.9. Recognition results for a given color

Development of the function of recognition and counting of berries Fig.10. The graph of the dependence of the amount of the recognized berries on the size of the color range of the berries

Development of the function of recognition and counting of berries

Development of the function of recognition and counting of berries Noises: presence of parasitic objects of similar color; complex lighting conditions that require a larger range of berry color.

Development of the function of recognition and counting of berries A contour is a line that delimits an object and characterizes its shape. Border - the place of sharp changes in color, brightness or other color coordinates.

Development of the function of recognition and counting of berries Выделение границ с использованием контрастности Fig.11. Searching for borders on image with transformation by the Laplace operator

Methods of preprocessing Fig.12. Contrast Fig.13. Sharpness

Methods of preprocessing Fig.14. Searching for borders with contrast preprocessing Fig.15. Searching for borders with sharpness preprocessing

Development of the function of recognition and counting of berries To determine the conformity of the form, the Hafa transformation method is used.

Development of the function of recognition and counting of berries

Service information The average efficiency factor is used. Finding zones without berries. Fig.16. Service information

Advantages Simplify the maintenance of the robot. Increase the working time. Access to statistical indicators. Savings of money due to the lack of a permanent operator.

Conclusion A system for diagnosing of the harvesting unit of the robot for gathering wild plants was developed. Algorithms for detecting and counting cranberries in the image are created. Complex external conditions are taken into account. The output of service information

THANK YOU FOR ATTENTION A.I. Kuznetsov, A.V. Tyryshkin National Research Tomsk Polytechnic University Department of Control Systems and Mechatronics Tomsk, Russia kuznteh@gmail.com 2017