An Autonomous Weeding Robot for Organic Farming Tijmen Bakker1 , Kees van Asselt1 , Jan Bontsema2 , Joachim M¨ uller3 and 1 Gerrit van Straten 1 2 3
Wageningen University, Systems and Control Group, P.O. Box 17, 6700 AA Wageningen, The Netherlands,
[email protected] Agrotechnology and Food Innovations BV, P.O. Box 17, 6700 AA Wageningen, The Netherlands University of Hohenheim, Institute for Agricultural Engineering, 70593 Stuttgart, Germany
Summary. The objective of this research is the replacement of hand weeding in organic farming by a device working autonomously at field level. The autonomous weeding robot was designed using a structured design approach, giving a good overview of the total design. A vehicle was developed with a diesel engine, hydraulic transmission, four-wheel drive and four-wheel steering. The available power and the stability of the vehicle does not limit the freedom of research regarding solutions for intra-row weed detection and weeding actuators. To fulfill the function of navigation along the row a new machine vision algorithm was developed. A test in sugar beet in a greenhouse showed that the algorithm was able to find the crop row with an average error of less than 25 mm. The vehicle is a versatile design for an autonomous weeding robot in a research context. The result of the design has good potential for autonomous weeding in the near future.
Keywords: Systematic design, machine vision, GPS, robotics, intra-row weed control, autonomous weeding robot, organic farming
1 Introduction Weeds in agricultural production are mainly controlled by herbicides. As in organic farming no herbicides can be used, weed control is a major problem. While there is sufficient equipment available to control the weeds in between the rows, weed control in the rows (intra-row weeding) still requires a lot of manual labour. This is especially the case for crops that are slowly growing and shallowly sown like sugar beet, carrots and onions. In 1998, on average 73 hours per hectare sugar beet were spent on hand weeding in the Netherlands [4]. The required labour for hand weeding is expensive and often not
P. Corke and S. Sukkarieh (Eds.): Field and Service Robotics, STAR 25, pp. 579–590, 2006. © Springer-Verlag Berlin Heidelberg 2006
580
T. Bakker et al.
available. An autonomous weeding robot replacing this labour, could mean an enormous stimulus for organic farming. This paper presents the design of such an autonomous weeding robot currently being developed at Wageningen University.
2 The Design Procedure 2.1 Method The autonomous weeding robot is designed using a phase model as the design method[3]. In this phase model the design of a product is represented as a process consisting of a problem definition phase, alternatives definition phase and a forming phase (figure 1). The results of the different phases are solutions on different levels of abstraction. The problem definition phase starts with defining the objective of the design. In the problem definition phase also the set of requirements is established. The requirements can be split into fixed and variable requirements. A design that does not satisfy the fixed requirements is rejected. Variable requirements have to be fulfilled to a certain extent. To what extent these requirements are fulfilled, determines the quality of the design. The variable requirements are also used as criteria for the evaluation of possible concept solutions. The last part of the problem definition phase consists of the definition of the functions of the robot. A function is an action that has to be performed by the robot to reach a specific goal. In our case, important functions are ’intra-row weeding’ and ’navigate along the row’. The functions are grouped in a function structure, which represents a solution on the first level of abstraction. The function structure consists of several functions. Every function can be accomplished by several alternative principles, e.g. mechanical and thermal principles for weed removal. In the alternatives definition phase, possible alternative principles for the various functions are presented in a morphological chart (fig. 3). The left column lists the functions and the rows display the
Fig. 1. The design process
An Autonomous Weeding Robot for Organic Farming
581
alternative principles. By selecting one alternative for each function and by combining these alternatives, concept solutions can be established. These concept solutions are represented by lines drawn in the morphological chart. The best concept solution can be selected using a rating procedure. In the forming phase this selected concept solution is worked out into a prototype. 2.2 Application for the Weeding Robot The objective of the research is formulated as ’replacement of hand weeding in organic farming by a device working autonomously at field level’. Starting from this objective, the first step in the problem definition phase was to establish the set of requirements. For this purpose interviews were held with potential users, scientists and consultants related to organic farming. The resulting requirements are as follows: Fixed requirements: • Replacing hand weeding in organic farming. • Applicable in combination with other weed control measures. • Manual control of the vehicle must be possible for moving the vehicle over short distances. • Weeding a field autonomously. • Ability to work both day and night. • The weeding robot should not cross the field boundaries. • The weeding robot must be self restarting in absence of emergency.
Fig. 2. The function structure
582
T. Bakker et al.
• The weeding robot informs the farmer when the weeding robot stopped definitely (e.g. due to security reasons) or when it is ready. • The weeding robot sends its operational status to the user at request • The weeding robot must function properly in sugar beet. Variable requirements: • Removing more than 90 percent of the weeds in the row. • The costs per hectare may be at least comparable to the costs of hand weeding. • Damage to the crop is as low as possible. • The soil pressure under the weeding robot must be comparable or less than for hand weeding. • Energy efficient. • Safe for people, animals and property. • Suitable as research platform. • Limited noise production • Reliable functioning. • Easy to use. After establishing the set of requirements the functions of the the weeding robot were identified. These functions were grouped into a function block scheme. This scheme is represented in figure 2. The lines in the scheme indicate flows of energy, material or information. Functions located in parallel lines can be performed simultaneously. The navigation system consists of four functions. Firstly, the weeding robot should constantly determine if it is located in- or outside the field. Secondly, if within the field, it should determine if it is on one of the headlands or not. Thirdly, in case it is not on the headlands, it should navigate along the row and perform the intra-row weeding. Fourthly, if the weeding robot arrives on the headland, it should stop the intra-row weeding and start to navigate to the next crop rows to be weeded. This sequence repeats until the whole field, except the headlands, is weeded. Weeding of the headlands is left out of consideration. An increasing number of farmers in the Netherlands do not grow sugar beet at the headlands because they think it is not cost-effective. In the alternatives definition phase possible alternative principles for the various functions are listed in a morphological chart (fig. 3). Four people involved in the project drew lines indicating possible concept solutions in the chart. These concept solutions were then weighed against each other using the variable requirements listed before. The concept solution indicated by the line in figure 3 is the final concept solution. In the forming phase described in section 3 the concept solution was worked out into a prototype.
An Autonomous Weeding Robot for Organic Farming
Fig. 3. Morphologic chart
583
584
T. Bakker et al.
2.3 Results of the Design Process Determine where intra-row weeding has to be performed To determine where intra-row weeding has to be performed, pattern recognition of plant locations is going to be used. From earlier research [2] it is expected that the quality of detection of this method is at least as good as the quality of detection of other methods. Though combinations of methods like recognition of pattern, shape and colour are expected to have a potential for higher quality of detection, just pattern recognition is chosen because it is expected to be sufficient. Positioning of weeding To position the actuator at the location indicated by the detection system dead reckoning is going to be used. A wheel with encoder, giving a precise distance measurement, will be available already because it is also needed for the pattern recognition system. Intra-row weeding Intra-row weeding will be performed by a mechanical actuator. It is expected to be difficult to remove weeds growing close to a crop plant by air, flaming, electricity, hot water, freezing, microwaves or infrared without damaging the crop plants. In that respect laser would be an excellent solution. However, laser can not work under the ground surface, and has therefore less effect on certain weed species. On the other hand, not moving the soil prevents buried seeds from germinating. A greater disadvantage of laser is its high price. High power laser is needed to reach reasonable performance, and this involves high costs. Water-jet could also probably be a good solution for intra-row weeding, but this needs much more investigation than a mechanical solution. Determine if within field GPS is selected to determine wether the weeding robot is within the field or not. The determination if the weeding robot is located within the field or not, needs to be guaranteed correctly at any time. A combination of vision and dead reckoning can not give this guarantee as good as a solution in which GPS is used. Dead reckoning could improve the position determination by GPS. However, if a GPS is selected with sufficient accuracy, additional dead reckoning is not needed. Navigate along the row Machine vision is selected for navigation along the row. Machine vision makes it possible to navigate along the row by relative positioning to the row. Therefore the weeding robot can work in any field without requiring absolute coordinates of a path to be followed. Absolute positioning by means of GPS,
An Autonomous Weeding Robot for Organic Farming
585
possibly combined with other sensors, requires knowledge of the absolute position of crop rows in a field. Navigation along the row by relative positioning to the row could be done also using tactile, ultrasonic or optical sensors combined with dead reckoning. Tactile sensors are not going to be used because in case of sugar beet they could harm the crop. Machine vision is preferred over ultrasonic or optical sensors, because of the ability to look forward, which contributes to a more accurate control of the position of the weeding robot relative to the crop row. It is not clear wether dead reckoning could substantially contribute to the navigation accuracy feasible with machine vision. Determine if on headland GPS is selected to determine if the weeding robot is located on the headland. Using GPS requires some labour for recording the border of the headlands in advance, but will result in a correct headland detection. If a high accuracy GPS is selected, accuracy does not have to be improved by dead reckoning. Tactile, ultrasonic or optical sensors in combination with dead reckoning could also be used to determine wether the robot is on the headland, by detecting the end of the row, i.e. if over some predefined distance no row is detected. However, another crop may grow on the headland (seeded to prevent germinating of weeds) or crop rows seeded at the headland can cross the crop row to be followed. In these situations the latter solutions can not guarantee a correct detection of the end of row, and therefore also not a correct headland detection. Machine vision could give more reliable results, but it is still difficult because headland to be detected is not so structured. Navigate on headland For navigation on the headland GPS is selected. On the headland the weeding robot has to make a turn and position itself in front of the next rows to be weeded. At the moment the robot arrives at the headland, a virtual path is planned to a position in front of the next rows to be weeded. Navigating over this path is going to be done by GPS. Locomotion related functions A diesel engine with a hydraulic transmission was selected for the locomotion related functions. For weeding quite some power could be required and the available power should not be limiting for realizing the objective of autonomous weeding of a field. A diesel engine with an hydraulic transmission is a proven concept in agriculture. A gearbox limits the possible combinations of the number of engine revolutions and driving speed and shuffling is difficult to automate. A continuously variable or hydraulic transmission is therefore preferred over a gearbox. Hydraulics makes it possible to design a compact wheel construction preventing damage to the crop. A design with four wheels is preferred over one with three because of stability.
586
T. Bakker et al.
It was decided that four wheels is also preferred over two or four tracks. The most important advantages of tracks in practice are the better traction and the less soil compaction. But it is expected that if four wheels are used for such a light-weight vehicle (not more than 1500 kg) soil compaction will be acceptable. Traction when using wheels is expected to be good enough because of the limited weight and the limited need of traction for intra-row weeding. Four wheel drive and four wheel steering were chosen to have the possibility to investigate all kinds of driving strategies. Communication with the user Specific settings for a field will be defined by a board computer. Any moment a user wants to know the status of the weeding robot, the weeding robot status will be accessible via the internet. A website gives good opportunities to represent information in an orderly way and it is easily accessible from everywhere. In case the weeding robot needs help from its user, the weeding robot notifies its user by sending an SMS (Short Message Service) message by the GSM network. In the Netherlands any place is covered by the GSM network. From the alternatives listed, SMS is the solution that gives the highest assurance that the user really receives the message shortly after it is sent. Detect unsafe situations Detecting unsafe situations will be done super canopy all around the weeding robot. Situations in which this solution is not sufficient are hardly imaginable. Ideally the weeding robot should detect every unsafe situation, at every level and direction. Even if somebody is lying in between the crop rows below canopy level this should be detected. Because of the research effort involved in reaching the ideal objective mentioned and the possible high costs for such a solution, detecting around and only super canopy is preferred.
3 The Vehicle The size of the vehicle was determined by the standard track width used in agriculture in the Netherlands which is 1.50 m. This track width also makes the design versatile in the sense that it is suitable for crops grown in beds like carrots an onions. See figure 4 for the resulting vehicle. Sugar beets are grown at a row distance of 50 cm so the weeding robot covers three rows. The engine power is selected so that it has enough power for driving and steering under field conditions and for driving three actuators. The required power for the actuators was calculated based on an actuator specially designed for intra-row weeding by Bontsema et al. [2]. The engine is a 31.3 kW Kubota V1505-T. The ground clearance is about 50 cm to prevent the crop from being damaged by the vehicle. The vehicle is 2.5 m. long to have enough space for mounting
An Autonomous Weeding Robot for Organic Farming
587
Fig. 4. The weeding robot
actuators under the vehicle in the middle between the front and rear wheels. The tyre width of 16 cm leaves enough space for steering in between crop rows while soil compaction is expected to be acceptable. The weight of the vehicle is about 1250 kg. The engine drives two hydraulic pumps. One supplies the oil for steering and driving, and the other for driving the actuators. The oil for driving and steering flows to a electrically controllable valve block with eight sections. Four are used for steering and four are used for controlling wheel speed, so wheel speeds and wheel angles can be controlled individually. The wheels are driven by radial piston motors. The required driving speed range for intra-row weeding is 0.025 m/s - 2 m/s continually variable. A desired top speed of 5.6 m/s was specified for fast moving of the robot within a field. It appeared that hydraulics could not be designed to have a variable work speed from 0.025 m/s to 5.6 m/s. A solution was found by designing the hydraulics so that two speed ranges exist. The working speed ranges up to 3.2 m/s. A maximum travel speed of 6.4 m/s is realized by changing to two wheel drive by combining the oil flows of four wheels into two flows. Each wheel is steered by an hydraulic motor with a reduction gear. The maximum steering speed is 360 degrees per second. The angles of the wheels are measured by angle sensors. The oil for driving the wheels flows via a turnable oil throughput. This makes it possible to turn the wheels in any angle from 0-360 degrees. The weeding robot electronics consists of 6 units connected by a CAN bus
588
T. Bakker et al.
Fig. 5. Electronics architecture
using the ISO 11783 protocol. In figure 5 an schematic overview is given of this system with vehicle control related sensors and valves. Four micro controllers are located near the four wheels to measure the wheel speed and the wheel angle. Angles, wheel speeds and wheel direction are transmitted using the CAN bus. Via the CAN bus and two other micro controllers the hydraulic valve block is controlled. One laptop processes images supplied by the camera connected and returns the location of the crop rows in relation to the vehicle position in a CAN bus message. Another laptop does the vehicle control. It gathers wheel speed, wheel direction, crop row location data and GPS data and controls the vehicle by sending messages to the units connected to the valve block. This laptop is also the user interface of the weeding robot. A remote control is connected to this laptop via a radio modem for manual control of the weeding robot. Besides the sensors directly related to navigation and control, there are some more sensors connected to the modules. These sensors indicate oil filters functioning, oil temperature and oil level are also interfaced to the laptop. If a sensor indicates an emergency, the weeding robot will turn off automatically.
4 Navigation Along the Row As explained in section 2.2 part of the navigation system of the weeding robot will consist of navigating along the row using machine vision. The machine vision algorithm was developed and tested on a sugar beet field prepared in a greenhouse. The area covered by one image was 2.5 meters long in row direction and 1.5 meters wide at the side closest to the camera. This means that three complete rows are visible in the image. The first step in the row recognition algorithm, is transforming the RGB image to a grey scale image with enhanced contrast between green plants and soil background. The next step is to correct the images for perspective by an inverse perspective transformation. In the corrected image three rectangular sections of crop row spacing are selected. The first section is selected in the middle of the image. The other two are selected on both sides of the first section. The sections are combined by summing up the grey values of the sections to a combined image. To the
An Autonomous Weeding Robot for Organic Farming
A
B
589
C
Fig. 6. Typical images together with estimated row position at different growth stage and weed pressure
resulting combined image grey scale Hough transform is applied. Measurements show that the algorithm is able to find the row independent of the crop stage. Furthermore, the algorithm has found the crop row in images with a high weed density. Some typical results can be found in figure 6. The quality of the row position estimation is determined by comparing the lines found by the algorithm with lines positioned over the crop rows by hand in the original image. The average deviation between the estimated and real crop row varied from 6 to 223 mm. The higher deviations in this range can be explained by the number of plants visible in early crop stage, overexposure of the camera, and the presence of a lot of green algae due to our experimental setup. Ignoring the measurements under these extreme conditions, the algorithm was able to find the row with an average error of less then 25 mm. The measured processing time varied from 1 to 1.5 seconds per image. This variation can be explained by the varying amount of weed. The more light pixels there are, the more pixels have to be processed by the Hough transform. Details can be found in [1].
5 Conclusions The advantage of using a structured design procedure is that it provides a good overview of the complete design. Also, the design method forces the designer to look at alternative solutions. Because of the structured sequence of design activities, it is easy to keep track of the progress of design. In a research context it is easy to identify alternative subjects that are worthwhile to investigate further. But in the mean time the main line of the research remains clear.
590
T. Bakker et al.
Applying the design procedure for the autonomous weeding robot resulted in a flexible research vehicle. The design consisting of diesel engine, hydraulic transmission, four wheel drive and 360 degrees four wheel steering is a good concept for an autonomous weeding robot in a research context. The available power and the stability of the vehicle does not limit the freedom regarding research to solutions for intra-row weed detection and weeding actuators. From the established functions only for navigation along the row a new algorithm to detect sugar beet rows is discussed. The algorithm is able to find the row with an average error of less than 25 mm. Processing one image covering 2.5 meters row length takes less than 1.5 second. It is not expected that the attainable driving speed of about 1.5 m/s will be limiting. From earlier research it is expected that the actuator will limit the driving speed to 1 m/s or less. So it can be concluded that the results of the algorithm give good perspectives to navigate an autonomous vehicle along rows in a sugar beet field. The planning for the current year is to finish the autonomous navigation and control of the weeding robot. This will be tested in a sugar beet field. Adding an intra-row weeding system is planned for next year. The ultimate test will then be to show that it is possible to weed a whole sugar beet field autonomously by a weeding robot.
References [1] T. Bakker, H. Wouters, C.J. van Asselt, J. Bontsema, J. M¨ uller, G. van Straten, and L. Tang. A vision based row detection system for sugar beet. In Computer-Bildanalyse in der Landwirtschaft. Workshop 2004, Bornimer Agrartechnische Berichte, pages 42–55, Braunschweig, Germany, 2004. Institut f¨ ur Agrartechnik Bornim e.V. [2] J. Bontsema, C.J. van Asselt, P.W.J. Lempens, and G. van Straten. Intrarow weed control: a mechatronics approach. In 1st IFAC Workshop on Control Applications and Ergonomics in Agriculture, pages 93–97, Athens, Greece, 1998. [3] H.H. van den Kroonenberg and F.J. Siers. Methodisch ontwerpen. Ontwerpmethoden, voorbeelden, cases, oefeningen. Educatieve Partners Nederland BV, Houten, 1998. [4] R.Y. van der Weide, L.A.P. Lotz, P.O. Bleeker, and R.M.W. Groeneveld. Het spanningsveld tussen beheren en beheersen van onkruiden op biologische bedrijven. In F.G. Wijnands, J.J. Schroder, W. Sukkel, and R. Booij, editors, Themaboek 303. Biologisch bedrijf onder de loep, pages 129–138. Wageningen Universiteit, Wageningen, 2002.