The electronic eye conquers agriculture
CCTV, 3D, infrared, artificial vision the lenses pointed at roads, crops, parts of agricultural vehicles are multiplying. They are used to facilitate maneuvers, reduce the risk of collisions, and even to make some operations or the whole vehicle autonomous
There is an electronic eye – or rather thousands of electronic eyes – that constantly keep watch over the fields, headlands and yards of our farms. It displays images, collects data, activates when it detects movement or upon a command from the operator. It can see at night, distinguish a cultivated plant from a weed, and even drive a driverless vehicle. In recent years, with increasing frequency, the video camera has become a valuable ally in agricultural work, at various levels. Its smaller size (and cost) and improved performance is seeing it be employed more and more in various areas for the simplest and more obvious of uses, such as the surveillance of premises and rooms, the checking of blind spots on vehicles, on up to more recent and still somewhat experimental applications in automation and robotics. Let's try to become more familiar with this component that is increasingly present on agricultural machinery of all types and sizes.
CCTV. We begin by mentioning the surveillance functions common to agricultural, industrial, commercial and household applications. Given their low cost, excellent functionality and the ever-present threats from humans or animals, CCTV networks are now common in agricultural and livestock farms. They are used to monitor yards, entrances, tool sheds as well as stables and sheepfolds. They have night vision and motion detectors, which are activated only when something or someone approaches. Of greater interest for an agricultural mechanics magazine no doubt are the cameras mounted on various types of equipment and machinery that basically make the work of the operators easier by rendering blind spots or moments in procedures that require specific oversight more visible. The most classic example is that of the camera positioned behind bulky vehicles such as trailers, combine harvesters, forage harvesters, beet harvesters/potato diggers and the like. It activates automatically when the reverse gear is engaged, allowing the driver to maneuver with greater peace of mind and with a lower risk of accidents. Similarly, this type of camera might be placed on a ballast or tank (for fertilizer or grain) carried by the front lift. Here it would be used to assist the driver when operating in risky locations – gates, entrances to agricultural land on particularly narrow driveways – and exiting from secondary roads during transfers, when visibility from the cabin is obstructed by buildings or parked vehicles. Yet another potential application, very popular in recent times, is on machine parts that perform particularly delicate tasks or are out of the operator's direct line of sight. We now regularly see cameras mounted on the exhaust pipe of combine harvesters and on the spout of forage harvesters, for example, but also on the conveyor belt of beet harvesters or other harvesting machines. Moving on to less demanding uses, a camera can be useful on the rear of a baler to check the quality of the bales produced, or on a weeding tool, so as to keep the weeding elements in between the rows of plants without the need for a second operator or without the driver being forced to constantly look back.
Then, there are the less common, but no less useful, applications, such as inside of the grain tanks of a combine harvester, or near the cylindrical threshing drums that rotate and clean the beets from soil and stones. In the agricultural sector, cameras are used on telescopic handlers to get a clear view of where the load is going to be positioned. They can be mounted on the head of the telescopic arm, or on the forks of forklifts. In this instance, the operator can see exactly where he is going to deposit the pallet and what maneuver space is available to him.
Cameras for automation. Not all cameras used in modern digital farming are intended for the human eye. On the contrary, the images collected by many of them will never be viewed except by algorithms or artificial intelligence. These cameras are in fact used for the various levels of automation that are possible today: driving, controlling some particularly delicate work phases, and in autonomous vehicles. At present they are mainly prototypes, but in some cases they are already at work in fields all over the world. Let's start with one of the most common operations: automatic driving. This is usually achieved with a satellite connection, but in some cases a camera system can assist or replace the GPS signal. This occurs, for example, when working between rows, or on particular crops, such as row crops (corn, for example). In these cases, laser or artificial vision systems are more immediate and precise than satellite positioning. A camera mounted on the cutting bar, for example, allows a combine harvester to be easily guided, occupying the entire header and freeing the driver from having to constantly control the direction. An interesting development in this sphere is a recent application proposed by John Deere: a camera system to keep the tractor between the rows of crops when they have not been sown with the aid of satellite. Human planting never forms perfectly straight lines, and using a subsequent self-guided application – for example, weeding – would end up damaging the plants that fall outside the uniform pattern. The camera, on the other hand, diligently follows the rows, even if irregular, keeping the tractor wheels in the inter-row and even controlling the rear equipment. In other cases, the camera provides a surveillance function to certain activities that cannot be controlled by the human eye. For example, a camera placed in the combine harvester's auger tube is able to assess both the amount to recover (grain that escaped the first threshing) and grain breakage, which obviously represent a processing error. No human eye could count the broken kernels in a continuous, rushing flow of grain, but an electronic eye can. Similarly, special cameras are used on tomato harvesters: based on the color of the fruit, they are able to discard the unripe ones, dumping them in the field during harvest. Manual selection is always necessary at the end of the process, but the operators' task is greatly facilitated by the first sorting. In some cases, cameras are used to locate vegetation during specific operations, such as treatments with agrochemicals or herbicides. Due to the cost and danger of the products used, it is extremely important that the spraying is as targeted as possible, and some vehicles use computer vision to automatically turn the nozzles on and off. It should also be said, alternatively, that it is possible to use radar systems for these tasks, which are less affected by the fog created by the vaporization of agrochemicals.
At an even higher level of technology, cameras, paired with specific software and electric actuators, are capable of enabling specific operations, such as attaching the milking head to a cow's udder, targeted weeding or the hoeing of vegetables. In this case, the camera identifies the position and shape of the udder (or the weed), which will then be sprayed by the nozzle, or - in another instance - it is able to move a hoeing blade between one plant and another on a row of vegetables, in order to eradicate the weeds without damaging the crop. Another recent automation linked to cameras, is the automatic carriage filling function on some harvesting machines (typically forage harvesters).
The camera monitors where the product falls and moves the discharge chute back and forth to automatically and evenly fill the entire carriage.
The eyes of robots. Going from automatic filling to autonomous driving is not a short step: you have to go from a single repetitive operation to a complex activity, which involves moving freely in space, avoiding obstacles, finding the work field and, above all, making non-programmed decisions. However, the first Automatic Guided Vehicles (AGVs) are already on the market and many others are in the (fairly advanced) prototype stage. Whether on land or in the air, an autonomous vehicle needs to orient itself in space. This can occur in several ways. By use of a satellite system, for example, obviously using corrections that allow for a reduced margin of error. Or by laser devices, using the well-known Light Detection and Ranging (Lidar) method that works a bit like radar: light waves are emitted that bounce off objects and provide information about their shape and distance.
Robot vehicles can also use cameras. The latter, if applied to autonomous driving, are mostly 3D or stereoscopic: essentially two cameras are combined exactly like two human eyes. Composing two images through a specific algorithm allows the vehicle to recognize not only the objects surrounding it, but also their distance and thickness. A primary use of cameras is for safety: by training artificial intelligence (experts talk about neural networks) with thousands and thousands of images, computers can be "taught" to recognize human beings. Aitronik, a Pisan startup company specializing in software for AGVs deals with this. “Using three-dimensional cameras, our software recognizes humans in mixed environments, even identifying details and depth,” explains Alessandro Procopio, an engineer at Aitronik, adding that the camera systems are particularly useful for obstacle recognition, both outdoors and indoors. How to act if there is a human within its range of action is determined by the software combined with the vision devices.
The vehicle – let's say a tractor – can for example stop, or slow down, or change route, depending on the distance from the person in the frame and according to the latter's behavior. Information of this kind is more easily obtained with a three-dimensional camera, while classic monoscopic cameras are limited to recognizing the presence of a human being, without much other data. However, they do have the advantage of costing about a tenth of a good stereoscopic camera.
For autonomous driving, the camera already mounted on board can be used for safety reasons or for detecting vegetation or even its good condition. This alternative solution to Lidar reduces the manufacturing costs of autonomous vehicles. In other words, taking into account the cost of these devices, the aim is to make them multi-functional, meaning that they are capable of carrying out multiple similar tasks: environmental control, guidance, identification of obstacles or target plants. In any case, speaking with insiders one thing seems certain: within a few years – from three to five – the number of cameras on the market will multiply.
Not a month goes by without some new company arriving from the Far East offering cameras of various types, and even European companies with decidedly high-level reputations, which for now makes them preferable to their Eastern competitors, are working hard on these products. Whether two- or three-dimensional, black and white or infrared, the golden age of the camera appears to be only just beginning.
Factory-mounted cameras on the rise
OEM or aftermarket? Acronyms and foreign language models aside: are cameras mounted by the manufacturer better than those purchased and subsequently installed by the user? In reality, both have advantages and disadvantages. An initial deciding factor is the age of the tractor: on the most recent ones, especially if high-end or fully loaded with optional equipment, it is increasingly common to find at least one pre-installed camera (usually considered optional and as such paid for separately). On the other hand, cameras are practically mandatory on larger vehicles, such as combine harvesters and various harvesting machines. The main advantage of an OEM accessory is full integration with the machine's operating system. In the case of the rear view camera, for example, it is activated automatically when reverse gear is engaged. Furthermore, the images are displayed on the tractor's main monitor, which can often be divided into multiple screens so that the video footage is always available. A typical layout, for example, might be automatic guidance-camera-performance summary screen-ISOBUS screen of the equipment used. The market also offers dozens of solutions for surveillance cameras to purchase and mounted separately, sold by the most well-known groups. Installation can be done directly at the dealership, upon delivery of the vehicle, or by the owner: mounting them has in fact become extremely simple. As they say in the English-speaking world: plug and play. Latest generation cameras are in fact wireless. They connect via Wi-Fi to a monitor that can be installed in the cab with a suction cup or by attaching it to a support and are powered by rechargeable batteries, often with an attached power bank, i.e. a large-capacity battery that provides around 20 hours of battery life, enough for a few days of work. Assembly sometimes does not even require a screwdriver, since they can be attached to the vehicle using a magnet. So, it is clear that these solutions can be used by everyone and on all machinery. For example, they can be attached to a tool and then detached and moved to another tool once that type of work is finished. Some kits are supplied in a case for easier storage and transport specifically for this purpose. However, a wireless camera is not always the ideal solution. Not, for example, for some particular equipment, such as sewage tanks which, being bulky and made entirely of steel, might interfere with the signal and prevent it from reaching the display. In this case, a more traditional camera is needed, equipped with wires connecting it to the cabin and a power supply connected to the tractor's electrical system.
Finding Fruit and Problems with Computer Vision
An interesting application of cameras in agriculture is the monitoring of healthy vegetative condition, through multispectral cameras which contribute to provide a Normalized Difference Vegetation Index (NDVI) used to deduce crop areas with development problems (due to water or nutrient scarcity, soil subject to flooding or other). These tools can be mounted on a drone or even on a tractor or other off-road vehicle that drives across the field, mapping out its condition. In some cases, the camera is combined with a fertilizer spreader, which adjusts the dosage in real time to the health detected by the artificial vision system. Even more sophisticated are the systems - now in the prototype stage - that can detect produce, a foundational ability when building robots capable of autonomously picking the fruit.
