Deep learning for automated weed monitoring in grain crops

Crop yield reduction under the influence of biotic stressors such as weeds remains a pressing issue in agroecosystems. Traditional methods of weed infestation monitoring based on visual assessment are laborintensive and subjective. In this paper, an approach to automated weed identification in spring wheat and barley crops using ResNet convolutional neural networks (CNNs) is proposed. The objective of the study was to develop classifiers based on ResNet-18, ResNet-34, and ResNet-50 architectures to detect 16 weed species and determine the need for herbicide treatment. The dataset included 138 images with a resolution of 1340 × 1790 pixels obtained from a mobile camera and phytosanitary monitoring data from 66 survey plots (0.25 m²). To compensate for the small amount of data, augmentation was used (aug_transforms library, PyTorch) with the following operations: random rotations, scaling, brightness and contrast correction. This allowed us to expand the sample by at least 5 images per class. The images were preprocessed: scaling to 512 × 512 pixels with subsequent compression to 224 × 224 for compatibility with ResNet. The models were trained for 100 epochs with the Adam optimizer (batch size = 16), the quality metrics were accuracy_multi (the proportion of correct classifications) and F1-score. All models achieved high accuracy (accuracy_multi > 95%), but the F1-score ranged from 0.60 to 0.74, which reflects the complexity of multi-class classification. ResNet-18 demonstrated the highest F1 scores (0.74). Confusion matrix analysis revealed problematic classes: Convolvulusarvensis (6 errors), Sinapisarvensis (4 errors), which is due to insufficient data representativeness. Classes 4, 7–10, and 14 were recognized with the highest accuracy. Confusion Matrix also revealed ambiguous effectiveness of deep ResNet architectures.

1. Lesk C., Anderson W., Rigden A., Coast O., Jägermeyr J., McDermid S., Davis KF., Konar M., Compound heat and moisture extreme impacts on global crop yields under climate change, Nature Reviews Earth & Environment, 2022, Vol. 3, No. 12, pp. 872–889, DOI: 10.1038/s43017-022-00368-8.

2. Zandalinas SI., Mittler R., Balfagón D., Arbona V., Gómez‐Cadenas A., Plant adaptations to the combination of drought and high temperatures, Physiologia plantarum, 2018, Vol. 162, No. 1, pp. 2–12, DOI: 10.1111/ppl.12540.

3. Hussain H. A., Hussain S., Khaliq A., Ashraf U., Anjum S. A., Men S., Wang L., Chilling and drought stresses in crop plants: implications, cross talk, and potential management opportunities, Frontiers in plant science, 2018, Vol. 9, Р. 393, DOI: 10.3389/fpls.2018.00393.

4. Zimdahl R.L., Basinger N.T., Fundamentals of weed science, Elsevier, 2024, P. 560.

5. MacLaren C., Storkey J., Menegat A., Metcalfe H., Dehnen-Schmutz K., An ecological future for weed science to sustain crop production and the environment. A review, Agronomy for Sustainable Development, 2020, Vol. 40, pp. 1–29, DOI: 10.1007/s13593-020-00631-6.

6. Simard M.J., Laforest M., Early detection and management of herbicide-resistant weeds, Canadian Journal of Plant Science, 2024, Vol. 104, No. 6, pp. 533–539, DOI: 10.1139/cjps-2024-0021.

7. Geddes C.M., Page E.R., Bae J., Forging new paths to manage herbicide-resistant weeds, Canadian Journal of Plant Science, 2024, Vol. 105, pp. 1–5, DOI: 10.1139/cjps-2024-0190.

8. Kamilaris A., Prenafeta-Boldú F.X., Deep learning in agriculture: A survey, Computers and electronics in agriculture, 2018, Vol. 147, pp. 70–90, DOI: 10.1016/j.compag.2018.02.016.

9. Torres-Sánchez J., Mesas-Carrascosa F.J., Jiménez-Brenes F.M., de Castro A.I., & López-Granados F., Early detection of broad-leaved and grass weeds in wide row crops using artificial neural networks and UAV imagery, Agronomy, 2021, Vol. 11, No. 4, Р. 749, DOI: 10.3390/agronomy11040749.

10. Mesías-Ruiz G.A., Pérez-Ortiz M., Dorado J., De Castro A.I., Peña J.M., Boosting precision crop protection towards agriculture 5.0 via machine learning and emerging technologies: A contextual review, Frontiers in Plant Science, 2023, Vol. 14, Р. 1143326, DOI: 10.3389/fpls.2023.1143326.

11. Sapkota R., Stenger J., Ostlie M., Flores P., Towards reducing chemical usage for weed control in agriculture using UAS imagery analysis and computer vision techniques, Scientific reports, 2023, Vol. 13, No. 1, Р. 6548, DOI: 10.1038/s41598-023-33042-0.

12. He K., Zhang X., Ren S., Sun J., Deep residual learning for image recognition, Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778, DOI: 10.1109/CVPR.2016.90

13. Zhang X., Zhou X., Lin M., Sun J., Shufflenet: An extremely efficient convolutional neural network for mobile devices, Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 6848–6856.

14. Mulla D.J., Twenty five years of remote sensing in precision agriculture: Key advances and remaining knowledge gaps, Biosystems engineering, 2013, Vol. 114, No. 4, pp. 358–371, DOI: 10.1016/j.biosystemseng.2012.08.009.

15. Rejeb A., Abdollahi A., Rejeb K., Treiblmaier H., Drones in agriculture: A review and bibliometric analysis, Computers and electronics in agriculture, 2022, Vol. 198, P. 107017, DOI: 10.1016/j.compag.2022.107017.

16. Hu W.J., Fan J., Du Y.X., Li B.S., Xiong N., Bekkering E., MDFC–ResNet: An agricultural IoT system to accurately recognize crop diseases, IEEE Access, 2020, Vol. 8, pp. 115287–115298, DOI: 10.1109/ACCESS.2020.3001237.

17. Yağ İ., Altan A., Artificial intelligence-based robust hybrid algorithm design and implementation for real-time detection of plant diseases in agricultural environments, Biology, 2022, Vol. 11, No. 12, pp. 1732, DOI: 10.3390/biology11121732.

18. Kouadio L., El Jarroudi M., Belabess Z., Laasli S.-E., Roni M.Z.K., Amine I.D.I., Mokhtari N., Mokrini F., Junk J., Lahlali R., A review on UAV-based applications for plant disease detection and monitoring, Remote Sensing, 2023, Vol. 15, No. 17, pp. 4273, DOI: 10.3390/rs15174273.

19. Peteinatos G.G., Reichel P., Karouta J., Andújar D., Gerhards R., Weed identification in maize, sunflower, and potatoes with the aid of convolutional neural networks, Remote Sensing, 2020, Vol. 12, No. 24, pp. 4185, DOI: 10.3390/rs12244185.

20. Li D., Li Y., Zhang Z., Analysis of convolutional neural networks-based approaches in fruit disease detection for smart agriculture applications, Journal of Optics, 2024, Vol. 53, pp. 4256–4265, DOI: 10.1007/s12596-023-01592-1.

21. Weihs B.J., Tang Z., Tian Z., Heuschele D.J., Siddique A., Terrill T.H., Zhang Z., York L.M., Zhang Z., Xu Z., Phenotyping alfalfa (Medicago sativa L.) root structure architecture via integrating confident machine learning with ResNet-18, Plant Phenomics, 2024, Vol. 6, P. 0251, DOI: 10.34133/plantphenomics.0251.

22. de Camargo T., Schirrmann M., Landwehr N., Dammer K. H., Pflanz M., Optimized deep learning model as a basis for fast UAV mapping of weed species in winter wheat crops, Remote Sensing, 2021, Vol. 13, No. 9, pp. 1704, DOI: 10.3390/rs13091704.

23. Riksen V.S., SHpak V.A., Sibirskij vestnik sel’skohozyajstvennoj nauki, 2024, Vol. 54, No. 12 (313), pp. 5–14. (In Russ.)

24. Hovard D., Gugger S., Glubokoe obuchenie s fastai i PyTorch: minimum formul, minimum koda, maksimum effektivnosti (Deep Learning with Fastai and PyTorch: Minimum Formulas, Minimum Code, Maximum Efficiency), SanktPeterburg, 2023, 624 p. (In Russ.)