GPT-4 as Evaluator: Evaluating Large Language Models on Pest Management in Agriculture
In the rapidly evolving field of artificial intelligence (AI), the application of large language models (LLMs) in agriculture, particularly in pest management, remains nascent. We aimed to prove the feasibility by evaluating the content of the pest management advice generated by LLMs, including the Generative Pre-trained Transformer (GPT) series from OpenAI and the FLAN series from Google. Considering the context-specific properties of agricultural advice, automatically measuring or quantifying the quality of text generated by LLMs becomes a significant challenge. We proposed an innovative approach, using GPT-4 as an evaluator, to score the generated content on Coherence, Logical Consistency, Fluency, Relevance, Comprehensibility, and Exhaustiveness. Additionally, we integrated an expert system based on crop threshold data as a baseline to obtain scores for Factual Accuracy on whether pests found in crop fields should take management action. Each model's score was weighted by percentage to obtain a final score. The results showed that GPT-3.4 and GPT-4 outperform the FLAN models in most evaluation categories. Furthermore, the use of instruction-based prompting containing domain-specific knowledge proved the feasibility of LLMs as an effective tool in agriculture, with an accuracy rate of 72%, demonstrating LLMs' effectiveness in providing pest management suggestions.
Code (0)
등록된 구현이 없습니다.
Tasks
ManagementMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
PestMA: LLM-based Multi-Agent System for Informed Pest Management
Effective pest management is complex due to the need for accurate, context-specific decisions. Recent advancements in large language models (LLMs) open new possibilities for addressing these challenges by providing sophi…
ManagementPestVL-Net: Enabling Multimodal Pest Learning via Fine-grained Vision-Language Interaction
Effective pest recognition and management are crucial for sustainable agricultural development. However, collecting pest data in real scenarios is often challenging. Compared to other domains, pests exhibit a wide variet…
Evaluating Pest Management Strategies: A Robust Method and its Application to Strawberry Disease Management
Farmers use pesticides to reduce yield losses. The efficacies of pesticide treatments are often evaluated by analyzing the average treatment effects and risks. The stochastic efficiency with respect to a function is ofte…
Managementquantile regressionResistance Management for Cancer: Lessons from Farmers
One of the main reasons we have not been able to cure cancers is that drugs select for drug-resistant cancer cells. Pest managers face similar challenges with pesticides selecting for pesticide-resistant organisms. Lesso…
ManagementGraph-Augmented Reasoning with Large Language Models for Tobacco Pest and Disease Management
This paper proposes a graph-augmented reasoning framework for tobacco pest and disease management that integrates structured domain knowledge into large language models. Building on GraphRAG, we construct a domain-specif…
parameter-efficient fine-tuningGraph Neural NetworkAnswer Generation