Akademik Veri Yönetim Sistemi
International Journal of Management and Data Analytics (IJMADA), cilt.5, sa.1, ss.111-123, 2025 (Hakemli Dergi)
This study investigates the optimization of token consumption in large language models (LLMs) through prompt engineering, specifically comparing full-sentence prompts with keyword-based alternatives. Analyzing data from multiple LLM providers across four task types (Question-Answer, Duty, Summary, and Creativity), the research examined token usage patterns and response quality metrics. The study utilized a comprehensive dataset (N=1,231) and employed various evaluation methods, including BERTScore, ROUGE-L, and perplexity analysis. Results demonstrated significant token savings with keyword-based prompts (reduction in cost of 16,7%) while maintaining comparable response quality. Analysis revealed task-specific variations in performance, with duty-related tasks showing no significant quality degradation, while question-answering and summary tasks exhibited minimal quality differences. The findings suggest that keyword-based prompting offers a viable cost optimization strategy for businesses implementing LLM solutions, particularly in duty-related applications. Statistical analysis confirmed significant differences in token consumption (p < .001) with substantial effect sizes, while quality metrics showed only marginal decreases in semantic similarity (ΔBERTScore = -0.005) and surface-level similarity (ΔROUGE-L = -0.019). This research provides practical insights for organizations seeking to optimize their LLM implementation costs while maintaining response quality.