RESEARCH ARTICLE
Accuracy and Error Patterns of Generative AI on the Korean National Dental Hygienist Licensing Examination Across Prompt Engineering Strategies
Sang-Kyeom Kim
*
Department of Dental Hygiene, Namseoul University, Cheonan, 31020, Korea
*Corresponding Author: Sang-Kyeom Kim, E-mail: kyeoms19@nsu.ac.kr
Volume 14, Number 3, Pages 209–219, September 2026.
Korean J Clin Dent Hyg 2026;14(3):209–219. https://doi.org/10.12972/kjcdh.2026.14.3.4
Received on June 19, 2026, Revised on July 15, 2026, Accepted on August 04, 2026, Published on September 30, 2026.
Copyright © 2026 by Korean Society of Infection Control and Prevention in Dental Hygiene.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License(http://creativecommons.org/licenses/by-nc/4.0/).
Abstract
Objective: This study aimed to compare GPT-4o accuracy across five prompt engineering strategies using items from the Korean National Dental Hygienist Licensing Examination and to examine differences by question domain and type. Methods: A total of 399 items from the 2024-2025 examinations were classified by domain (Basic/Clinical) and type (Recall/Reasoning) and presented to GPT-4o (GPT-4o-2024-11-20, temperature=0) under five prompt conditions. Differences in accuracy across prompt conditions were assessed using Cochran’s Q test. For reasoning-type items, exploratory pairwise comparisons were performed using exact McNemar tests with Holm correction. Results: Overall accuracy ranged from 75.2% to 78.9%, exceeding the overall threshold of 60% under all prompt conditions. Cochran’s Q showed no significant overall difference among the five prompt conditions (Q=6.462, p=0.167). For reasoning-type items (n=80), the combined prompt yielded the highest accuracy(75.0%) compared with 65.0%– 67.5% for the other conditions. However, the overall difference among prompt conditions was not statistically significant (Q=6.759, p=0.149), and no pairwise comparison remained significant after Holm correction. Qualitative analysis identified recurring hallucinations involving Korea- specific regulations. Conclusions: GPT-4o achieved accuracy exceeding the 60% overall threshold regardless of prompt strategy, suggesting that the overall effect of prompt engineering was limited. Although the combined prompt showed a favorable trend for reasoning-type items, statistical evidence for this effect was limited. These findings highlight the importance of AI literacy for the critical evaluation of AI-generated rationales.
Keywords
Artificial intelligence, Dental hygiene education, Dental hygienists, Large language models, Prompt engineering