Korean Journal of Clinical Dental Hygiene (Korean J Clin Dent Hyg)

Open Access, Peer-reviewed
Indexed in Korea Ciation Index

pISSN 2287-9641
eISSN 2465-7743

RESEARCH ARTICLE

Accuracy and Error Patterns of Generative AI on the Korean National Dental Hygienist Licensing Examination Across Prompt Engineering Strategies

Department of Dental Hygiene, Namseoul University, Cheonan, 31020, Korea

*Corresponding Author: Sang-Kyeom Kim, E-mail: kyeoms19@nsu.ac.kr

Volume 14, Number 3, Pages 209–219, September 2026.
Korean J Clin Dent Hyg 2026;14(3):209–219. https://doi.org/10.12972/kjcdh.2026.14.3.4
Received on June 19, 2026, Revised on July 15, 2026, Accepted on August 04, 2026, Published on September 30, 2026.
Copyright © 2026 by Korean Society of Infection Control and Prevention in Dental Hygiene.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License(http://creativecommons.org/licenses/by-nc/4.0/).

Abstract

Objective: This study aimed to compare GPT-4o accuracy across five prompt engineering strategies using items from the Korean National Dental Hygienist Licensing Examination and to examine differences by question domain and type. Methods: A total of 399 items from the 2024-2025 examinations were classified by domain (Basic/Clinical) and type (Recall/Reasoning) and presented to GPT-4o (GPT-4o-2024-11-20, temperature=0) under five prompt conditions. Differences in accuracy across prompt conditions were assessed using Cochran’s Q test. For reasoning-type items, exploratory pairwise comparisons were performed using exact McNemar tests with Holm correction. Results: Overall accuracy ranged from 75.2% to 78.9%, exceeding the overall threshold of 60% under all prompt conditions. Cochran’s Q showed no significant overall difference among the five prompt conditions (Q=6.462, p=0.167). For reasoning-type items (n=80), the combined prompt yielded the highest accuracy(75.0%) compared with 65.0%– 67.5% for the other conditions. However, the overall difference among prompt conditions was not statistically significant (Q=6.759, p=0.149), and no pairwise comparison remained significant after Holm correction. Qualitative analysis identified recurring hallucinations involving Korea- specific regulations. Conclusions: GPT-4o achieved accuracy exceeding the 60% overall threshold regardless of prompt strategy, suggesting that the overall effect of prompt engineering was limited. Although the combined prompt showed a favorable trend for reasoning-type items, statistical evidence for this effect was limited. These findings highlight the importance of AI literacy for the critical evaluation of AI-generated rationales.
Keywords

Artificial intelligence, Dental hygiene education, Dental hygienists, Large language models, Prompt engineering

Section