Enhancing Non-Cognitive Assessments with GPT: Innovations in Item Generation and Translation for the University Belonging Questionnaire

This study explores the application of GPT-3.5-turbo for item generation and translation in non-cognitive educational assessments, specifically focusing on the University Belonging Questionnaire (UBQ). The UBQ, designed to measure university students' sense of belonging across three dimensions, was expanded to include a new dimension on peer relationships, and translated into Chinese using GPT-3.5-turbo. A total of 25 new items, including those for the new dimension were generated and translated into Chinese, out of which 14 items passed the expert review. Psychometric analyses of the expanded and translated UBQ were conducted to evaluate reliability, internal structure, and external validity. The results demonstrate that the UBQ, with its new and translated items, maintains strong reliability and satisfactory internal structure, although the new items introduced some noise. Correlation analyses with the general belongingness scale revealed moderate associations with the acceptance dimension but weak associations with the overall scale. The study highlights GPT’s potential in efficiently expanding and translating non-cognitive assessment tools. This work addresses crucial needs in educational assessments and provides a foundation for future advancements in item development and translation.