The Role of Motivation and Perceived Response Quality in GPT Usage for Chemistry Learning at University
DOI:
https://doi.org/10.14421/jtcre.2025.72-04Keywords:
chemistry learning, frequency of use, GPT, motivation, response qualityAbstract
The utilization of artificial intelligence in learning is growing, one of which is through the use of a Generative Pre-trained Transformer (GPT) as a learning tool. This study aims to analyze the relationship between student motivation, response quality perspective, and frequency of GPT use in chemistry learning. This study uses a quantitative approach with multiple regression methods to predict the frequency of GPT use based on two predictor variables, namely motivation and response quality evaluation. Data were collected through a survey of 58 students from chemistry and chemistry education study programs at the undergraduate, master's, and doctoral levels. Correlation analysis results showed a significant positive relationship between motivation and response quality evaluation (r(56) = .59, p < .001) and between response quality evaluation and frequency of GPT use (r(56) = .49, p < .001). However, the relationship between motivation and frequency of GPT use was weaker (r(56) = .32, p = .015). Regression analysis showed that evaluation of response quality significantly predicted the frequency of GPT use (β = .57, p < .001), whereas motivation had a smaller effect (β = .21, p < .01). The R² value of .25 indicated that 25% of the variability in the frequency of GPT use could be explained by both predictor variables. This finding suggests that although motivation has a role in the use of GPT, college students are more likely to use it when they rate the response quality as high. The implications of the results of this study can be the basis for developing strategies to increase the utilization of GPT in chemistry learning as well as academic policies related to the use of AI in education.
Downloads
References
Ali, J. K. M., Shamsan, M. A. A., Hezam, T. A., & Mohammed, A. A. Q. (2023). Impact of ChatGPT on Learning Motivation: Journal of English Studies in Arabia Felix, 2(1), 41–49. https://doi.org/10.56540/jesaf.v2i1.51
Annamalai, N., Rashid, R. A., Munir Hashmi, U., Mohamed, M., Harb Alqaryouti, M., & Eddin Sadeq, A. (2023). Using chatbots for English language learning in higher education. Computers and Education: Artificial Intelligence, 5(June), 100153. https://doi.org/10.1016/j.caeai.2023.100153
Ardyansyah, A. (2024). Enhancing Chemistry Education Through The Integration of Rote Ndao Cultural Practices: An Ethnographic Exploration of Ethnochemistry. Journal of Educational Chemistry (JEC), 6(2), 111–126. https://doi.org/10.21580/jec.2024.6.2.22321
Ardyansyah, A., Yuwono, A. B., Rahayu, S., Alsulami, N. M., & Sulistina, O. (2024). Students’ Perspectives on the Application of a Generative Pre-Trained Transformer (GPT) in Chemistry Learning: A Case Study in Indonesia. Journal of Chemical Education, 101(9), 3666–3675. https://doi.org/10.1021/acs.jchemed.4c00220
Beege, M., Hug, C., & Nerb, J. (2024). AI in STEM education: The relationship between teacher perceptions and ChatGPT use. Computers in Human Behavior Reports, 16, 100494. https://doi.org/https://doi.org/10.1016/j.chbr.2024.100494
Budi, A. D. A. S., Septiana, L., & Mahendra, B. E. P. (2024). Memahami Asumsi Klasik dalam Analisis Statistik: Sebuah Kajian Mendalam tentang Multikolinearitas, Heterokedastisitas, dan Autokorelasi dalam Penelitian. Jurnal Multidisiplin West Science, 3(01), 01–11. https://doi.org/10.58812/jmws.v3i01.878
Cahyani, F. G. (2020). Hubungan Antara Kecerdasan Logis-Matematis Dengan Hasil Belajar Dan Kecerdasan Interpersonal Dengan Motivasi Belajar Kimia. Journal of Tropical Chemistry Research and Education, 2(2), 99–107. https://doi.org/10.14421/jtcre.2020.22-06
Castro Nascimento, C. M., & Pimentel, A. S. (2023). Do Large Language Models Understand Chemistry? A Conversation with ChatGPT. Journal of Chemical Information and Modeling, 63(6), 1649–1655. https://doi.org/10.1021/acs.jcim.3c00285
Clark, T. M., Anderson, E., Dickson-Karn, N. M., Soltanirad, C., & Tafini, N. (2023). Comparing the Performance of College Chemistry Students with ChatGPT for Calculations Involving Acids and Bases. Journal of Chemical Education, 100(10), 3934–3944. https://doi.org/10.1021/acs.jchemed.3c00500
Clark, T. M., & Tafini, N. (2024). Exploring the AI–Human Interface for Personalized Learning in a Chemical Context. Journal of Chemical Education, 101(11), 4916–4923. https://doi.org/10.1021/acs.jchemed.4c00967
Creswell, J. W. (2012). Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research (4th ed.). Pearson.
Dindorf, C., Weisenburger, F., Bartaguiz, E., Dully, J., Klappenberger, L., Lang, V., Zimmermann, L., Fröhlich, M., & Seibert, J.-N. N. (2024). Exploring Decision-Making Competence in Sugar-Substitute Choices: A Cross-Disciplinary Investigation among Chemistry and Sports and Health Students. Education Sciences, 14(5), 531. https://doi.org/10.3390/educsci14050531
Emenike, M. E., & Emenike, B. U. (2023). Was This Title Generated by ChatGPT? Considerations for Artificial Intelligence Text-Generation Software Programs for Chemists and Chemistry Educators. Journal of Chemical Education, 100(4), 1413–1418. https://doi.org/10.1021/acs.jchemed.3c00063
Farazouli, A., Cerratto-Pargman, T., Bolander-Laksov, K., & McGrath, C. (2023). Hello GPT! Goodbye home examination? An exploratory study of AI chatbots impact on university teachers’ assessment practices. Assessment and Evaluation in Higher Education, 49(3), 363–375. https://doi.org/10.1080/02602938.2023.2241676
Fernández, A. A., López-Torres, M., Fernández, J. J., & Vázquez-García, D. (2024). ChatGPT as an Instructor’s Assistant for Generating and Scoring Exams. Journal of Chemical Education, 101(9), 3780–3788. https://doi.org/10.1021/acs.jchemed.4c00231
Følstad, A., & Brandtzaeg, P. B. (2017). Chatbots and the New World of HCI. Interactions, 24(4), 38–42. https://doi.org/10.1145/3085558
Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to Design and Ecaluate Research in Education (8th ed.). Mc-Graw Hill.
Fu, C.-J., Silalahi, A. D. K., Huang, S.-C., Phuong, D. T. T., Eunike, I. J., & Yu, Z.-H. (2024). The (Un)Knowledgeable, the (Un)Skilled? Undertaking Chat-GPT Users’ Benefit-Risk-Coping Paradox in Higher Education Focusing on an Integrated, UTAUT and PMT. International Journal of Human–Computer Interaction, 1–31. https://doi.org/10.1080/10447318.2024.2365028
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023). How Does ChatGPT Perform on the United States Medical Licensing Examination? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Medical Education, 9, 1–9. https://doi.org/10.2196/45312
Haleem, A., Javaid, M., & Singh, R. P. (2022). An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 2(4), 100089. https://doi.org/https://doi.org/10.1016/j.tbench.2023.100089
Hallal, K., Hamdan, R., & Tlais, S. (2023). Exploring the potential of AI-Chatbots in organic chemistry: An assessment of ChatGPT and Bard. Computers and Education: Artificial Intelligence, 5(October), 100170. https://doi.org/10.1016/j.caeai.2023.100170
Hamid, H., Zulkifli, K., Naimat, F., Che Yaacob, N. L., & Ng, K. W. (2023). Exploratory study on student perception on the use of chat AI in process-driven problem-based learning. Currents in Pharmacy Teaching and Learning, 15(12), 1017–1025. https://doi.org/10.1016/j.cptl.2023.10.001
Hsu, W.-L., & Silalahi, A. D. K. (2024). Exploring the paradoxical use of ChatGPT in education: Analyzing benefits, risks, and coping strategies through integrated UTAUT and PMT theories using a hybrid approach of SEM and fsQCA. Computers and Education: Artificial Intelligence, 7, 1–21. https://doi.org/10.1016/j.caeai.2024.100329
Humphry, T., & Fuller, A. L. (2023). Potential ChatGPT Use in Undergraduate Chemistry Laboratories. Journal of Chemical Education, 100(4), 1434–1436. https://doi.org/10.1021/acs.jchemed.3c00006
Joyce, A. (2023). Embracing AI: Using Cat-GPT to Encourage Classroom Discussion. College Teaching, 1–3. https://doi.org/10.1080/87567555.2023.2251074
Karakose, T., & Tülübaş, T. (2023). How Can ChatGPT Facilitate Teaching and Learning: Implications for Contemporary Education. Educational Process: International Journal, 12(4), 7–16. https://doi.org/10.22521/EDUPIJ.2023.124.1
Labadze, L., Grigolia, M., & Machaidze, L. (2023). Role of AI chatbots in education: systematic literature review. International Journal of Educational Technology in Higher Education, 20, 56. https://doi.org/10.1186/s41239-023-00426-1
Leite, B. S. (2024). Generative Artificial Intelligence in chemistry teaching: ChatGPT, Gemini, and Copilot’s content responses. Journal of Applied Learning & Teaching, 7(2). https://doi.org/10.37074/jalt.2024.7.2.13
Liang, Y., Zou, D., Xie, H., & Wang, F. L. (2023). Exploring the potential of using ChatGPT in physics education. Smart Learning Environments, 10, 52. https://doi.org/10.1186/s40561-023-00273-7
Lolinco, A. T., & Holme, T. A. (2023). Developing a Curated Chatbot as an Exploratory Communication Tool for Chemistry Learning. Journal of Chemical Education, 100(10), 4092–4098. https://doi.org/10.1021/acs.jchemed.3c00520
Muniasamy, A., & Alasiry, A. (2020). Deep learning: The impact on future eLearning. International Journal of Emerging Technologies in Learning, 15(1), 188–199. https://doi.org/10.3991/IJET.V15I01.11435
Munoz, S. A. S., Gayoso, G. G., Huambo, A. C., Tapia, R. D. C., Incaluque, J. L., Aguila, O. E. P., Cajamarca, J. C. R., Acevedo, J. E. R., Rivera, H. V. H., & Gonzales, J. L. A. (2023). Examining the Impacts of ChatGPT on Student Motivation and Engagement. Przestrzen Spoleczna, 23(01), 1–28.
Nam, B. H., & Bai, Q. (2023). ChatGPT and its ethical implications for STEM research and higher education: a media discourse analysis. International Journal of STEM Education, 10(1). https://doi.org/10.1186/s40594-023-00452-5
Osborne, J. W., & Waters, E. (2003). Four assumptions of multiple regression that researchers should always test. Practical Assessment, Research and Evaluation, 8(2).
Pratiwi, Y. N., Analita, R. N., Rohmah, R. S., & Rahayu, W. (2024). Hubungan Kemampuan Penalaran Formal Dengan Prestasi Kimia Siswa Kelas XII Dan Kontribusinya Terhadap Tingkat Pemahamannya Di Pendidikan Tinggi. Jurnal Ilmiah Kanderang Tingang, 15(1), 175–185. https://doi.org/10.37304/jikt.v15i1.324
Ramos, B., & Condotta, R. (2024). Enhancing Learning and Collaboration in a Unit Operations Course: Using AI as a Catalyst to Create Engaging Problem-Based Learning Scenarios. Journal of Chemical Education, 101(8), 3246–3254. https://doi.org/10.1021/acs.jchemed.4c00244
Romero-Rodríguez, J. M., Ramírez-Montoya, M. S., Buenestado-Fernández, M., & Lara-Lara, F. (2023). Use of ChatGPT at University as a Tool for Complex Thinking: Students’ Perceived Usefulness. Journal of New Approaches in Educational Research, 12(2), 323–339. https://doi.org/10.7821/naer.2023.7.1458
Saif, N., Khan, S. U., Shaheen, I., Alotaibi, A., Alnfiai, M. M., & Arif, M. (2024). Chat-GPT; validating Technology Acceptance Model (TAM) in education sector via ubiquitous learning mechanism. Computers in Human Behavior, 154, 108097.
Sallam, M., Al-Salahat, K., Eid, H., Egger, J., & Puladi, B. (2024). Human versus Artificial Intelligence: ChatGPT-4 Outperforming Bing, Bard, ChatGPT-3.5 and Humans in Clinical Chemistry Multiple-Choice Questions. Advances in Medical Education and Practice, Volume 15, 857–871. https://doi.org/10.2147/AMEP.S479801
Saranza, C., Villamar, E., Arlan, E., Francia, J., Lopio-Alas, L., & Buca, R. (2024). Exploring the Impact of Usage Frequency on Perceived Value of ChatGPT among University Students: The Moderating Role of Income. International Journal of Social Science and Human Research, 07(12). https://doi.org/10.47191/ijsshr/v7-i12-77
Schiff, D. (2022). Education for AI, not AI for Education: The Role of Education and Ethics in National AI Policy Strategies. International Journal of Artificial Intelligence in Education, 32(3), 527–563. https://doi.org/10.1007/s40593-021-00270-2
Scoggin, J., & Smith, K. C. (2023). Enabling general chemistry students to take part in experimental design activities. Chemistry Education Research and Practice, 24(4), 1229–1242. https://doi.org/10.1039/D3RP00088E
Strzelecki, A. (2023). Students’ Acceptance of ChatGPT in Higher Education: An Extended Unified Theory of Acceptance and Use of Technology. Innovative Higher Education, 0123456789. https://doi.org/10.1007/s10755-023-09686-1
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10, 15. https://doi.org/10.1186/s40561-023-00237-x
UNESCO. (2023). Guidance for generative AI in education and research. In Guidance for generative AI in education and research. https://doi.org/10.54675/ewzm9535
Vasconcelos, M. A. R., & dos Santos, R. P. (2023). Enhancing STEM learning with ChatGPT and Bing Chat as objects to think with: A case study. Eurasia Journal of Mathematics, Science and Technology Education, 19(7), em2296. https://doi.org/10.29333/ejmste/13313
Wandelt, S., Sun, X., & Zhang, A. (2023). AI-driven assistants for education and research? A case study on ChatGPT for air transport management. Journal of Air Transport Management, 113, 102483. https://doi.org/10.1016/j.jairtraman.2023.102483
West, J. K., Franz, J. L., Hein, S. M., Leverentz-Culp, H. R., Mauser, J. F., Ruff, E. F., & Zemke, J. M. (2023). An Analysis of AI-Generated Laboratory Reports across the Chemistry Curriculum and Student Perceptions of ChatGPT. Journal of Chemical Education, 100(11), 4351–4359. https://doi.org/10.1021/acs.jchemed.3c00581
Williams, M. N., Alberto, C., & Grajales, G. (2013). Assumptions of Multiple Regression: Correcting Two Misconceptions. Practical Assessment, Research & Evaluation, 18(11), 1–14.
Wu, E. H. K., Lin, C. H., Ou, Y. Y., Liu, C. Z., Wang, W. K., & Chao, C. Y. (2020). Advantages and constraints of a hybrid model K-12 E-Learning assistant chatbot. IEEE Access, 8, 77788–77801. https://doi.org/10.1109/ACCESS.2020.2988252
Yamada, M., Goda, Y., Matsukawa, H., Hata, K., & Yasunami, S. (2016). A Computer- Supported Learning Design Interaction. IEEE Computer Society, 23(Cmc), 48–59.
Yilmaz, H., Maxutov, S., Baitekov, A., & Balta, N. (2023). Student Attitudes towards Chat GPT: A Technology Acceptance Model Survey. International Educational Review, 1(1), 57–83. https://doi.org/10.58693/ier.114
Young, J. D., Dawood, L., & Lewis, S. E. (2024). Chemistry Students’ Artificial Intelligence Literacy through their Critical Reflections of Chatbot Responses. Journal of Chemical Education, 101, 2466–2474. https://doi.org/10.1021/acs.jchemed.4c00154
Yudanti, N. A., & Premono, S. (2021). Hubungan antara Minat dan Motivasi Terhadap Hasil Belajar pada Pembelajaran Block System Proses Industri Kimia. Journal of Tropical Chemistry Research and Education, 3(1), 10–17. https://doi.org/10.14421/jtcre.2021.31-02
Yu, H. (2023). Reflection on whether Chat GPT should be banned by academia from the perspective of education and teaching. Frontiers in Psychology, 14, 1181712. https://doi.org/10.3389/fpsyg.2023.1181712
Yuriev, E., Wink, D. J., & Holme, T. A. (2023). Virtual Special Issue Call for Papers: Investigating the Uses and Impacts of Generative Artificial Intelligence in Chemistry Education. In Journal of Chemical Education (Vol. 100, Issue 9, pp. 3168–3170). https://doi.org/10.1021/acs.jchemed.3c00829
.jpg){kind=small}
