UC Davis

Purpose: Generative artificial intelligence (GenAI), especially Large Language Model (LLM)-based chatbots such as ChatGPT, has reshaped students’ learning and engagement in higher education. Yet, technical details of GenAI are largely inapproachable to most students. This article develops a learning module for GenAI and seeks to examine whether this module can potentially affect students’ perceptions toward GenAI. Methods: We implemented a one-lecture (60-min) module on GenAI models, with primary focus on structures of LLM-based chatbots, during the last week of a Biomedical Engineering (BME) Machine Learning course. A mixed-methods survey on perceptions of GenAI was distributed to the students before and after the module. Paired t-tests and regression analyses were used to analyze the Likert-scale quantitative questions and thematic coding was performed for the free-response questions. Results: Students (N = 13) reported significantly stronger approval on favorability to use GenAI in medicine (p = 0.015), understanding of LLM-based chatbots (p < 0.001), confidence on using LLM-based chatbots (p = 0.027), optimism on future development of LLMs (p = 0.020), and perception of instructor’s attitude toward GenAI (p = 0.033). Students maintained a neutral view on accuracy of LLM-generated answers and a negative view on the ability of generating bias-free answers in LLMs. The primary contributors identified in students’ intentions to use LLMs are self-efficacy in using the LLM outputs and lower precepted bias of LLMs. The impression of GenAI for students shifted from primarily LLM-based chatbots and generative work to components and training process of GenAI. After the module, students reported a clear understanding of tokenizers and word embeddings while expressing confusion on transformers. Conclusion: A module on the details of GenAI models shifted the students’ attitudes to GenAI models positively while still being acutely aware of its limitations. We believe that inclusion of such modules in a modern engineering curriculum will help students achieve AI literacy.