Performance of Large Language Models in a Computer Science Degree Program

07/24/2023
by   Tim Krüger, et al.
0

Large language models such as ChatGPT-3.5 and GPT-4.0 are ubiquitous and dominate the current discourse. Their transformative capabilities have led to a paradigm shift in how we interact with and utilize (text-based) information. Each day, new possibilities to leverage the capabilities of these models emerge. This paper presents findings on the performance of different large language models in a university of applied sciences' undergraduate computer science degree program. Our primary objective is to assess the effectiveness of these models within the curriculum by employing them as educational aids. By prompting the models with lecture material, exercise tasks, and past exams, we aim to evaluate their proficiency across different computer science domains. We showcase the strong performance of current large language models while highlighting limitations and constraints within the context of such a degree program. We found that ChatGPT-3.5 averaged 79.9 tested modules, BingAI achieved 68.4 variant, 20 degree program - due to limitations in mathematical calculations.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset