JUCS - Journal of Universal Computer Science 32(6): 851-875, doi: 10.3897/jucs.169476
Are Large Language Models Smart Coders? Evaluating Correctness and Efficiency of LLM-generated Algorithms
expand article infoJefferson Martins, Rodrigo Andrade, Luis F. Alves Pereira
‡ Universidade Federal do Agreste de Pernambuco, Garanhuns, Brazil
Open Access
Abstract
Skilled software developers play a crucial role in creating energy-efficient code, contributing to the reduction of carbon emissions and promoting sustainable computing practices. In parallel, Large Language Models (LLMs) have recently emerged as powerful programming assistants, providing support for developers in complex coding tasks. In this context, this study investigates the ability of LLMs to generate computationally efficient implementations of classic algorithms - such as Heap Sort and Binary Search - across different programming languages, with a primary focus on optimizing energy consumption, memory usage, and execution time. Our results show that, in most cases, different LLMs are able to generate these algorithm implementations correctly. Furthermore, in the majority of the evaluated scenarios, LLM-generated code demonstrates superior performance over human-written algorithm code with regard to energy consumption, memory efficiency, and execution time.
Keywords
LLM, Correctness, Energy-efficiency, Memory-efficiency, Time-efficiency
login to comment