JUCS - Journal of Universal Computer Science 26(9): 1095-1127, doi: 10.3897/jucs.2020.058
Recent Progress in Automated Code Generation from GUI Images Using Machine Learning Techniques
expand article infoDaniel de Souza Baulé‡, Christiane Gresse von Wangenheim§, Aldo von Wangenheim‡, Jean Carlo Rossa Hauck§
‡ Federal University of Santa Catarina, Florianópolis, Brazil§ Federal University of Santa Catarina, Florianópolis/SC, Brazil

Corresponding author: Daniel Baulé ( daniel.baule@grad.ufsc.br )

© Daniel Baulé, Christiane Gresse von Wangenheim, Aldo Wangenheim, Jean Carlo Rossa Hauck.
This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY-ND 4.0). This license allows reusers to copy and distribute the material in any medium or format in unadapted form only, and only so long as attribution is given to the creator. The license allows for commercial use.
Citation: Baulé DS, Wangenheim CG, Wangenheim A, Hauck JCR (2020) Recent Progress in Automated Code Generation from GUI Images Using Machine Learning Techniques. JUCS - Journal of Universal Computer Science 26(9): 1095-1127. https://doi.org/10.3897/jucs.2020.058
Open Access
Abstract
The manual transformation of a user interface design into code is a costly and time-consuming process. A solution can be the automation of the generation of code based on sketches or GUI design images. Recently, Machine Learning approaches have shown promising results in detecting GUI elements for such automation. Thus, to provide an overview of existing approaches, we performed a systematic mapping study. As a result, we identified and compared 20 approaches, that demonstrate good performance results being considered useful. These results can be used by researchers and practitioners in order to improve the efficiency of the GUI design process as well as continue to evolve and improve approaches for its support.
Keywords
user interface design, machine learning, systematic mapping