Software Readability Metrics: A Systematic Literature Review

Abstract

Software quality is an aspect directly linked to future maintenance costs, and is generally quantified by means of quality metrics of the software products. One of the main aspects to evaluate software quality is its maintainability, since it has a high impact on the total costs of software projects. In particular, it is estimated that 70% of the maintenance time is dedicated to understand the code, so it is important to be able to measure the readability of a source code fragment properly. Readability is defined as the ease with which a person can read and understand a piece of code written by another person, this feature is crucial to facilitate the understanding of the code during software maintenance tasks. In this sense, it is necessary to be able to measure the readability of the source code. Therefore, over the past few years, researchers in the area have proposed multiple metrics to measure the readability of source code. However, it remains a challenge to be able to accurately assess the readability of source code in a widely accepted way. Therefore, it is essential to give continuity to this type of research and, for this, it is necessary to know the most recent advances. This paper presents a synthesis and analysis of code readability metrics, through a systematic review of literature, showing a compilation of the characteristics and methods used for their measurement. Results of this work will be useful for researchers in the area to propose new source code readability metrics.

Keywords

software readability, source code, software quality, maintainability, metrics, software engineering

Author Biography

Ingrid Sofía Echavarría-Flórez, M.Sc.

Roles: Formal analisys, Investigation, Data curation, Software, Writing – original draft.

Felipe Restrepo-Calle, Ph. D.

Roles: Conceptualization, Methodology, Supervision, Validation, Writing – review & editing.

References

[1] R. P. L. Buse, and W. R. Weimer, “Learning a Metric for Code Readability,” IEEE Transactions on Software Engineering, vol. 36 (4), pp. 546-558, Jul. 2010. https://doi.org/10.1109/TSE.2009.70

[2] S. Scalabrino, G. Bavota, C. Vendome, M. Linares-Vasquez, D. Poshyvanyk, and R. Oliveto, “Automatically assessing code understandability: How far are we?,” in 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE), 2017, pp. 417-427.

[3] D. Posnett, A. Hindle, and P. Devanbu, “A simpler model of software readability,” in Proceeding of the 8th working conference on Mining software repositories, 2011, pp. 73-82. https://doi.org/10.1145/1985441.1985454

[4] M. Akour, and B. Falah, “Application domain and programming language readability yardsticks,” in 7th International Conference on Computer Science and Information Technology (CSIT), 2016, pp. 1-6. https://doi.org/10.1109/CSIT.2016.7549476

[5] U. A. Mannan, I. Ahmed, and A. Sarma, “Towards understanding code readability and its impact on design quality,” in Proceedings of the 4th ACM SIGSOFT International Workshop on NLP for Software Engineering, 2018, pp. 18-21. https://doi.org/10.1145/3283812.3283820

[6] B. Kitchenham, O. Pearl Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman, “Systematic literature reviews in software engineering – A systematic literature review,” Information and Software Technology, vol. 51 (1), pp. 7-15, 2009. https://doi.org/10.1016/j.infsof.2008.09.009

[7] D. Alawad, M. Panta, M. Zibran, and R. Islam, “An Empirical Study of the Relationships between Code Readability and Software Complexity,” in 27th Int Conf on Software Engineering and Data Engineering, 2018, pp. 1-7.

[8] M. Allamanis, E. T. Barr, C. Bird, and C. Sutton, “Suggesting accurate method and class names,” in Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, 2015, pp. 38-49. https://doi.org/10.1145/2786805.2786849

[9] H. Aman, S. Amasaki, T. Sasaki, and M. Kawahara, “Lines of Comments as a Noteworthy Metric for Analyzing Fault-Proneness in Methods,” IEICE Transactions on Information and Systems, vol. E98.D (12), pp. 2218-2228, 2015. https://doi.org/10.1587/transinf.2015EDP7107

[10] J. Borstler, and B. Paech, “The Role of Method Chains and Comments in Software Readability and Comprehension-An Experiment,” IEEE Transactions on Software Engineering, vol. 42 (9), pp. 886-898, Sep. 2016. https://doi.org/10.1109/TSE.2016.2527791

[11] H. M. Chen, W. H. Chen, and C. C. Lee, “An automated assessment system for analysis of coding convention violations in Java programming assignments,” Journal of Information Science and Engineering, vol. 34 (5), pp. 1203-1221, 2018. https://doi.org/10.6688/JISE.201809_34(5).0006

[12] S. Choi, S. Kim, J.-H. Lee, J. Kim, and J.-Y. Choi, “Measuring the Extent of Source Code Readability Using Regression Analysis,” in International Conference on Computational Science and Its Applications, 2018, pp. 410-421. https://doi.org/10.1007/978-3-319-95171-3_32

[13] R. Coleman, “Aesthetics Versus Readability of Source Code,” International Journal of Advanced Computer Science and Applications, vol. 9 (9), pp. 12-18, 2018. https://doi.org/10.14569/IJACSA.2018.090902

[14] A. De Renzis, M. Garriga, A. Flores, A. Cechich, C. Mateos, and A. Zunino, “A domain independent readability metric for web service descriptions,” Computer Standards & Interfaces, vol. 50, pp. 124-141, Feb. 2017. https://doi.org/10.1016/j.csi.2016.09.005

[15] R. M. dos Santos, and M. A. Gerosa, “Impacts of coding practices on readability,” in Proceedings of the 26th Conference on Program Comprehension, 2018, pp. 277-285. https://doi.org/10.1145/3196321.3196342

[16] S. Fakhoury, Y. Ma, V. Arnaoudova, and O. Adesope, “The effect of poor source code lexicon and readability on developers’ cognitive load,” in IEEE/ACM 26th International Conference on Program Comprehension, 2018, pp. 286-296.

[17] L. Frunzio, B. Lin, M. Lanza, and G. Bavota, “RETICULA: Real-time code quality assessment,” in IEEE 25th International Conference on Software Analysis, Evolution and Reengineering, 2018, pp. 542-546. https://doi.org/10.1109/SANER.2018.8330256

[18] Y. Liu, X. Sun, and Y. Duan, “Analyzing program readability based on WordNet,” in Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering, 2015, pp. 1-2. https://doi.org/10.1145/2745802.2745837

[19] Q. Mi, J. Keung, X. Mei, Y. Xiao, and W. Chan, “A Gamification Technique for Motivating Students to Learn Code Readability in Software Engineering,” in International Symposium on Educational Technology, 2018, pp. 250-254. https://doi.org/10.1109/ISET.2018.00062

[20] Q. Mi, J. Keung, Y. Xiao, S. Mensah, and Y. Gao, “Improving code readability classification using convolutional neural networks,” Information and Software Technology, vol. 104, pp. 60-71, Dec. 2018. https://doi.org/10.1016/j.infsof.2018.07.006

[21] Q. Mi, J. Keung, Y. Xiao, S. Mensah, and X. Mei, “An Inception Architecture-Based Model for Improving Code Readability Classification,” in Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering, 2018, pp. 139-144. https://doi.org/10.1145/3210459.3210473

[22] Q. Mi, J. Keung, and Y. Yu, “Measuring the Stylistic Inconsistency in Software Projects using Hierarchical Agglomerative Clustering,” in The 12th International Conference on Predictive Models and Data Analytics in Software Engineering, 2016, pp. 1-10. https://doi.org/10.1145/2972958.2972963

[23] D. J. Ferreira Novais, M. J. V. Pereira, and P. R. Henriques, “Program analysis for Clustering Programmers’ Profile,” in Federated Conf on Computer Science and Information Systems, Prague, 2017, pp. 701-705. https://doi.org/10.15439/2017F147

[24] A. Pahal, and R. S. Chillar, “Code Readability: A Review of Metrics for Software Quality,” International Journal of Computer Trends and Technology, vol. 46 (1), pp. 1-4, 2017. https://doi.org/10.14445/22312803/IJCTT-V46P101

[25] S. Scalabrino, M. Linares-Vásquez, R. Oliveto, and D. Poshyvanyk, “A comprehensive model for code readability,” Journal of Software: Evolution and Process, vol. 30 (6), e1958, Jun. 2018. https://doi.org/10.1002/smr.1958

[26] T. Sedano, “Code Readability Testing, an Empirical Study,” in IEEE 29th International Conference on Software Engineering Education and Training, 2016, pp. 111-117. https://doi.org/10.1109/CSEET.2016.36

[27] A. Wulff-Jensen, K. Ruder, E. Triantafyllou, and L. E. Bruni, “Gaze Strategies Can Reveal the Impact of Source Code Features on the Cognitive Load of Novice Programmers,” in International Conference on Applied Human Factors and Ergonomics, 2019. https://doi.org/10.1007/978-3-319-94866-9_9

[28] W. Xu, D. Xu, and L. Deng, “Measurement of Source Code Readability Using Word Concreteness and Memory Retention of Variable Names,” in IEEE 41st Annual Computer Software and Applications Conf, 2017, pp. 33-38. https://doi.org/10.1109/COMPSAC.2017.166

[29] S. Butler, M. Wermelinger, Yijun Yu, and H. Sharp, “Exploring the Influence of Identifier Names on Code Quality: An Empirical Study,” in Proceedings of the 14th European Conference on Software Maintenance and Reengineering, 2010, pp. 156-165. https://doi.org/10.1109/CSMR.2010.27

[30] J. Dorn, A General Software Readability Model, 2012. https://www.cs.virginia.edu/~weimer/students/dorn-mcs-paper.pdf

[31] I. B. Sampaio, and L. Barbosa, “Software readability practices and the importance of their teaching,” in 7th International Conference on Information and Communication Systems, 2016, pp. 304-309. https://doi.org/10.1109/IACS.2016.7476069