An Empirical Study of Adoption of ChatGPT for Bug Fixing among Professional Developers

Haotong Ge; Yuemeng Wu

doi:10.61187/ita.v1i1.19

Authors

Haotong Ge Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada
Yuemeng Wu Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada

Keywords:

software engineering, adoption, social factors, ChatGPT

Abstract

ChatGPT is a powerful tool that assists software engineers in identifying and rectifying errors in code. One of its primary advantages is its ability to engage in natural language conversation with humans, which allows it to collaborate more closely with engineers in improving and optimizing the code. However, despite its potential advantages, software developers do not always utilize ChatGPT as a tool for bug fixing. In this study, we aim to examine the factors that influence the adoption of ChatGPT for bug fixing among professional software developers, based on the Unified Theory of Acceptance and Use of Technology (UTAUT) theory. To accomplish this, we conducted 50 semi-structured interviews with professional software developers and other stakeholders. Our findings indicate that the performance expectancy and effort expectancy of professional software developers, as well as social influence, facilitating conditions, data security, and trust are the key factors of adoption. These findings suggest that understanding these factors can be critical in promoting the adoption and use of ChatGPT in the software development industry.

Downloads

Download data is not yet available.

References

Xiangyu Li, Shaowei Zhu, Marcelo d’Amorim, and Alessandro Orso. 2018. Enlightened debugging. In Proceedings of the 40th International Conference on Software Engineering. 82–92.

Min Xie and Bo Yang. 2003. A study of the effect of imperfect debugging on software development cost. IEEE Transactions on Software Engineering 29, 5 (2003), 471–473. DOI: https://doi.org/10.1109/TSE.2003.1199075

Strategic Planning. 2002. The economic impacts of inadequate infrastructure for software testing. National Institute of Standards and Technology 1 (2002).

Abid Haleem, Mohd Javaid, and Ravi Pratap Singh. 2023. An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. Bench Council Transactions on Benchmarks, Standards and Evaluations (2023), 100089. DOI: https://doi.org/10.1016/j.tbench.2023.100089

Nigar M Shafiq Surameery and Mohammed Y Shakor. 2023. Use ChatGPT to solve programming bugs. International Journal of Information Technology & Computer Engineering (IJITC) ISSN: 2455-5290 3, 01 (2023), 17–22. DOI: https://doi.org/10.55529/ijitc.31.17.22

Dominik Sobania, Martin Briesch, Carol Hanna, and Justyna Petke. 2023. An Analysis of the Automatic Bug Fixing Performance of ChatGPT. arXiv preprint arXiv:2301.08653 (2023). DOI: https://doi.org/10.1109/APR59189.2023.00012

Viswanath Venkatesh, Michael G Morris, Gordon B Davis, and Fred D Davis. 2003. User acceptance of information technology: Toward a unified view. MIS quarterly (2003), 425–478. DOI: https://doi.org/10.2307/30036540

Tore Dybå and Torgeir Dingsøyr. 2008. Empirical studies of agile software development: A systematic review. Information and software technology 50, 9-10 (2008), 833–859. DOI: https://doi.org/10.1016/j.infsof.2008.01.006

Pavneet Singh Kochhar, Tegawendé F Bissyandé, David Lo, and Lingxiao Jiang. 2013. An empirical study of adoption of software testing in open source projects. In 2013 13th International Conference on Quality Software. IEEE, 103–112. DOI: https://doi.org/10.1109/QSIC.2013.57

Hart O Awa, John P Uko, and Ojiabo Ukoha. 2017. An empirical study of some critical adoption factors of ERP software. International Journal of Human– Computer Interaction 33, 8 (2017), 609–622. DOI: https://doi.org/10.1080/10447318.2016.1265828

Everett M Rogers, Arvind Singhal, and Margaret M Quinlan. 2014. Diffusion of innovations. In An integrated approach to communication theory and research. Routledge, 432–448.

Shundan Xiao, Jim Witschey, and Emerson Murphy-Hill. 2014. Social influences on secure development tool adoption: why security tools spread. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. 1095–1106. DOI: https://doi.org/10.1145/2531602.2531722

Hemank Lamba, Asher Trockman, Daniel Armanios, Christian Kästner, Heather Miller, and Bogdan Vasilescu. 2020. Heard it through the Gitvine: an empirical study of tool diffusion across the npm ecosystem. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 505–517. DOI: https://doi.org/10.1145/3368089.3409705

Linda G Wallace and Steven D Sheetz. 2014. The adoption of software measures: A technology acceptance model (TAM) perspective. Information & Management 51, 2 (2014), 249–259. DOI: https://doi.org/10.1016/j.im.2013.12.003

Martin Fishbein. 1979. A theory of reasoned action: some applications and implications. (1979).

Richard M Ryan and Edward L Deci. 2000. Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary educational psychology 25, 1 (2000), 54–67. DOI: https://doi.org/10.1006/ceps.1999.1020

Icek Ajzen. 1991. The theory of planned behavior. Organizational behavior and human decision processes 50, 2 (1991), 179–211. DOI: https://doi.org/10.1016/0749-5978(91)90020-T

Shirley Taylor and Peter A Todd. 1995. Understanding information technology usage: A test of competing models. Information systems research 6, 2 (1995), 144–176. DOI: https://doi.org/10.1287/isre.6.2.144

Ronald L Thompson, Christopher A Higgins, and Jane M Howell. 1991. Personal computing: Toward a conceptual model of utilization. MIS quarterly (1991), 125–143. DOI: https://doi.org/10.2307/249443

A. Bandura. 1986. Social foundations of thought and action: A social cognitive theory. Prentice Hall, Englewood Cliffs, NJ.

C.B. Seaman. 1999. Qualitative methods in empirical studies of software engineering. IEEE Transactions on Software Engineering 25, 4 (1999), 557–572. https://doi.org/10.1109/32.799955 DOI: https://doi.org/10.1109/32.799955

Jonathan Smith, Paul Flowers, and Michael Larkin. 2009. Interpretative Phenomenological Analysis: Theory, Method and Research. Qualitative Research in Psychology 6 (01 2009). DOI: https://doi.org/10.1080/14780880903340091

Bambang Leo Handoko. 2019. Application of UTAUT theory in higher education online learning. In Proceedings of the 2019 10th International Conference on E- business, Management and Economics. 259–264. DOI: https://doi.org/10.1145/3345035.3345047