Comparative Evaluation of ChatGPT, Gemini, Claude, DeepSeek, and Grok for Descriptive Statistical Analysis of Likert-Scale UAT Data

Yoseph Satria Praka

doi:10.21512/ijcshai.v3i2.16335

Authors

Yoseph Satria Praka Politeknik Negeri Jakarta (PNJ)

DOI:

https://doi.org/10.21512/ijcshai.v3i2.16335

Keywords:

Large Language Models, User Acceptance Testing, Descriptive Statistics, Comparative Analysis

Abstract

Large Language Models (LLMs) may be employed as an aid for data analysis such as calculating descriptive statistics from questionnaire responses. Nevertheless, how well the calculations of UAT data are accurate is in need of additional evaluation. The purpose of this study was to assess ChatGPT, Gemini, Claude, DeepSeek, and Grok when analyzing five-point Likert-scale UAT data; the reference software was IBM SPSS Statistics 26. The study utilized questionnaire responses from ten participants to five items. All models were provided with the same data set and prompt to calculate respondent-level scores and descriptive measures including minimum, maximum, mean, standard error, standard deviation, and variance. Then, the calculated results were compared to the corresponding SPSS results. Overall, the majority of the results obtained by the models agreed with the SPSS results; however, there were some discrepancies in the variance results for Q1, Q3, and Q4. Additionally, DeepSeek had a slightly different result for the standard deviation of the total scores. There was no additional calculation or controlled test conducted to identify why the models differed. The study generally demonstrated that the results of the five models were consistent with the SPSS generated results for the data set analyzed. However, this does not suggest that the models will yield similar results when applied to other data sets, or when utilizing alternative statistical methods. Researchers conducting future studies may want to apply larger data sets, conduct multiple replications of experiments, and apply different forms of statistical analysis to evaluate how effectively LLMs operate under varying experimental conditions. Should a study require high degrees of numeric precision, researchers may wish to compare the results produced by an LLM to those produced by a widely accepted statistical software package.

Dimensions

Author Biography

Yoseph Satria Praka, Politeknik Negeri Jakarta (PNJ)

Program Studi Teknik Multimedia Digital

References

[1] R. Watkins, “Guidance for researchers and peer-reviewers on the ethical use of Large Language Models (LLMs) in scientific research workflows,” AI Ethics, vol. 4, no. 4, pp. 969–974, Nov. 2024, doi: 10.1007/s43681-023-00294-5.

[2] N. L. Rane, A. Tawde, S. P. Choudhary, and J. Rane, “Contribution and performance of ChatGPT and other Large Language Models (LLM) for scientific and research advancements: a double-edged sword,” Int. Res. J. Mod. Eng. Technol. Sci., vol. 05, no. 10, pp. 875–899, 2023, doi: 10.56726/irjmets45213.

[3] D. Prandner, D. Wetzelhütter, and S. Hese, “ChatGPT as a data analyst: an exploratory study on AI-supported quantitative data analysis in empirical research,” Front. Educ., vol. 9, Jan. 2025, doi: 10.3389/feduc.2024.1417900.

[4] E. Supriyadi and S. Pahmi, “Exploring the Potential of AI Tools in Education: A Thematic Analysis of Gemini.ai,” Gema Wiralodra, vol. 15, no. 3, Dec. 2024, doi: 10.31943/gw.v15i3.771.

[5] G. Ling, S. Zhong, and R. Huang, “Agent Skills: A Data-Driven Analysis of Claude Skills for Extending Large Language Model Functionality,” arXiv, Feb. 2026.

[6] Z. Deng et al., “Exploring DeepSeek: A Survey on Advances, Applications, Challenges and Future Directions,” IEEE/CAA J. Autom. Sin., vol. 12, no. 5, pp. 872–893, May 2025, doi: 10.1109/JAS.2025.125498.

[7] E.R. Hurtado-Terranova, “IMPACT OF GROK AI ON THE DEVELOPMENT OF SCIENTIFIC WRITING SKILLS IN HIGHER EDUCATION STUDENTS,” Genet. Mol. Res., vol. 25, no. 7S, pp. 1–5, 2026, doi: https://doi.org/10.4238/ddfv0549.

[8] L. Xie, C. Zheng, H. Xia, H. Qu, and C. Zhu-Tian, “WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualization,” in Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, New York, NY, USA: ACM, Oct. 2024, pp. 1–14. doi: 10.1145/3654777.3676374.

[9] M. Shukla, D. Pandey, S. Kaur, M. Agarwal, A. Goyal, and H. Sharma, “Evaluating the Accuracy and Explanatory Quality of Large Language Models ChatGPT, Claude, DeepSeek, Gemini, Grok, and Le Chat in Statistical Test Selection for Hypothesis Testing Decisions,” Cureus, Oct. 2025, doi: 10.7759/cureus.94949.

[10] M. E. D. C. Souza and L. Weigang, “Grok, Gemini, ChatGPT and DeepSeek: Comparison and Applications in Conversational Artificial Intelligence,” Intell. Artif., pp. 1–7, 2025, doi: 10.5281/zenodo.14885243.

[11] S. Altmäe, A. Sola-Leyva, and A. Salumets, “Artificial intelligence in scientific writing: a friend or a foe?,” Reprod. Biomed. Online, vol. 47, no. 1, pp. 3–9, Jul. 2023, doi: 10.1016/j.rbmo.2023.04.009.

[12] R. T. Y. Tong, Y. K. Yuan, N. W. Dong, and R. K. Ramasamy, “A Review: Methods of Acceptance Testing,” 2022, pp. 76–86. doi: 10.2991/978-94-6463-080-0_7.

[13] W. Wulandari, N. Nofiyani, and H. Hasugian, “USER ACCEPTANCE TESTING (UAT) PADA ELECTRONIC DATA PREPROCESSING GUNA MENGETAHUI KUALITAS SISTEM,” J. Mhs. Ilmu Komput., vol. 4, no. 1, pp. 20–27, Mar. 2023, doi: 10.24127/ilmukomputer.v4i1.3383.

[14] D. Meva and H. Kukadiya, “Performance Evaluation of Large Language Models: A Comprehensive Review,” Int. Res. J. Comput. Sci., vol. 12, no. 03, p. 109, Mar. 2025, doi: 10.26562/irjcs.2025.v1203.03.

[15] S. Karpurapu et al., “Comprehensive Evaluation and Insights Into the Use of Large Language Models in the Automation of Behavior-Driven Development Acceptance Test Formulation,” IEEE Access, vol. 12, pp. 58715–58721, 2024, doi: 10.1109/ACCESS.2024.3391815.

[16] N.-L. Hsueh, H.-J. Lin, and L.-C. Lai, “Applying Large Language Model to User Experience Testing,” Electronics, vol. 13, no. 23, p. 4633, Nov. 2024, doi: 10.3390/electronics13234633.

[17] M. Calvano, A. Curci, R. Lanzilotti, A. Piccinno, and A. Ragone, “Leveraging Large Language Models for Usability Testing: a Preliminary Study,” in Companion Proceedings of the 30th International Conference on Intelligent User Interfaces, New York, NY, USA: ACM, Mar. 2025, pp. 78–81. doi: 10.1145/3708557.3716341.

[18] M. Agarwal, P. Sharma, and P. Wani, “Evaluating the Accuracy and Reliability of Large Language Models (ChatGPT, Claude, DeepSeek, Gemini, Grok, and Le Chat) in Answering Item-Analyzed Multiple-Choice Questions on Blood Physiology,” Cureus, Apr. 2025, doi: 10.7759/cureus.81871.

[19] A. Alsajri, H. A. Salman, and A. Steiti, “Generative Models in Natural Language Processing: A Comparative Study of ChatGPT and Gemini,” Babylonian J. Artif. Intell., vol. 2024, pp. 134–145, Nov. 2024, doi: 10.58496/BJAI/2024/015.

[20] K. Shahzad and S. Iqbal, “Comparative Analysis of ChatGPT, DeepSeek, and Gemini for Automated Code Generation,” in 2025 18th International Conference on Engineering of Modern Electric Systems (EMES), IEEE, May 2025, pp. 1–4. doi: 10.1109/EMES65692.2025.11045587.

[21] D. M. Colaco and D. Antao, “Exploring self-learning efficacy: a comparative analysis of ChatGPT and Gemini,” Interact. Technol. Smart Educ., vol. 22, no. 4, pp. 688–698, Oct. 2025, doi: 10.1108/ITSE-12-2023-0251.

[22] A. P. Kusuma and A. Yufron, “ANALYSIS OF USER ACCEPTANCE TESTING ON A SHIPPING APPLICATION TO DETERMINE THE QUALITY OF THE SYSTEM,” Antivirus J. Ilm. Tek. Inform., vol. 18, no. 2, pp. 234–243, Nov. 2024, doi: 10.35457/antivirus.v18i2.4002.

[23] N. Dufey-Portilla et al., “Assessing the validity of ChatGPT-4o and Google Gemini Advanced when responding to frequently asked questions in endodontics,” J. Appl. Oral Sci., vol. 33, 2025, doi: 10.1590/1678-7757-2025-0321.

[24] D. Zhang et al., “DataSciBench: An LLM Agent Benchmark for Data Science,” in Findings of the Association for Computational Linguistics: ACL 2026, Stroudsburg, PA, USA: Association for Computational Linguistics, 2026, pp. 3685–3728. doi: 10.18653/v1/2026.findings-acl.181.

[25] V. Najahah and U. Pujianto, “Automatic Classification of Artificial Intelligence Generated Question Difficulty Levels,” Indones. J. Innov. Stud., vol. 27, no. 1, Jan. 2026, doi: 10.21070/ijins.v27i1.1880.