International Journal of

ADVANCED AND APPLIED SCIENCES

EISSN: 2313-3724, Print ISSN: 2313-626X

Frequency: 12

line decor
  
line decor

 Volume 12, Issue 4 (April 2025), Pages: 152-163

----------------------------------------------

 Original Research Paper

Psychometric evaluation of the UTAUT scale using the graded response model

 Author(s): 

 Faridah Hanim Yahya 1, *, Aszunarni Ayob 2, Mohd Ridhuan Mohd Jamil 1, Abdussakir Abdussakir 3, Nurul Ain Mohd Daud 1

 Affiliation(s):

  1Faculty of Human Development, Universiti Pendidikan Sultan Idris, 35900 Tanjong Malim, Perak, Malaysia
  2Matriculation Division, Ministry of Education Malaysia, Complex E, 62604 Putrajaya, Malaysia
  3Faculty of Tarbiyah and Teacher Training, Universitas Islam Negeri Maulana Malik Ibrahim Malang, Malang, Indonesia

 Full text

    Full Text - PDF

 * Corresponding Author. 

   Corresponding author's ORCID profile:  https://orcid.org/0000-0002-0972-473X

 Digital Object Identifier (DOI)

  https://doi.org/10.21833/ijaas.2025.04.017

 Abstract

This study examined the reliability and validity of the Unified Theory of Acceptance and Use of Technology (UTAUT) measurement instrument. The sample included 202 mathematics teachers randomly selected from national secondary schools in Malaysia. The dataset was analyzed using the MIRT (Multidimensional Item Response Theory) and LTM (Latent Trait Models) packages in R software. The psychometric properties of the UTAUT scale were assessed using the graded response model (GRM), a type of Item Response Theory (IRT) model. The findings indicate that the scale effectively differentiates between various levels of technological acceptance, with most items showing high discrimination values. The threshold parameters suggest that higher response categories correspond to greater levels of agreement. The scale provides the highest accuracy in the middle range of traits but is less precise at the lower and upper extremes. However, the UTAUT scale still demonstrates good model fit and reliability.

 © 2025 The Authors. Published by IASE.

 This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/).

 Keywords

 Technology acceptance, Measurement validity, Psychometric analysis, Item response theory, Mathematics teachers

 Article history

 Received 6 October 2024, Received in revised form 12 February 2025, Accepted 24 April 2025

 Acknowledgment

This research project was co-funded by the Universiti Pendidikan Sultan Idris, Malaysia under University Research Grant Special Interest Group (SIG) 2022 (Code: 2022-0028-106-01).

  Compliance with ethical standards

  Ethical considerations

This study upheld informed consent, confidentiality, and ethical compliance while protecting participants' rights and data security.

  Conflict of interest: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

 Citation:

 Yahya FH, Ayob A, Mohd Jamil MR, Abdussakir A, and Mohd Daud NA (2025). Psychometric evaluation of the UTAUT scale using the graded response model. International Journal of Advanced and Applied Sciences, 12(4): 152-163

  Permanent Link to this page

 Figures

  Fig. 1  Fig. 2  Fig. 3  Fig. 4  Fig. 5  Fig. 6

 Tables

  Table 1  Table 2  Table 3  Table 4  Table 5  Table 6  Table 7  Table 8 

----------------------------------------------   

 References (48)

  1. Abbad MMM (2021). Using the UTAUT model to understand students’ usage of e-learning systems in developing countries. Education and Information Technologies, 26(6): 7205-7224.  https://doi.org/10.1007/s10639-021-10573-5    [Google Scholar] PMid:34025204 PMCid:PMC8122219
  2. Ahmed RR, Štreimikienė D, and Štreimikis J (2021). The extended UTAUT model and learning management system during COVID-19: Evidence from PLS-SEM and conditional process modeling. Journal of Business Economics and Management, 23(1): 82-104.  https://doi.org/10.3846/jbem.2021.15664    [Google Scholar]
  3. Akaike H (2011). Akaike’s information criterion. In: Lovric M (Ed.), International encyclopedia of statistical science: 25-25. Springer Berlin Heidelberg, Berlin, Germany.  https://doi.org/10.1007/978-3-642-04898-2_110    [Google Scholar]
  4. Andrews JE, Ward H, and Yoon J (2021). UTAUT as a model for understanding intention to adopt AI and related technologies among librarians. The Journal of Academic Librarianship, 47(6): 102437.  https://doi.org/10.1016/j.acalib.2021.102437    [Google Scholar]
  5. Bollen K and Lennox R (1991). Conventional wisdom on measurement: A structural equation perspective. Psychological Bulletin, 110(2): 305-314.  https://doi.org/10.1037//0033-2909.110.2.305    [Google Scholar]
  6. Byrd IV TF, Kim JS, Yeh C, Lee J, and O'Leary KJ (2021). Technology acceptance and critical mass: Development of a consolidated model to explain the actual use of mobile health care communication tools. Journal of Biomedical Informatics, 117: 103749.  https://doi.org/10.1016/j.jbi.2021.103749    [Google Scholar] PMid:33766780
  7. Chen L and Aklikokou AK (2020). Determinants of e-government adoption: Testing the mediating effects of perceived usefulness and perceived ease of use. International Journal of Public Administration, 43(10): 850-865.  https://doi.org/10.1080/01900692.2019.1660989    [Google Scholar]
  8. Cook KF, Kallen MA, and Amtmann D (2009). Having a fit: Impact of number of items and distribution of data on traditional criteria for assessing IRT’s unidimensionality assumption. Quality of Life Research, 18(4): 447-460.  https://doi.org/10.1007/s11136-009-9464-4    [Google Scholar] PMid:19294529 PMCid:PMC2746381
  9. Cortina JM (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1): 98-104.  https://doi.org/10.1037//0021-9010.78.1.98    [Google Scholar]
  10. Dunn TJ, Baguley T, and Brunsden V (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3): 399-412.  https://doi.org/10.1111/bjop.12046    [Google Scholar] PMid:24844115
  11. Dwivedi YK, Rana NP, Chen H, and Williams MD (2011). A meta-analysis of the unified theory of acceptance and use of technology (UTAUT). In: Nüttgens M, Gadatsch A, Kautz K, Schirmer I, Blinn N (Eds), Governance and sustainability in information systems: Managing the transfer and diffusion of IT: 155-170. Springer, Berlin, Germany.  https://doi.org/10.1007/978-3-642-24148-2_10    [Google Scholar]
  12. Embretson SE and Reise SP (2013). Item response theory. Psychology Press, Hove, UK.  https://doi.org/10.4324/9781410605269    [Google Scholar]
  13. Green SB and Yang Y (2009). Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 74(1): 155-167.  https://doi.org/10.1007/s11336-008-9099-3    [Google Scholar]
  14. Gu F, Little TD, and Kingston NM (2013). Misestimation of reliability using coefficient alpha and structural equation modeling when assumptions of tau-equivalence and uncorrelated errors are violated. Methodology, 9(1): 30-40.  https://doi.org/10.1027/1614-2241/a000052    [Google Scholar]
  15. Hair JF, Black WC, Babin BJ, and Anderson RE (2019). Multivariate data analysis. 8 th Edition, Cengage Learning, Andover, UK.    [Google Scholar]
  16. Hair JF, Matthews LM, Matthews RL, and Sarstedt M (2017). PLS-SEM or CB-SEM: Updated guidelines on which method to use. International Journal of Multivariate Data Analysis, 1(2): 107-123.  https://doi.org/10.1504/IJMDA.2017.087624    [Google Scholar]
  17. Hambleton RK (2021). Handbook of item response theory. Chapman and Hall, London, UK.    [Google Scholar]
  18. Hambleton RK and Jones RW (1993). Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12(3): 38-47.  https://doi.org/10.1111/j.1745-3992.1993.tb00543.x    [Google Scholar]
  19. Hambleton RK, Jones RW, and Rogers HJ (1993). Influence of item parameter estimation errors in test development. Journal of Educational Measurement, 30(2): 143-155.  https://doi.org/10.1111/j.1745-3984.1993.tb01071.x    [Google Scholar]
  20. Izkair AS and Lakulu MM (2021). Experience moderator effect on the variables that influence intention to use mobile learning. Bulletin of Electrical Engineering and Informatics, 10(5): 2875-2883. https://doi.org/10.11591/eei.v10i5.3109    [Google Scholar]
  21. Izkair AS and Lakulu MM (2023). The moderating effects of gender on factors affecting the intention to use mobile learning. Journal of Information Technology Education: Research, 22: 199-233.  https://doi.org/10.28945/5094    [Google Scholar]
  22. Kim S, Lee KH, Hwang H, and Yoo S (2016). Analysis of the factors influencing healthcare professionals’ adoption of mobile electronic medical record (EMR) using the unified theory of acceptance and use of technology (UTAUT) in a tertiary hospital. BMC Medical Informatics and Decision Making, 16(1): 12.  https://doi.org/10.1186/s12911-016-0249-8    [Google Scholar] PMid:26831123 PMCid:PMC4736616
  23. Li W (2021). The role of trust and risk in citizens’ e-government services adoption: A perspective of the extended UTAUT model. Sustainability, 13(14): 7671.  https://doi.org/10.3390/su13147671    [Google Scholar]
  24. Marangunić N and Granić A (2015). Technology acceptance model: A literature review from 1986 to 2013. Universal Access in the Information Society, 14(1): 81-95.  https://doi.org/10.1007/s10209-014-0348-1    [Google Scholar]
  25. McNeish D (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3): 412-433.  https://doi.org/10.1037/met0000144    [Google Scholar] PMid:28557467
  26. Neath AA and Cavanaugh JE (2012). The Bayesian information criterion: Background, derivation, and applications. WIREs Computational Statistics, 4(2): 199-203.  https://doi.org/10.1002/wics.199    [Google Scholar]
  27. Ployhart RE and Vandenberg RJ (2009). Longitudinal research: The theory, design, and analysis of change. Journal of Management, 36(1): 94-120.  https://doi.org/10.1177/0149206309352110    [Google Scholar]
  28. Polit DF (2014). Getting serious about test–retest reliability: A critique of retest research and some recommendations. Quality of Life Research, 23(6): 1713-1720.  https://doi.org/10.1007/s11136-014-0632-9    [Google Scholar] PMid:24504622
  29. Raykov T (1997). Scale reliability, Cronbach's coefficient alpha, and violations of essential tau-equivalence with fixed congeneric components. Multivariate Behavioral Research, 32(4): 329-353.  https://doi.org/10.1207/s15327906mbr3204_2    [Google Scholar] PMid:26777071
  30. Reise SP, Bonifay WE, and Haviland MG (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95(2): 129-140.  https://doi.org/10.1080/00223891.2012.725437    [Google Scholar] PMid:23030794
  31. Rouidi M, Elouadi AE, Hamdoune A, Choujtani K, and Chati A (2022). TAM-UTAUT and the acceptance of remote healthcare technologies by healthcare professionals: A systematic review. Informatics in Medicine Unlocked, 32: 101008.  https://doi.org/10.1016/j.imu.2022.101008    [Google Scholar]
  32. Salloum SA and Shaalan K (2019). Factors affecting students’ acceptance of e-learning system in higher education using UTAUT and structural equation modeling approaches. In: Hassanien A, Tolba M, Shaalan K, & Azar A (Eds), Proceedings of the international conference on advanced intelligent systems and informatics 2018: 469–480. Springer, Cham, Switzerland.  https://doi.org/10.1007/978-3-319-99010-1_43    [Google Scholar]
  33. Samejima F (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 34(4, Pt 2): 100-100.  https://doi.org/10.1007/BF03372160    [Google Scholar]
  34. Schuberth F, Zaza S, and Henseler J (2023). Partial least squares is an estimator for structural equation models: A comment on Evermann and Rönkkö (2021). Communications of the Association for Information Systems, 52(1): 711-714.  https://doi.org/10.17705/1CAIS.05232    [Google Scholar]
  35. Sezer B and Yilmaz R (2019). Learning management system acceptance scale (LMSAS): A validity and reliability study. Australasian Journal of Educational Technology, 35(3): 15-30.  https://doi.org/10.14742/ajet.3959    [Google Scholar]
  36. Sijtsma K (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1): 107-120. https://doi.org/10.1007/s11336-008-9101-0    [Google Scholar] PMid:20037639 PMCid:PMC2792363
  37. Tamilmani K, Rana NP, and Dwivedi YK (2021). Consumer acceptance and use of information technology: A meta-analytic evaluation of UTAUT2. Information Systems Frontiers, 23(4): 987-1005.  https://doi.org/10.1007/s10796-020-10007-6    [Google Scholar]
  38. Teresi JA and Fleishman JA (2007). Differential item functioning and health assessment. Quality of Life Research, 16(1): 33-42.  https://doi.org/10.1007/s11136-007-9184-6    [Google Scholar] PMid:17443420
  39. Toland MD (2013). Practical guide to conducting an item response theory analysis. The Journal of Early Adolescence, 34(1): 120-151.  https://doi.org/10.1177/0272431613511332    [Google Scholar]
  40. Trizano-Hermosilla I and Alvarado JM (2016). Best alternatives to Cronbach's alpha reliability in realistic conditions: Congeneric and asymmetrical measurements. Frontiers in Psychology, 7: 769.  https://doi.org/10.3389/fpsyg.2016.00769    [Google Scholar] PMid:27303333 PMCid:PMC4880791
  41. Ustun AB, Karaoglan-Yilmaz FG, and Yilmaz R (2023). Educational UTAUT-based virtual reality acceptance scale: A validity and reliability study. Virtual Reality, 27(2): 1063-1076.  https://doi.org/10.1007/s10055-022-00717-4    [Google Scholar]
  42. Venkatesh V, Morris MG, Davis GB, and Davis FD (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3): 425-478.  https://doi.org/10.2307/30036540    [Google Scholar]
  43. Venkatesh V, Thong JYL, and Xu X (2012). Consumer acceptance and use of information technology: Extending the unified theory of acceptance and use of technology. MIS Quarterly, 36(1): 157-178.  https://doi.org/10.2307/41410412    [Google Scholar]
  44. Wang H, Tao D, Yu N, and Qu X (2020). Understanding consumer acceptance of healthcare wearable devices: An integrated model of UTAUT and TTF. International Journal of Medical Informatics, 139: 104156.  https://doi.org/10.1016/j.ijmedinf.2020.104156    [Google Scholar] PMid:32387819
  45. Wong KT, Teo T, and Russo S (2013). Interactive whiteboard acceptance: Applicability of the UTAUT model to student teachers. The Asia-Pacific Education Researcher, 22: 1-10.  https://doi.org/10.1007/s40299-012-0001-9    [Google Scholar]
  46. Xue L, Rashid AM, and Ouyang S (2024). The unified theory of acceptance and use of technology (UTAUT) in higher education: A systematic review. Sage Open, 14(1).  https://doi.org/10.1177/21582440241229570    [Google Scholar]
  47. Yeop MA, Yaakob MFMW, Wong KT, Don Y, and Zain FM (2019). Implementation of ICT policy (blended learning approach): Investigating factors of behavioural intention and use behaviour. International Journal of Instruction, 12(1): 767-782.  https://doi.org/10.29333/iji.2019.12149a    [Google Scholar]
  48. Yu S and Chen T (2024). Understanding older adults’ acceptance of Chatbots in healthcare delivery: An extended UTAUT model. Frontiers in Public Health, 12: 1435329.  https://doi.org/10.3389/fpubh.2024.1435329    [Google Scholar] PMid:39628811 PMCid:PMC11611720