Item analysis of the Grit Assessment Scale: International version in Portuguese language

Tatiana de Cassia Nakano, Ana Paula Porto Noronha, Ligia de Santis, Leandro Almeida

Abstract


Abstract: Grit can be understood as an element of personality, being a positive and non-cognitive human characteristic, composed of two components: perseverance of effort and consistency of interests. Due to its social, professional, and educational impact, grit has become a construct of great interest. Although some instruments have been developed and adapted for its evaluation in different contexts, the absence of measures in Portuguese led to the development of the Grit Assessment Scale – International Version in Portuguese Language (EAGrit-LP), based on samples from Brazil and Portugal. Even though previous studies have investigated some evidence of validity for the instrument, its items had not yet been individually evaluated. Therefore, the objective of the present study was to perform an item analysis, using Item Response Theory. For this purpose, a sample of 1.050 Brazilians and 656 Portuguese individuals responded to the EAGrit-LP (aged between 17 and 71 years old; M = 24.1 years; SD = 9.2). The results showed that all items had appropriate infit and outfit indices, and that items difficulty was, in general, low. Six items displayed differential item functioning (DIF), based on participants’ gender and country of origin. These results can be used to revise the scale and guide the development of more challenging items, as well as to the development of normative tables that are specific to gender and country of origin.


Keywords


Positive psychology, Psychometrics, Tests construction, Psychological assessment.

Full Text:

PDF

References


Ackerman, T. A., & Ma, Y. (2024). Examining differential item functioning from a multidimensional IRT perspective. Psychometrika, 89, 4-41. https://doi.org/10.1007/s11336-024-09965-6

Aguerri, M. E., Galibert, M. S., Lozzia, G. S., Abal, F. J. P., & Attorresi, H. F. (2007). Medida del funcionamento diferencial del item en el marco de la teoria de respuesta al item. Interdisciplinaria, 24(1), 95-110. https://www.scielo.org.ar/pdf/interd/v24n1/v24n1a04.pdf

Ahangari, S., & Semiyari, S. R. (2022). Examining differential item functioning (DIF) for Iranian EFL test takers with different fields of study. Research in English Pedagogy, 10(1), 169-190. https://doi.org/10.30486/RELP.2021.1935588.1295

Alhadabi, A. (2023). Psychometric properties of Grit-S Scale in university students: An application of the graded response model. Cogent Education, 10(1), 2162699. https://doi.org/10.1080/2331186X.2022.2162699

American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME]. (2014). Standards for educational and psychological testing. American Educational Research Association.

Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561-573. https://doi.org/10.1007/BF02293814

Arco-Tirado, J. L., Fernández-Martín, F. D., & Hoyle, R. H. (2018). Development and validation of a Spanish version of the Grit-S Scale. Frontiers in Psychology, 9(96). https://doi.org/10.3389/fpsyg.2018.00096

Areepattamannil, S., & Khine, M. S. (2017). Evaluating the psychometric properties of the original Grit Scale using Rasch analysis in an Arab adolescent sample. Journal of Psychoeducational Assessment, 36(8), 856-862. https://doi.org/10.1177/0734282917719976

Beri, N., & Sharma, A. (2019). An evaluative study of reliability and validity of Grit 12 item scale in Indian context. Journal of Indian Association for Child & Adolescent Mental Health, 15(3), 48-60. https://doi.org/10.1177/0973134220190304

Bock, R. D., & Gibbons, R. D. (2021). Item response theory. Wiley & Sons.

Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). Routledge/Taylor & Francis Group.

Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456-466. https://doi.org/10.1037/0033-2909.105.3.456

Cormier, D. L., Dunn, J. G. H., & Dunn, J. C. (2019). Examining the domain specificity of grit. Personality and Individual Differences, 139, 349-354. https://doi.org/10.1016/j.paid.2018.11.026

Credé, M., Tynan, M. C., & Harms, P. D. (2017). Much ado about grit: A meta-analytic synthesis of the grit literature. Journal of Personality and Social Psychology, 113(3), 492-511. https://doi.org/10.1037/pspp0000102

Datu, J. A. D. (2020). Beyond passion and perseverance: Review and future research initiatives on the science of grit. Frontiers in Psychology, 11, 545526. https://doi.org/10.3389/fpsyg.2020.545526

Diaz, J. A. S., Köhler, C., & Hartig, J. (2022). Performance of infit and outfit confidence intervals calculated via parametric bootstrapping. Applied Measurement in Education, 35(2), 116-132. https://doi.org/10.1080/08957347.2022.2067540

Dijksma, I., Stuiver, M., Lucas, C., & Lindeboom, R. (2023). Evaluating the psychometric properties of the Grit Scale in Marine recruits using Rasch analysis. BMJ Military Health, 169, 425-429. https://militaryhealth.bmj.com/content/jramc/169/5/425.full.pdf

Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: Perseverance and passion for long-term goals. Journal of Personality and Social Psychology, 92(6), 1087-1101. https://doi.org/10.1037/0022-3514.92.6.1087

Duckworth, A. L., & Quinn, P. D. (2009). Development and validation of the Short Grit Scale (Grit-S). Journal of Personality Assessment, 91(2), 166-174. https://doi.org/10.1080/00223890802634290

Duckworth, A. L., Quinn, P. D., & Tsukayama, E. (2021). Revisiting the factor structure of grit: A commentary on Duckworth and Quinn (2009). Journal of Personality Assessment, 103(5), 573-575. https://doi.org/10.1080/00223891.2021.194022

Ebadi, S., Weisi, H., & Khaksar, Z. (2018). Developing an Iranian ELT context-specific grit instrument. Journal of Psycholinguistic Research, 47(4), 975-997. https://doi.org/10.1007/s10936-018-9571-x

Embretson, S., & Reise, S. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates.

Farmer, A. Y., Wei, Y., Gale, A., & Peterson, N. A. (2025). IRT analysis of the Grit-S Scale: Evaluation with racially/ethnically diverse adolescents in the US. Journal of Psychoeducational Assessment, 43(2). https://doi.org/10.1177/07342829241302860

Fidalgo, A. M., & Scalon, J. D. (2012). Uso dos métodos Mantel-Haenszel para a detecção do funcionamento diferencial dos itens e software relacionado [Using Mantel-Haenzel methods for detecting differential item functioning]. Psicologia Reflexão e Crítica, 25(1), 60-68. https://doi.org/10.1590/S0102-797220120001000008

Fitriani, A., & Situmorang, D. D. B. (2023). The Differential Item Functioning (DIF) testing for the WOCC (Ways of Coping Checklist) instrument based on gender. Heliyon, 9, e14769. https://doi.org/10.1016/j.heliyon.2023.e14769

Fruchart, E., & Rulence-Pâques, P. (2024). Development and validation of the Sports Grit Scale. International Journal of Sport and Exercise Psychology, 1-14. https://doi.org/10.1080/1612197X.2024.2342029

Gao, F., Wei, Q., Dong, X., Gao, J., Lu, S., & Liu, Y. (2024). Developing and validating a domain-specific Grit Scale for college athletic students. Scientific Reports, 14, 11897. https://doi.org/10.1038/s41598-024-62771-z

Gladstone, J. R., Morell, M., Yang, J. S., Ponnock, A., Turci Faust, L., & Wigfield, A. (2022). You can’t compare if you don’t prepare: Differential item functioning in measures of Grit, STEM self-regulation, and motivation. The Journal of Experimental Education, 91(3), 557-575. https://doi.org/10.1080/00220973.2022.2062584

Gonzalez, O., Canning, J. R., Smyth, H., & MacKinnon, D. P. (2020). A psychometric evaluation of the Short Grit Scale: A closer look at its factor structure and scale functioning. European Journal of Psychology Assessment, 36, 646-657. https://doi.org/10.1027/1015-5759/a000535

Guelmami, N., Chalghaf, N., Tannoubi, A., Puce, L., Azaiez, F., & Bragazzi, N. L. (2022). Initial development and psychometric evidence of Physical Education Grit Scale (PE-Grit). Frontiers in Public Health, 10, 818749. https://doi.org/10.3389/fpubh.2022.818749

Gyamfi, A., & Acquaye, R. (2023). Parameters and models of Item Response Theory (IRT): A review of literature. Acta Educationis Generalis, 13(3), 68-78. https://doi.org/10.2478/atd-2023-0022

Hasan, H. E. A., Munawar, K., & Khaiyom, J. H. A. (2022). Psychometric properties of developed and transadapted grit measures across cultures: A systematic review. Current Psychology, 41, 6894-6912. https://doi.org/10.1007/s12144-020-01137-w

Hu, Z., Lin, L., Wang, Y., & Li, J. (2021). The integration of classical testing theory and item response theory. Psychology, 12(9), 1397-1409. https://doi.org/10.4236/psych.2021.129088

Jafaripour, S., Tabatabaei, O., Salehi, H., & Dastjerdi, H. V. (2024). Applying IRT model to determine gender and discipline-based DIF and DDF: A study of the IAU English proficiency test. International Journal of Language Testing, 14(1), 56-74. https://doi.org/10.22034/IJLT.2023.407117.1268

Jiang, L., Zhang, S., Li, X., & Luo, F. (2021). How grit influences high school students’ academic performance and the mediation effect of academic self-efficacy and cognitive learning strategies. Current Psychology. https://doi.org/10.1007/s12144-020-01306-x

Jie, W., Xinyi, W., & Tao, X. (2024). Reliability and validity of simple Chinese version of grit scale for elementary school students. BMC Psychology, 12, 304. https://doi.org/10.1186/s40359-024-01809-3

Jones, E. A., Wind, S. A., Tsai, C. L., & Ge, Y. (2023). Comparing person-fit and traditional indices across careless response patterns in surveys. Applied Psychological Measurement, 47(5-6), 365-385. https://doi.org/10.1177/01466216231194358

Kuruveettissery, S., Gupta, S., & Rajan, S. K. (2023). Development and psychometric validation of the Three-Dimensional Grit Scale. Current Psychology, 42, 5280-5289. https://doi.org/10/1007/s12144-021-019-862-w

Li, J., Fang, M., Wang, W., Sun, G., & Cheng, Z. (2018). The influence of grit on life satisfaction: Self-esteem as a mediator. Psychologica Belgica, 58(1), 51-66. https://doi.org/10.5334/pb.400

Li, C., & Yang, Y. (2023). Domain-general grit and domain-specific grit: Conceptual structures, measurement, and associations with the achievement of German as a foreign language. International Review of Applied Linguistics in Language Teaching, 62(4), 1513-1537. https://doi.org/10.1515/iral-2022-0196

Lin, Y.-H., & Shih, C.-L. (2021). Analysis on mathematics Grit Scale and its gender-based DIF. Asian Journal of Research in Education and Social Sciences, 3(2), 73-79. https://myjms.mohe.gov.my/index.php/ajress/article/view/14080

Linacre, J. M., & Wright, B. D. (2002). Construction of measures from many-facet data. Journal of Applied Measurement, 3(4), 486-512. https://pubmed.ncbi.nlm.nih.gov/12486312

Marentes-Castillo, M., Zamarripa, J., & Castillo, I. (2019). Validation of the Grit Scale and the Treatment Self-Regulation Questionnaire (TSRQ) in the Mexican context. Revista Latinoamericana de Psicología, 51(1), 918. https://doi.org/10.14349/rlp.2019.v51.n1.2

Matore, M. E. E. M., Mohamad, N., Khairani, A. Z., & Segar, T. N. (2023). Psychometric assessment of the short Grit Scale (Grit-S) using the Rasch Model in a Malaysian polytechnic students’ sample. Environment-Behavior Proceedings Journal, 8(SI15), 127-132. https://doi.org/10.21834/e-bpj.v8iSI15.5088

Midkiff, B., Langer, M., Demetriou, C., & Panter, A. (2017). Measuring grit among first-generation college students: A psychometric analysis. In L. A. Van der Ark, M. Wiberg, S. A. Culpepper, J. A. Douglas, & W. C. Wang (Eds.), Quantitative psychology: IMPS 2016 (Vol. 196, pp. 407-420). Springer. https://doi.org/10.1007/978-3-319-56294-0_35

Monteiro, S., Postigo, Á., & Almeida, L. (2023). Escala Grit de Oviedo: Precisão e validade dos resultados da versão portuguesa [Oviedo Grit Scale accuracy and validity of results of the Portuguese version]. Revista Iberoamericana de Diagnóstico y Evaluación Psicologica, 67(1), 147-156. https://doi.org/10.21865/RIDEP67.1.11

Morell, M., Yang, J. S., Gladstone, J. R., Turci Faust, L., Ponnock, A. R., Lim, H. J., & Wigfield, A. (2021). Grit: The long and short of it. Journal of Educational Psychology, 113(5), 1038-1058. https://doi.org/10.1037/edu0000594

Muenks, K., Wigfield., A., Yang, J. S., & O’Neal, C. R. (2017). How true is Grit? Assessing its relations to high school and college students’ personality characteristics, self-regulation, engagement, and achievement. Journal of Educational Psychology, 109(5), 599-620. https://doi.org/10.1037/edu0000153

Nakano, T. C., Primi, R., & Nunes, C. H. S. S. (2015). Análise de itens e teoria de resposta ao item [Item analysis and item response theory]. In C. S. Hutz, D. R. Bandeira, & C. M. Trentini (Orgs.), Psicometria (pp. 97-124). Artmed.

Neroni, J., Meijs, C., Kirschner, P. A., Kate, M. X., & Groot, R. H. M. (2022). Academic self-efficacy, self-esteem, and grit in higher online education: Consistency of interests predicts academic success. Social Psychology of Education, 25, 951-975. https://doi.org/10.1007/s11218-022-09696-5

Noronha, A. P. P., & Almeida, L. S. (2022). A construção e estudos psicométricos da Escala de Avaliação da Garra: Versão internacional em Língua Portuguesa (EAGrit-LP) Psicologia, Educação e Cultura, 26(1), 8-23. http://hdl.handle.net/10400.26/40365

Noronha, A. P. P., Reppold, C. T., & Almeida, L. S. (2024). Internal structure and invariance analysis of EAGrit-LP-R for Grit Assessment. International Journal of Testing, 24(2), 130-144. https://doi.org/10.1080/15305058.2024.2312105

Pajestka, G., & Poraj-Weder, M. (2023). Persistence as a multidimensional construct: A new insight into the multidimensional persistence scale. International Journal of Selection and Assessment, 32(2), 195-209. https://doi.org/10.1111/ijsa.12457

Park, H., Lee, K., & Shin, N. (2020). Development and validation of the Clinical Nurses Grit Scale (CN-Grit). Journal of Korean Academy of Nursing Administration, 26(1), 55-64. https://doi.org/10.11111/jkana.2020.26.1.55

Peixoto, E. M., Honda, G. C., Gagnon, J., Nakano, T. C., Rocha, G. M. A., Zanini, D. S., & Balbinotti, M. A. A. (2019). Questionário de Fusão Cognitiva (CFQ): Novas evidências de validade e invariância transcultural. Psico, 50(1), 1-11. https://doi.org/10.15448/1980-8623.2019.1.27851

Pendyala, A. G., & Vyas, M. (2023). Moving away from the WEIRD grit: A critical review. Indian Journal of Positive Psychology, 14(1), 67-71.

Postigo, A., Cuesta, M., García-Cueto, E., Menéndez-Aller, A., Gonzáles-Nuevo, C., & Muniz, J. (2021). Grit assessment: Is one dimension enough?. Journal of Personality Assessment, 103(6), 786-796. https://doi.org/10.1080/00223891.2020.1848853

Reysen, R., Reysen, M., Perry, P., & Knight, R. D. (2019). Not so soft skills: The importance of Grit to college student success. Journal of College Orientation, Transition, and Retention, 26(2). https://doi.org/10.24926/jcotr.v26i2.2397

Sass, D. A. (2011). Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment, 29(4), 347-363. https://doi.org/10.1177/0734282911406661

Schimschal, S. E., Visentin, D., Kornhaber, R., & Cleary, M. (2022). Development of a scale to measure the psychological resources of grit in adults. Nursing and Health Sciences, 24(3), 752-763. https://doi.org/10.1111/nhs.12973

Schmidt, F. T. C., Sudzina, F., & Botek, M. (2021). Psychometric assessment of the Short Grit Scale among Czech young adults. Journal of Psychoeducational Assessment, 39(4), 508-513. https://doi.org/10.1177/0734282920974817

Sigmundosson, H., Haga, M., & Hermundsdottir, F. (2020). Passion, grit, and mindset in young adults: Exploring the relationship and gender differences. New Ideas in Psychology, 59, e100795. https://doi.org/10.1016/j.newideapsaych.2020.100795

Smith, E. V. (2004). Detecting and evaluating the impact of multidimensionality using the fit statistics and principal component analysis of residuals. In E. V. Smith & R. M. Smith (Eds.), Introduction to Rasch measurement: Theory, models, and applications (pp. 575-600). JAM Press.

Sordia, N. (2020). Psychometric properties of the Georgian version of the Grit Scale. Prizren Social Science Journal, 4(1), 8-13. https://doi.org/10.32936

Sudina, E., Vernon, T., Foster, H., Del Villano, H., Hernandez, S., Beck, D., & Plonsky, L. (2020). Development and initial validation of the L2-Teacher Grit Scale. TESOL Quarterly, 55(1), 156-184. https://doi.org/10.1002/tesq.581

Tan, C., Low, S. R., Chong, H. Y., Chong, S. L., Ong, A. W., Siah, P. C., Phang, S. F., Ong, Z. Q., Tan, W. H., Wong, J. A., & Lew, W. H. (2019). Exploratory and confirmatory factor analyses of the Short Grit Scale (Grit-S) for Malaysian undergraduate students. Makara Human Behavior Studies in Asia, 23(1), 27-33. https://doi.org/10.7454/hubs.asia.2120519

Teimouri, Y., Sudina, E., & Plonsky, L. (2021). On domain-specific conceptualization and measurement of Grit in L2 learning. Journal for the Psychology of Language Learning, 3(2), 156-165. https://doi.org/10.52598/jpll/3/2/10

Tynan, M. C. (2021). Deconstructing Grit’s validity: The case for revising Grit measures and theory. In L. E. van Zyl, C. Olckers, & L. van der Vaart (Eds.), Multidisciplinary Perspectives on Grit. Springer, Cham. https://doi.org/10.1007/978-3-030-57389-8_8

Tyumeneva, Y., Kardanova, E., & Kuzmina, J. (2019). Grit: Two related but independent constructs instead of one. Evidence from item response theory. European Journal of Psychological Assessment, 35(4), 469-478. https://doi.org/10.1027/1015-5759/a000424

Tyumeneva, Y., Kuzmina, J., & Kardanova, E. (2014). IRT analysis and validation of the Grit Scale: A Russian investigation. National Research University. https://www.researchgate.net/publication/314496371_IRT_Analysis_and_Validation_of_the_Grit_Scale_A_Russian_Investigation

van Zyl, L. E., Heijenk, B., Klibert, J., Shanbkland, R., Verger, N. B., Rothmann, S., Cho, V., Feng, K., See-To, E. W. K., Roll, L. C., & van der Meij, L. (2022). Grit across nations: The cross-national equivalence of the Grit-O Scale. Journal of Happiness Studies, 23, 3179-3213. https://doi.org/10.1007/s10902-022-00543-0

Wright, B. D., & Masters, G. N. (1982). Rating scale analysis: Rasch measurement. Mesa Press.

Yiğiter, M. S., & Boduroğlu, E. (2024). Item response theory: A comprehensive review of studies with document analysis. International Journal of Educational Studies and Policy, 5(2), 119-138.

Yupanqui-Lorenzo, D. E., Becerra-Herrera, W. A., Díaz-Leon, I., Sernaqué, M. A. C., & Olivera-Carhuaz, E. S. (2024). Short Grit Scale (Grit-S): New evidence based on CFA and MIRT models. Revista Fuentes, 26(1), 109-119. https://doi.org/10.12795/revistafuentes.2024.23182

Zhang, T., Park, D., Tsukayama, E., Duckworth, A., & Luo, L. (2023). Sparking virtuous cycles: A longitudinal study of subjective well-being and grit during early adolescence. Journal of Youth and Adolescence. https://doi.org/10.1007/s10964-023-01862-y




DOI: https://doi.org/10.14417/ap.2139

Refbacks

  • There are currently no refbacks.


Nº ERC: 107494 | ISSN (in print): 0870-8231 | ISSN (online): 1646-6020 | Copyright © ISPA - CRL, 2012 | Rua Jardim do Tabaco, 34, 1149-041 Lisboa | NIF: 501313672 | The portal and metadata are licensed under the license Creative Commons CC BY-NC