The Global Engagement Measurement Scale (GEMS): A New Scale for Assessing the Impact of Education Abroad and Campus Internationalization
DOI:
https://doi.org/10.36366/frontiers.v26i1.368Keywords:
Global Engagement Measurement, Assessment, Impact of study abroad, Study abroad, Education abroad, Campus internationalizationAbstract
We describe the development and validation of a new instrument to measure the non-academic impact of education abroad, the Global Engagement Measurement Scale (GEMS). Based on a review of the extant literature, we developed a 37-item instrument containing scales for four specific constructs of interest: a) cultural engagement; b) ambiguity tolerance; c) knowledge of the host site; and d) diversity openness. The measure was piloted with 263 education abroad students before and after a 5- week education abroad program. We examined the scales’ underlying factor structure, reliability, and sensitivity to education abroad program effects. Results indicate strong reliability and statistically significant increases across all scales following education abroad experiences. The GEMS instrument is brief and easy to administer, publically available, and includes important constructs for assessing the impact of education abroad. Additionally, the GEMS could be used to assess campus internationalization efforts with domestic and international students.
Downloads
Download data is not yet available.
References
Ashwill, M.A., & Hoang Oanh, D.T. (2009). Developing Globally Competent Citizens: The Contrasting Cases of the United States and Vietnam. In D.K. Deardorff (Ed.), The SAGE Handbook of Intercultural Competence (pp. 141-157). Thousand Oaks, CA: Sage Publications.
Bakalis & Joiner. (2004). Participation in tertiary study abroad programs: the role of personality. International Journal of Educational Management, 18 (5), 286-291.
Black, T.H. & Duhon, D.L. (2006). Assessing the impact of business study abroad programs on cultural awareness and personal development. Journal of Education for Business, 81 (3), 140-144. doi: 10.3200/JOEB.81.3.140-144
Braskamp, L.A., Braskamp, D.C., & Engberg, M.E. (2013). Global Perspective Inventory (GPI) Manual. Chicago, IL: Global Perspectives Institute Inc.
Browne & Cudeck. (1993). Alternate ways of assessing model fit. In: K.A. Bollen & J.S. Long (Eds.), Testing structural equation models, (pp.136-162). Beverly Hills, CA: Sage.
Byram, M. (1997). Teaching and assessing intercultural communicative competence. Philadelphia, PA: Multilingual Matters.
Chieffo, L and Griffiths L (2004) Large-scale assessment of student attitudes after a short-term study abroad program. Frontiers: The Interdisciplinary Journal of Study Abroad, X, 165-178.
Clarke, I., Flaherty, T.B., Wright, N.D., & McMillen, R.M. (2009). Student international proficiency from study abroad programs. Journal of Marketing Education, 31 (2), 173-181. doi: 10.1177/0273475309335583
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.
Deardorff, D.K. (2004). The identification and assessment of as a student outcome of internationalization at institutions of higher education in the United States. Unpublished doctoral dissertation. North Carolina State University, Raleigh, NC.
Deardorff, D.K. (2006) Identification and Assessment of Intercultural Competence as a Student
Outcome of Internationalization. Journal of Studies in International Education, 10(3), 241–266.
Deardorff, D.K. (2009). Implementing Intercultural Competence Assessment. In D.K. Deardorff (Ed.), The SAGE Handbook of Intercultural Competence (pp. 477-491). Thousand Oaks, CA: Sage Publications.
Dewaele, J. & Wei, L. (2013). Is multilingualism linked to a higher tolerance for ambiguity? Bilingualism: Language and Cognition, 16, 231-240. Doi:10.1017/S1366728912000570
Dunlap, W.P., Cortina, J.M., Vaslow, J.B., & Burke, M.J. (1996). Meta-analysis of experiments With matched groups or repeated measures designs. Psychological Methods, 1 (2), 170-177.
Fantini, A.E. (2000). A central concern : Developing intercultural competence. SIT Occasional Paper Series, 1. Retrieved April 1, 2014, from: http://www.sit.edu/publications/docs.pdf.
Ganster, D.C., Hennessey, H.W., & Luthans, F. (1983). Social desirability response effects: Three alternative models. Academy of Management Journal, 26 (2), 321-331.
Hammer, M.R. (2010). The Intercultural Development Inventory manual. Berlin, MD: IDI.
Hett, E. J. (1993). The development of an instrument to measure globalmindedness. Unpublished Doctoral Dissertation, University of San Diego.
Hu, L. & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6 (1), 1-55. Doi: 10.1080/10705519909540118.
Institute of International Education. (2013).Open Doors Report on International Educational Exchange. Retrieved from http://www.iie.org/opendoors.
Kazdin, A.E. (1995). Preparing and evaluating research reports. Psychological Assessment, 7 (3), 228-237.
Kenny, D.A. & McCoach, D.B. (2003). Effect of the number of variables on measures of fit indices in structural equation modeling. Structural Equation Modeling, 10, 333-351.
Li, Y. (2013). Cultivating student global competence: A pilot experimental study. Decision Sciences Journal of Innovative Education, 11, 125-143.
McLain, D.L. (1993). The MSTAT-I: A new measure of an individual’s tolerance for ambiguity. Educational and Psychological Measurement, 53 (1), 183-189.
McLain, D.L. (2009). Evidence of the properties of an ambiguity tolerance measure: The Multiple stimulus types ambiguity tolerance scale-II. Psychological Reports, 105(3), 975-988.
Muthen, L.K. & Muthen, B.O. (1998-2008). Mplus User’s Guide. Fifth Edition. Los Angeles, CA: Muthen & Muthen.
Pascarella, E.T., Edison, M., Nora, A., Hagedorn, L.S., & Terenzini, P.T. (1996). Influences on students’ openness to diversity and challenges in the first year of college. Journal of Higher Education, 67 (2), 179-x.
Raykov, T. & Marcoulides, G.A. (2008). An introduction to applied multivariate analysis. New York, NY: Routledge.
Shealy, C.N. (2005). Justifying the justification hypothesis: scientific-humanism, Equilintigration (EI) theory, and the Beliefs, Events, and Values Inventory (BEVI). Journal of Clinical Psychology, 61 (1), 81-106.
Tabachnick, B.G. & Fidell, L.S. (2007). Using multivariate statistics (5th ed.). Boston, MA: Allyn & Bacon.
Wastson, J.R., Siska, P., & Wolfel, R.L. (2013). Assessing gains in language proficiency, cross-cultural competence, and regional awareness during study abroad: A preliminary study. Foreign Language Annals, 46 (1), 62-79. Doi: 10.1111/flan.12016
Williams, T.R. (2005) The reflexive model of Intercultural Competency: a multidimensional, qualitative approach to study abroad assessment. Frontiers Journal, 356-371.
Zhai, L. (2000). The influence of study abroad programs on college student development in the College of Food, Agriculture, and Environmental Science at the Ohio State University. Unpublished doctoral dissertation. Ohio State University, Columbus.
Bakalis & Joiner. (2004). Participation in tertiary study abroad programs: the role of personality. International Journal of Educational Management, 18 (5), 286-291.
Black, T.H. & Duhon, D.L. (2006). Assessing the impact of business study abroad programs on cultural awareness and personal development. Journal of Education for Business, 81 (3), 140-144. doi: 10.3200/JOEB.81.3.140-144
Braskamp, L.A., Braskamp, D.C., & Engberg, M.E. (2013). Global Perspective Inventory (GPI) Manual. Chicago, IL: Global Perspectives Institute Inc.
Browne & Cudeck. (1993). Alternate ways of assessing model fit. In: K.A. Bollen & J.S. Long (Eds.), Testing structural equation models, (pp.136-162). Beverly Hills, CA: Sage.
Byram, M. (1997). Teaching and assessing intercultural communicative competence. Philadelphia, PA: Multilingual Matters.
Chieffo, L and Griffiths L (2004) Large-scale assessment of student attitudes after a short-term study abroad program. Frontiers: The Interdisciplinary Journal of Study Abroad, X, 165-178.
Clarke, I., Flaherty, T.B., Wright, N.D., & McMillen, R.M. (2009). Student international proficiency from study abroad programs. Journal of Marketing Education, 31 (2), 173-181. doi: 10.1177/0273475309335583
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.
Deardorff, D.K. (2004). The identification and assessment of as a student outcome of internationalization at institutions of higher education in the United States. Unpublished doctoral dissertation. North Carolina State University, Raleigh, NC.
Deardorff, D.K. (2006) Identification and Assessment of Intercultural Competence as a Student
Outcome of Internationalization. Journal of Studies in International Education, 10(3), 241–266.
Deardorff, D.K. (2009). Implementing Intercultural Competence Assessment. In D.K. Deardorff (Ed.), The SAGE Handbook of Intercultural Competence (pp. 477-491). Thousand Oaks, CA: Sage Publications.
Dewaele, J. & Wei, L. (2013). Is multilingualism linked to a higher tolerance for ambiguity? Bilingualism: Language and Cognition, 16, 231-240. Doi:10.1017/S1366728912000570
Dunlap, W.P., Cortina, J.M., Vaslow, J.B., & Burke, M.J. (1996). Meta-analysis of experiments With matched groups or repeated measures designs. Psychological Methods, 1 (2), 170-177.
Fantini, A.E. (2000). A central concern : Developing intercultural competence. SIT Occasional Paper Series, 1. Retrieved April 1, 2014, from: http://www.sit.edu/publications/docs.pdf.
Ganster, D.C., Hennessey, H.W., & Luthans, F. (1983). Social desirability response effects: Three alternative models. Academy of Management Journal, 26 (2), 321-331.
Hammer, M.R. (2010). The Intercultural Development Inventory manual. Berlin, MD: IDI.
Hett, E. J. (1993). The development of an instrument to measure globalmindedness. Unpublished Doctoral Dissertation, University of San Diego.
Hu, L. & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6 (1), 1-55. Doi: 10.1080/10705519909540118.
Institute of International Education. (2013).Open Doors Report on International Educational Exchange. Retrieved from http://www.iie.org/opendoors.
Kazdin, A.E. (1995). Preparing and evaluating research reports. Psychological Assessment, 7 (3), 228-237.
Kenny, D.A. & McCoach, D.B. (2003). Effect of the number of variables on measures of fit indices in structural equation modeling. Structural Equation Modeling, 10, 333-351.
Li, Y. (2013). Cultivating student global competence: A pilot experimental study. Decision Sciences Journal of Innovative Education, 11, 125-143.
McLain, D.L. (1993). The MSTAT-I: A new measure of an individual’s tolerance for ambiguity. Educational and Psychological Measurement, 53 (1), 183-189.
McLain, D.L. (2009). Evidence of the properties of an ambiguity tolerance measure: The Multiple stimulus types ambiguity tolerance scale-II. Psychological Reports, 105(3), 975-988.
Muthen, L.K. & Muthen, B.O. (1998-2008). Mplus User’s Guide. Fifth Edition. Los Angeles, CA: Muthen & Muthen.
Pascarella, E.T., Edison, M., Nora, A., Hagedorn, L.S., & Terenzini, P.T. (1996). Influences on students’ openness to diversity and challenges in the first year of college. Journal of Higher Education, 67 (2), 179-x.
Raykov, T. & Marcoulides, G.A. (2008). An introduction to applied multivariate analysis. New York, NY: Routledge.
Shealy, C.N. (2005). Justifying the justification hypothesis: scientific-humanism, Equilintigration (EI) theory, and the Beliefs, Events, and Values Inventory (BEVI). Journal of Clinical Psychology, 61 (1), 81-106.
Tabachnick, B.G. & Fidell, L.S. (2007). Using multivariate statistics (5th ed.). Boston, MA: Allyn & Bacon.
Wastson, J.R., Siska, P., & Wolfel, R.L. (2013). Assessing gains in language proficiency, cross-cultural competence, and regional awareness during study abroad: A preliminary study. Foreign Language Annals, 46 (1), 62-79. Doi: 10.1111/flan.12016
Williams, T.R. (2005) The reflexive model of Intercultural Competency: a multidimensional, qualitative approach to study abroad assessment. Frontiers Journal, 356-371.
Zhai, L. (2000). The influence of study abroad programs on college student development in the College of Food, Agriculture, and Environmental Science at the Ohio State University. Unpublished doctoral dissertation. Ohio State University, Columbus.
Downloads
Published
2015-11-11
How to Cite
Shadowen , N. L., Chieffo , L. P., & Guerra , N. G. (2015). The Global Engagement Measurement Scale (GEMS): A New Scale for Assessing the Impact of Education Abroad and Campus Internationalization. Frontiers: The Interdisciplinary Journal of Study Abroad, 26(1), 231–247. https://doi.org/10.36366/frontiers.v26i1.368
Issue
Section
Research Articles
