The Global Engagement Measurement Scale (GEMS): A New Scale for Assessing the Impact of Education Abroad and Campus Internationalization

  • Noel L. Shadowen
  • Lisa P. Chieffo
  • Nancy G. Guerra
Keywords: Global Engagement Measurement, Assessment, Impact of study abroad, Study abroad, Education abroad, Campus internationalization

Abstract

We describe the development and validation of a new instrument to measure the non-academic impact of education abroad, the Global Engagement Measurement Scale (GEMS). Based on a review of the extant literature, we developed a 37-item instrument containing scales for four specific constructs of interest: a) cultural engagement; b) ambiguity tolerance; c) knowledge of the host site; and d) diversity openness. The measure was piloted with 263 education abroad students before and after a 5- week education abroad program. We examined the scales’ underlying factor structure, reliability, and sensitivity to education abroad program effects. Results indicate strong reliability and statistically significant increases across all scales following education abroad experiences. The GEMS instrument is brief and easy to administer, publically available, and includes important constructs for assessing the impact of education abroad. Additionally, the GEMS could be used to assess campus internationalization efforts with domestic and international students.

Downloads

Download data is not yet available.

Author Biographies

Noel L. Shadowen

Noel Shadowen is a Ph.D. student in Clinical Science at the University of Delaware, focusing on cultural and international issues and mental health. Her current research examines links between trauma and negative mental health outcomes with marginalized elementary children in India. At the Institute for Global Studies Noel works on research involving both international students and education abroad student outcomes. Noel received her MA in International Development Studies from The George Washington University.

Lisa P. Chieffo

Lisa Chieffo is Associate Director for Study Abroad in the Institute for Global Studies at the University of Delaware. Overseeing some 70 global programs enrolling about 1,300 students annually, she is responsible for program development, crisis management, policy development and implementation, and assessment.  With 20 years’ experience in education abroad, Lisa has published widely and presented at multiple conferences. She earned her MA in German and Ed.D. in Educational Leadership from the University of Delaware.

Nancy G. Guerra

Dr. Nancy Guerra is the Director of the Institute for Global Studies at the University of Delaware and a faculty member in the Department of Psychological & Brain Sciences. Dr. Guerra workson many international projects funded by The World Bank, The IADB, and USAID. These projects emphasize improved educational and youth services in Latin America and the Caribbean. She has also served on several national panels. She received her doctorate from Harvard University.

References

Ashwill, M.A., & Hoang Oanh, D.T. (2009). Developing Globally Competent Citizens: The Contrasting Cases of the United States and Vietnam. In D.K. Deardorff (Ed.), The SAGE Handbook of Intercultural Competence (pp. 141-157). Thousand Oaks, CA: Sage Publications.

Bakalis & Joiner. (2004). Participation in tertiary study abroad programs: the role of personality. International Journal of Educational Management, 18 (5), 286-291.

Black, T.H. & Duhon, D.L. (2006). Assessing the impact of business study abroad programs on cultural awareness and personal development. Journal of Education for Business, 81 (3), 140-144. doi: 10.3200/JOEB.81.3.140-144

Braskamp, L.A., Braskamp, D.C., & Engberg, M.E. (2013). Global Perspective Inventory (GPI) Manual. Chicago, IL: Global Perspectives Institute Inc.

Browne & Cudeck. (1993). Alternate ways of assessing model fit. In: K.A. Bollen & J.S. Long (Eds.), Testing structural equation models, (pp.136-162). Beverly Hills, CA: Sage.

Byram, M. (1997). Teaching and assessing intercultural communicative competence. Philadelphia, PA: Multilingual Matters.

Chieffo, L and Griffiths L (2004) Large-scale assessment of student attitudes after a short-term study abroad program. Frontiers: The Interdisciplinary Journal of Study Abroad, X, 165-178.

Clarke, I., Flaherty, T.B., Wright, N.D., & McMillen, R.M. (2009). Student international proficiency from study abroad programs. Journal of Marketing Education, 31 (2), 173-181. doi: 10.1177/0273475309335583

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.

Deardorff, D.K. (2004). The identification and assessment of as a student outcome of internationalization at institutions of higher education in the United States. Unpublished doctoral dissertation. North Carolina State University, Raleigh, NC.

Deardorff, D.K. (2006) Identification and Assessment of Intercultural Competence as a Student
Outcome of Internationalization. Journal of Studies in International Education, 10(3), 241–266.

Deardorff, D.K. (2009). Implementing Intercultural Competence Assessment. In D.K. Deardorff (Ed.), The SAGE Handbook of Intercultural Competence (pp. 477-491). Thousand Oaks, CA: Sage Publications.

Dewaele, J. & Wei, L. (2013). Is multilingualism linked to a higher tolerance for ambiguity? Bilingualism: Language and Cognition, 16, 231-240. Doi:10.1017/S1366728912000570

Dunlap, W.P., Cortina, J.M., Vaslow, J.B., & Burke, M.J. (1996). Meta-analysis of experiments With matched groups or repeated measures designs. Psychological Methods, 1 (2), 170-177.

Fantini, A.E. (2000). A central concern : Developing intercultural competence. SIT Occasional Paper Series, 1. Retrieved April 1, 2014, from: http://www.sit.edu/publications/docs.pdf.

Ganster, D.C., Hennessey, H.W., & Luthans, F. (1983). Social desirability response effects: Three alternative models. Academy of Management Journal, 26 (2), 321-331.

Hammer, M.R. (2010). The Intercultural Development Inventory manual. Berlin, MD: IDI.

Hett, E. J. (1993). The development of an instrument to measure globalmindedness. Unpublished Doctoral Dissertation, University of San Diego.

Hu, L. & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6 (1), 1-55. Doi: 10.1080/10705519909540118.

Institute of International Education. (2013).Open Doors Report on International Educational Exchange. Retrieved from http://www.iie.org/opendoors.

Kazdin, A.E. (1995). Preparing and evaluating research reports. Psychological Assessment, 7 (3), 228-237.

Kenny, D.A. & McCoach, D.B. (2003). Effect of the number of variables on measures of fit indices in structural equation modeling. Structural Equation Modeling, 10, 333-351.

Li, Y. (2013). Cultivating student global competence: A pilot experimental study. Decision Sciences Journal of Innovative Education, 11, 125-143.

McLain, D.L. (1993). The MSTAT-I: A new measure of an individual’s tolerance for ambiguity. Educational and Psychological Measurement, 53 (1), 183-189.

McLain, D.L. (2009). Evidence of the properties of an ambiguity tolerance measure: The Multiple stimulus types ambiguity tolerance scale-II. Psychological Reports, 105(3), 975-988.

Muthen, L.K. & Muthen, B.O. (1998-2008). Mplus User’s Guide. Fifth Edition. Los Angeles, CA: Muthen & Muthen.

Pascarella, E.T., Edison, M., Nora, A., Hagedorn, L.S., & Terenzini, P.T. (1996). Influences on students’ openness to diversity and challenges in the first year of college. Journal of Higher Education, 67 (2), 179-x.

Raykov, T. & Marcoulides, G.A. (2008). An introduction to applied multivariate analysis. New York, NY: Routledge.

Shealy, C.N. (2005). Justifying the justification hypothesis: scientific-humanism, Equilintigration (EI) theory, and the Beliefs, Events, and Values Inventory (BEVI). Journal of Clinical Psychology, 61 (1), 81-106.

Tabachnick, B.G. & Fidell, L.S. (2007). Using multivariate statistics (5th ed.). Boston, MA: Allyn & Bacon.

Wastson, J.R., Siska, P., & Wolfel, R.L. (2013). Assessing gains in language proficiency, cross-cultural competence, and regional awareness during study abroad: A preliminary study. Foreign Language Annals, 46 (1), 62-79. Doi: 10.1111/flan.12016

Williams, T.R. (2005) The reflexive model of Intercultural Competency: a multidimensional, qualitative approach to study abroad assessment. Frontiers Journal, 356-371.

Zhai, L. (2000). The influence of study abroad programs on college student development in the College of Food, Agriculture, and Environmental Science at the Ohio State University. Unpublished doctoral dissertation. Ohio State University, Columbus.
Published
2015-11-11
How to Cite
Shadowen , N. L., Chieffo , L. P., & Guerra , N. G. (2015). The Global Engagement Measurement Scale (GEMS): A New Scale for Assessing the Impact of Education Abroad and Campus Internationalization. Frontiers: The Interdisciplinary Journal of Study Abroad, 26(1), 231-247. https://doi.org/10.36366/frontiers.v26i1.368
Section
Research Articles