Development and Validation of the Wesleyan Intercultural Competence Scale (WICS): A Tool for Measuring the Impact of Study Abroad Experiences
DOI:
https://doi.org/10.36366/frontiers.v24i1.335Keywords:
Wesleyan Intercultural Competence Scale, Impact, Study Abroad, Education AbroadAbstract
One of the most frequently cited aims of higher education institutions is to help students develop intercultural competence. Study abroad programs are a primary vehicle for helping to achieve this goal; however, it has been difficult to quantify their impact as most existing measures of intercultural competence rely on subjective self-report methods that are easy to fake and that suffer from ceiling effects when attempting to measure change over time. Building on Bennett’s (1986) developmental theory, the current paper describes a new test–the Wesleyan Intercultural Competence Scale (WICS)–that uses a situational judgment testing approach to measure the development of intercultural competence within the context of a study-abroad experience. A total of 97 study-abroad students from Wesleyan took the WICSalong with eight external validation measures and a background questionnaire. Thirty participants took the test at two time points–once at the beginning of a study-abroad program and once at the end. The results indicate that the WICShad strong evidence in support of its content, construct, and criterion-related validity. In addition, the WICSwas capable of detecting changes in the development of intercultural competence over time in a way that none of the other validation measures were. The substantive findings revealed that the amount of time spent speaking the local language and the number of different situations experienced were strong predictors of the development of intercultural competence. Implications and future directions are discussed.
Downloads
Download data is not yet available.
References
American Council on Education. (2002). Beyond September 11: A comprehensive national policy on international education. Washington, DC: American Council on Education.
Antonio, M.J., Chang, K.H., Kenny, D., Levin, S., & Milem, J.F. (2003). Effects of racial diversity on complex thinking in college students. Downloaded from: http://siher.stanford.edu/publicationsalphabytitle.html
Association of American Colleges and Universities. (2010). Raising the bar: Employer’s views on college learning in the wake of the economic downturn. Washington, DC: Hart Research Associates.
Bennett, M. J. (1986). A developmental approach to training for intercultural sensitivity. International Journal of Intercultural Relations, 10(2), 179-195.
Bhandari, R., & Chow, P. (2009). Open doors 2009: Report on international educational exchange. New York: Institute of International Education.
Black, J.S. (1990). The relationship of personal characteristics with adjustment of Japanese expatriate managers. Management International Review, 30, 119-134.
Blaich, C., & Wise, K. (2011). From gathering to using assessment results: Lessons from the Wabash national study. Occassional Paper #8. University of Illinois: National Institute for Learning Outcome Assessment.
Bond, S. (2003b). Untapped resources: Internationalization of the curriculum and classroom experience:A selected literature review (CBIE Research Millennium Series No. 7). Ottawa, ON: Canadian Bureau for International Education.
Braskamp, L.A., Braskamp, D.C., Merrill, K.C., & Engberg, M. (2011). Global Perspective Inventory (GPI): Its purpose, construction, potential uses, and psychometric characteristics. Downloaded May 29, 2012 from: https://gpi.central.edu/supportDocs/manual.pdf
Byram, M. (1997). Teaching and assessing intercultural communicative competence. Clevedon, UK: Mutlilingual Matters.
Burn, B. (2002). The curriculum as a global domain. Journal of Studies in International Education, 6(3), 253-261.
Caligiuri, P. M., Jacobs, R. R., & Farr, J. L. (2000). The Attitudinal and Behavioral Openness Scale: Scale development and construct validation. International Journal of Intercultural Relations, 24, 27-46.
Cohen, J., Cohen, P., West, S.G., & Aiken, L.S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd Ed.). Mahwah, NJ: Lawrence Erlbaum Publishers.
Corbitt, J. N. (1998). Global Awareness Profile: GAP facilitator's manual. Yarmouth, ME: Intercultural Press.
Davis, M.H. (1980). A multidimensional approach to individual differences in empathy. JSAS Catalog of Selected Documents in Psychology. Accessed July 22, 2011 from: http://www.uv.es/~friasnav/Davis_1980.pdf
Deardorff, D.K. (2006). Identification and assessment of intercultural competence as a student outcome of internationalization. Journal of Studies in International Education, 10(3), 241-266.
Deardorff, D.K. (Ed.). (2009). The SAGE handbook of intercultural competence. Thousand Oaks, CA: Sage Publications.
DeVellis, R. F. (1991). Scale development: Theory and applications (Vol. 26). Newbury Park, CA: Sage Publications.
Edwards, J., & Tonkin, H. (1990). Internationalizing the community college: Strategies for the classroom. New Directions for Community Colleges, 70, 17-26.
Engle, L., & Engle, J. (2004). Assessing language acquisition and intercultural sensitivity development in relation to study abroad program design. Frontiers: The Interdisciplinary Journal of Study Abroad, 10, 253-276.
Fantini, A.E. (2006). Exploring and assessing intercultural competence. Retrieved May 29, 2012 from http://digitalcollections.sit.edu/worldlearning_publications/1/
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. C. (2006). The International Personality Item Pool and the future of public-domain personality measures. Journal of Research in Personality, 40, 84-96.
Green, M. F., & Olson, C. (2003). Internationalizing the campus: A user's guide. Washington, DC: American Council on Education.
Gürüz, K. (2011). Higher Education and International Student Mobility in the Global Knowledge Economy (2nd Ed.). Albany, New York: SUNY Press.
Hammer, M. R. (2011). Additional cross-cultural validity testing of the Intercultural Development Inventory. International Journal of Intercultural Relations, 35, 474-487.
Hammer, M. R. (1998). A measure of intercultural sensitivity: The Intercultural Development Inventory. . In S. Fowler & M. Fowler (Eds.), The intercultural sourcebook (Vol. 2). Yarmouth, ME: Intercultural Press.: Intercultural Press.
Hammer, M. R., Bennett, M. J., & Wiseman, R. (2003). Measuring intercultural sensitivity: The Intercultural Development Inventory. International Journal of Intercultural Relations: Special Training Issue, 27(4), 421-443.
Hirsch, E. D. (1987). Cultural literacy : what every American needs to know. Boston: Houghton Mifflin.
Hoffa, W.W. (2010). A history of U.S. study abroad: Beginnings to 1965. A special publication of Frontiers: The interdisciplinary journal of study abroad and The forum on education abroad.
Holsti, O.R. (1969). Content analysis for the social sciences and humanities. Reading, MA: Addison-Wesley.
Hurtado, S. (2001). Linking diversity and educational purpose: How diversity impacts the classroom environment and student development. In G. Orfield and M. Kurlaender (Eds.), Diversity challenged: Evidence on the impact of affirmative action. Harvard Civil Rights Project.
International Personality Item Pool: A Scientific Collaboratory for the Development of Advanced Measures of Personality Traits and Other Individual Differences (http://ipip.ori.org).
Institute of International Education. (2011). Open doors report 2011. Downloaded May 18, 2012 from http://www.iie.org/en/Research-and-Publications/Open-Doors
Kosterman R., & Feshbach, S. (1989). Toward a measure of patriotic and nationalistic attitudes. Political Psychology, 10(2), 257-274.
Krippendorff, K. (2003). Content analysis: an introduction to its methodology (2nd Ed.). Newbury Park, CA: Sage.
Kuh, G. (2003). What we’re learning about student engagement from NSSE. Change, Mar-Apr., 24-32.
Lincoln Commission (2005). Global competence and national needs: One million Americans studying abroad. Final report from the Commission on the Abraham Lincoln Fellowship Program, Washington, DC.
Lou, K., & Bosley, G. (2008). Dynamics of cultural contexts: Meta-level interventions in the study abroad experience. In V. Savicki (Ed.). Developing intercultural competence and transformation: Theory, research, and applications in international education (pp. 276-296). Sterling, VA: Stylus.
MacDonald, A. P. (1970). Revised scale for ambiguity tolerance: reliability and validity. Psychological Reports, 26, 791-798.
McDaniel, M. A., & Nguyen, N. T. (2001). Situational judgment tests: A review of practice and constructs assessed. International Journal of Selection & Assessment, 9(1-2), 103-113.
Nilsson, B. (2000). Internationalising the curriculum. In P. Crowther, M. Joris, M. Otten, B. Nilsson, H. Teekens, & B. Wächter (Eds), Internationalisation at home: A position paper (pp. 21-27). Amsterdam: European Association for International Education.
Paige, R.M., Jacobs-Cassuto, M., Yershova, Y.A., & DeJaeghere, J. (2003). Assessing intercultural sensitivity: An empirical analysis of the Hammer and Bennett Intercultural Development Inventory. International Journal of Intercultural Relations, 27, 467-486.
Salisbury, M.H. (2011). The effect of study abroad on intercultural competence among undergraduate college students. Doctoral dissertation; University of Iowa. Downloaded April 14, 2012 from: http://ir.uiowa.edu/edt/1073.
Savicki, V., & Selby, R. (2008). Synthesis and conclusions. In V. Savicki (Ed.). Developing intercultural competence and transformation: Theory, research, and applications in international education (pp. 276-296). Sterling, VA: Stylus.
Schapper, J. N., & Mayson, S. E. (2004). Internationalisation of curricula: An alternative to the Taylorisation of academic work. Journal of Higher Education Policy and Management, 26(2), 189-205.
Schuerholz-Lehr, S., Caws, C., van Gyn, G., & Preece, A. (2007). Internationalizing the higher education curriculum: An emerging model for transforming faculty perspectives. Canadian Journal of Higher Education, 37(1), 67-94.
Sinicrope, C., Norris, J., & Watanabe, Y. (2007). Understanding and assessing intercultural competence: A summary of theory, research and practice. Second Language Studies, 26(1), 1-58.
Spitzberg, B.H., & Changnon, G. (2009). Conceptualizing intercultural competence. In D.K. Deardorff (Ed.), The SAGE handbook of intercultural competence (pp. 2-52). Thousand Oaks, CA: Sage publications.
Stemler. S.E. (2004). A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Practical Assessment, Research, and Evaluation, 9(4). Retrieved May 31, 2012 from: http://PAREonline.net/getvn.asp?v=9&n=4
Stemler, S.E. (2012). What should university admissions tests predict? Educational Psychologist, 41(1), 5-17.
Sternberg, R.J. (2004). Culture and intelligence. American psychologist, 59(5), 325-338.
Ting-Toomey, S., & Kurogia, A. (1998). Facework competence in intercultural conflict: An updated face-negotiation theory. International Journal of Intercultural Relations, 22, 187-25.
Vande Berg, M., Connor-Linton, J., & Paige, M. R. (2009). The Georgetown Consortium project:Interventions for student learning abroad. Frontiers: The Interdisciplinary Journal of International Education, 18, 1-76.
Van de Mortel, T.F. (2008). Faking it: social desirability response bias in self-report research. Australian Journal of Advanced Nursing, 25(4), 40-48.
Weekley, J. A., & Ployhart, R. E. (Eds.). (2006). Situational judgment tests. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Antonio, M.J., Chang, K.H., Kenny, D., Levin, S., & Milem, J.F. (2003). Effects of racial diversity on complex thinking in college students. Downloaded from: http://siher.stanford.edu/publicationsalphabytitle.html
Association of American Colleges and Universities. (2010). Raising the bar: Employer’s views on college learning in the wake of the economic downturn. Washington, DC: Hart Research Associates.
Bennett, M. J. (1986). A developmental approach to training for intercultural sensitivity. International Journal of Intercultural Relations, 10(2), 179-195.
Bhandari, R., & Chow, P. (2009). Open doors 2009: Report on international educational exchange. New York: Institute of International Education.
Black, J.S. (1990). The relationship of personal characteristics with adjustment of Japanese expatriate managers. Management International Review, 30, 119-134.
Blaich, C., & Wise, K. (2011). From gathering to using assessment results: Lessons from the Wabash national study. Occassional Paper #8. University of Illinois: National Institute for Learning Outcome Assessment.
Bond, S. (2003b). Untapped resources: Internationalization of the curriculum and classroom experience:A selected literature review (CBIE Research Millennium Series No. 7). Ottawa, ON: Canadian Bureau for International Education.
Braskamp, L.A., Braskamp, D.C., Merrill, K.C., & Engberg, M. (2011). Global Perspective Inventory (GPI): Its purpose, construction, potential uses, and psychometric characteristics. Downloaded May 29, 2012 from: https://gpi.central.edu/supportDocs/manual.pdf
Byram, M. (1997). Teaching and assessing intercultural communicative competence. Clevedon, UK: Mutlilingual Matters.
Burn, B. (2002). The curriculum as a global domain. Journal of Studies in International Education, 6(3), 253-261.
Caligiuri, P. M., Jacobs, R. R., & Farr, J. L. (2000). The Attitudinal and Behavioral Openness Scale: Scale development and construct validation. International Journal of Intercultural Relations, 24, 27-46.
Cohen, J., Cohen, P., West, S.G., & Aiken, L.S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd Ed.). Mahwah, NJ: Lawrence Erlbaum Publishers.
Corbitt, J. N. (1998). Global Awareness Profile: GAP facilitator's manual. Yarmouth, ME: Intercultural Press.
Davis, M.H. (1980). A multidimensional approach to individual differences in empathy. JSAS Catalog of Selected Documents in Psychology. Accessed July 22, 2011 from: http://www.uv.es/~friasnav/Davis_1980.pdf
Deardorff, D.K. (2006). Identification and assessment of intercultural competence as a student outcome of internationalization. Journal of Studies in International Education, 10(3), 241-266.
Deardorff, D.K. (Ed.). (2009). The SAGE handbook of intercultural competence. Thousand Oaks, CA: Sage Publications.
DeVellis, R. F. (1991). Scale development: Theory and applications (Vol. 26). Newbury Park, CA: Sage Publications.
Edwards, J., & Tonkin, H. (1990). Internationalizing the community college: Strategies for the classroom. New Directions for Community Colleges, 70, 17-26.
Engle, L., & Engle, J. (2004). Assessing language acquisition and intercultural sensitivity development in relation to study abroad program design. Frontiers: The Interdisciplinary Journal of Study Abroad, 10, 253-276.
Fantini, A.E. (2006). Exploring and assessing intercultural competence. Retrieved May 29, 2012 from http://digitalcollections.sit.edu/worldlearning_publications/1/
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. C. (2006). The International Personality Item Pool and the future of public-domain personality measures. Journal of Research in Personality, 40, 84-96.
Green, M. F., & Olson, C. (2003). Internationalizing the campus: A user's guide. Washington, DC: American Council on Education.
Gürüz, K. (2011). Higher Education and International Student Mobility in the Global Knowledge Economy (2nd Ed.). Albany, New York: SUNY Press.
Hammer, M. R. (2011). Additional cross-cultural validity testing of the Intercultural Development Inventory. International Journal of Intercultural Relations, 35, 474-487.
Hammer, M. R. (1998). A measure of intercultural sensitivity: The Intercultural Development Inventory. . In S. Fowler & M. Fowler (Eds.), The intercultural sourcebook (Vol. 2). Yarmouth, ME: Intercultural Press.: Intercultural Press.
Hammer, M. R., Bennett, M. J., & Wiseman, R. (2003). Measuring intercultural sensitivity: The Intercultural Development Inventory. International Journal of Intercultural Relations: Special Training Issue, 27(4), 421-443.
Hirsch, E. D. (1987). Cultural literacy : what every American needs to know. Boston: Houghton Mifflin.
Hoffa, W.W. (2010). A history of U.S. study abroad: Beginnings to 1965. A special publication of Frontiers: The interdisciplinary journal of study abroad and The forum on education abroad.
Holsti, O.R. (1969). Content analysis for the social sciences and humanities. Reading, MA: Addison-Wesley.
Hurtado, S. (2001). Linking diversity and educational purpose: How diversity impacts the classroom environment and student development. In G. Orfield and M. Kurlaender (Eds.), Diversity challenged: Evidence on the impact of affirmative action. Harvard Civil Rights Project.
International Personality Item Pool: A Scientific Collaboratory for the Development of Advanced Measures of Personality Traits and Other Individual Differences (http://ipip.ori.org).
Institute of International Education. (2011). Open doors report 2011. Downloaded May 18, 2012 from http://www.iie.org/en/Research-and-Publications/Open-Doors
Kosterman R., & Feshbach, S. (1989). Toward a measure of patriotic and nationalistic attitudes. Political Psychology, 10(2), 257-274.
Krippendorff, K. (2003). Content analysis: an introduction to its methodology (2nd Ed.). Newbury Park, CA: Sage.
Kuh, G. (2003). What we’re learning about student engagement from NSSE. Change, Mar-Apr., 24-32.
Lincoln Commission (2005). Global competence and national needs: One million Americans studying abroad. Final report from the Commission on the Abraham Lincoln Fellowship Program, Washington, DC.
Lou, K., & Bosley, G. (2008). Dynamics of cultural contexts: Meta-level interventions in the study abroad experience. In V. Savicki (Ed.). Developing intercultural competence and transformation: Theory, research, and applications in international education (pp. 276-296). Sterling, VA: Stylus.
MacDonald, A. P. (1970). Revised scale for ambiguity tolerance: reliability and validity. Psychological Reports, 26, 791-798.
McDaniel, M. A., & Nguyen, N. T. (2001). Situational judgment tests: A review of practice and constructs assessed. International Journal of Selection & Assessment, 9(1-2), 103-113.
Nilsson, B. (2000). Internationalising the curriculum. In P. Crowther, M. Joris, M. Otten, B. Nilsson, H. Teekens, & B. Wächter (Eds), Internationalisation at home: A position paper (pp. 21-27). Amsterdam: European Association for International Education.
Paige, R.M., Jacobs-Cassuto, M., Yershova, Y.A., & DeJaeghere, J. (2003). Assessing intercultural sensitivity: An empirical analysis of the Hammer and Bennett Intercultural Development Inventory. International Journal of Intercultural Relations, 27, 467-486.
Salisbury, M.H. (2011). The effect of study abroad on intercultural competence among undergraduate college students. Doctoral dissertation; University of Iowa. Downloaded April 14, 2012 from: http://ir.uiowa.edu/edt/1073.
Savicki, V., & Selby, R. (2008). Synthesis and conclusions. In V. Savicki (Ed.). Developing intercultural competence and transformation: Theory, research, and applications in international education (pp. 276-296). Sterling, VA: Stylus.
Schapper, J. N., & Mayson, S. E. (2004). Internationalisation of curricula: An alternative to the Taylorisation of academic work. Journal of Higher Education Policy and Management, 26(2), 189-205.
Schuerholz-Lehr, S., Caws, C., van Gyn, G., & Preece, A. (2007). Internationalizing the higher education curriculum: An emerging model for transforming faculty perspectives. Canadian Journal of Higher Education, 37(1), 67-94.
Sinicrope, C., Norris, J., & Watanabe, Y. (2007). Understanding and assessing intercultural competence: A summary of theory, research and practice. Second Language Studies, 26(1), 1-58.
Spitzberg, B.H., & Changnon, G. (2009). Conceptualizing intercultural competence. In D.K. Deardorff (Ed.), The SAGE handbook of intercultural competence (pp. 2-52). Thousand Oaks, CA: Sage publications.
Stemler. S.E. (2004). A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Practical Assessment, Research, and Evaluation, 9(4). Retrieved May 31, 2012 from: http://PAREonline.net/getvn.asp?v=9&n=4
Stemler, S.E. (2012). What should university admissions tests predict? Educational Psychologist, 41(1), 5-17.
Sternberg, R.J. (2004). Culture and intelligence. American psychologist, 59(5), 325-338.
Ting-Toomey, S., & Kurogia, A. (1998). Facework competence in intercultural conflict: An updated face-negotiation theory. International Journal of Intercultural Relations, 22, 187-25.
Vande Berg, M., Connor-Linton, J., & Paige, M. R. (2009). The Georgetown Consortium project:Interventions for student learning abroad. Frontiers: The Interdisciplinary Journal of International Education, 18, 1-76.
Van de Mortel, T.F. (2008). Faking it: social desirability response bias in self-report research. Australian Journal of Advanced Nursing, 25(4), 40-48.
Weekley, J. A., & Ployhart, R. E. (Eds.). (2006). Situational judgment tests. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Downloads
Published
2014-08-15
How to Cite
Stemler, S., Imada, T., & Sorkin, C. (2014). Development and Validation of the Wesleyan Intercultural Competence Scale (WICS): A Tool for Measuring the Impact of Study Abroad Experiences. Frontiers: The Interdisciplinary Journal of Study Abroad, 24(1), 25–58. https://doi.org/10.36366/frontiers.v24i1.335
Issue
Section
Research Articles
