Assessing Study Abroad: Rubric Shopping, Managerialism, and Audit Culture at the Neoliberal University
DOI:
https://doi.org/10.36366/frontiers.v35i1.766Keywords:
Study abroad assessment, taxonomies of learning, academic capitalism, intercultural competence, intercultural communication, global citizenship, assessment toolsAbstract
This article investigates the increasing prominence of intercultural and global skills assessment in study abroad administration and pedagogy, and how it influences the practice of international education and the roles it plays in the administrative spaces of U.S. higher education. Drawing on a series of interviews with administrators, faculty leaders, and international programs staff, as well as the authors’ experience in international education, this essay explores the diverse functions served by assessment rubrics in real-world educational contexts. Drawing on the work of Shore (2008), Wright and Shore (2017), Doerr (2015, 2017), Slaughter (2014), and Slaughter et al. (2004), we first discuss the rise in popularity of quantitative assessment tools that purport to measure individuals’ and groups’ intercultural abilities, awareness, or “competence,” in the context of neoliberalization and “audit culture” in higher education. We then describe the results of our qualitative research, focusing on (1) the diverse relationships different faculty, staff, and administrators have to assessment rubrics and their implementation; (2) their importance in administrative decision making and accreditation processes; (3) their relationship to neoliberalization concerns in international education and its increasing professionalization; and (4) the role these rubrics can play in promoting intentional program design and pedagogy. We demonstrate that international programs stakeholders engage in what we call rubric shopping, in which they move between different assessment tools and implement them in different ways and toward different ends, for a variety of contrasting reasons. We argue that these rubrics’ power in administrative spaces comes from the reification of authored concepts like “global citizenship” and “intercultural competence,” along with the scientism of the tools themselves; this cultivates a fuzzy, variable perception of positivism that is juxtaposed to and undermined by the ongoing practice of rubric shopping, among other factors. Finally, we encourage study abroad practitioners maintain a degree of critical distance from the field of intercultural skills assessment, while recognizing its utility in advocating for particular outcomes and pedagogical interventions.
Downloads
References
Acheson, K., Yngve, K. N., Stahl, A., & Jin, L. (2020). Introduction: Recent research in assessment in education abroad. Frontiers: The Interdisciplinary Journal of Study Abroad, 32(virtual), 1-6. https://doi.org/10.36366/frontiers.v32iVirtual.464 DOI: https://doi.org/10.36366/frontiers.v32iVirtual.464
Altbach, P. G., & Knight, J. (2007). The internationalization of higher education: Motivations and realities. Journal of Studies in International Education, 11(3-4), 290-305. https://doi.org/10.1177/1028315307303542 DOI: https://doi.org/10.1177/1028315307303542
Andreotti, V. (2006). Soft versus Critical Global Citizenship Education. In Development Education in Policy and Practice (pp. 21–31). Palgrave Macmillan UK. https://doi.org/10.1057/9781137324665_2 DOI: https://doi.org/10.1057/9781137324665_2
Astin, A. W., & Antonio, A. L. (2012). Assessment for excellence the philosophy and practice of assessment and evaluation in higher education (2nd ed.). Rowman & Littlefield Publishers.
Banta, T. W. (2002). Building a scholarship of assessment. Jossey-Bass.
Barkin, G. (2016). Undergraduate research on short-term, faculty-led study abroad. Council on Undergraduate Research Quarterly, 36(4), 26-32. DOI: https://doi.org/10.18833/curq/36/4/2
Barkin, G. (2018). Either Here or There: Short‐Term Study Abroad and the Discourse of Going. Anthropology & Education Quarterly, 49(3), 296–317. https://doi.org/10.1111/aeq.12248 DOI: https://doi.org/10.1111/aeq.12248
Barkin, G. (2021). The imperative of access in short-term study abroad: Provider agencies, liminality, and the mediation of cultural difference. In M. A. Di Giovine, & J. Bodinger de Uriarte (Eds.), Study abroad: Service, student travel and the quest for an anti-tourism experience (119-148). Lexington Books.
Bennett, M. J. (2010). A short conceptual history of intercultural learning in study abroad. In W. Hoffa, & S. C. DePaul (Eds.), A history of U.S. study abroad: 1965-present (pp. 419-449). Frontiers Special Publication.
Bennett, M. J. (2012). Paradigmatic assumptions and a developmental approach to intercultural learning. In M. Vande Berg, R. M. Paige & K. H. Lou (Eds.), Student learning abroad: What our students are learning, what they're not, and what we can do about it (pp. 90-114). Stylus.
Bennett, M. J. (2013). The ravages of reification: Considering the iceberg and cultural intelligence, towards de-reifying intercultural competence keynote presentation for FILE IV, colle val d’Elsa, Sept. 28, 2013.
Bennett, M. J., & Hammer, M. (1998). The intercultural development inventory (IDI) manual. The Intercultural Communication Institute.
Boellstorff, T. (2005). Between Religion and Desire: Being Muslim and Gay in Indonesia. American Anthropologist, 107(4), 575–585. https://doi.org/10.1525/aa.2005.107.4.575 DOI: https://doi.org/10.1525/aa.2005.107.4.575
Chen, X., & Gabrenya Jr, W. K. (2021). In search of cross-cultural competence: A comprehensive review of five measurement instruments. International Journal of Intercultural Relations, 82, 37–55. https://doi.org/10.1016/j.ijintrel.2021.02.003 DOI: https://doi.org/10.1016/j.ijintrel.2021.02.003
Collins, L. (2019). Letting the village be the teacher: A look at community-based learning in northern Thailand. Teaching in Higher Education, 24(5), 694-708. https://doi.org/10.1080/13562517.2019.1579708 DOI: https://doi.org/10.1080/13562517.2019.1579708
Collins, L. (2021). Offering space at the table: The work of hosting US study abroad students in northern Thailand. Third World Quarterly, 42(8), 1849-1865. https://doi.org/10.1080/01436597.2021.1937097 DOI: https://doi.org/10.1080/01436597.2021.1937097
Collins, L. (2020). Becoming global knowledge workers: How U.S. study abroad programs act in service to a hidden curriculum of consumer capitalism and national security. Journal of Philosophy and Theory in Higher Education, 2, 1-20. https://doi.org/10.3726/PTIHE022020.0001 DOI: https://doi.org/10.3726/PTIHE022020.0001
Croucher, S. M., Sommier, M., & Rahmani, D. (2015). Intercultural communication: Where we’ve been, where we’re going, issues we face. Communication Research and Practice, 1(1), 71–87. https://doi.org/10.1080/22041451.2015.1042422 DOI: https://doi.org/10.1080/22041451.2015.1042422
Cultural Intelligence Center. (2021). About cultural intelligence. https://culturalq.com/about-cultural-intelligence/
Danso, R. (2018). Cultural competence and cultural humility: A critical reflection on key cultural diversity concepts. Journal of Social Work, 18(4), 410-430. https://doi.org/10.1177/1468017316654341 DOI: https://doi.org/10.1177/1468017316654341
Deardorff, D. K. (2011). Assessing intercultural competence. New Directions for Institutional Research, 2011(149), 65-79. https://doi.org/10.1002/ir.381 DOI: https://doi.org/10.1002/ir.381
Doerr, N. M. (2015). Reproduction of Difference Through Learning About a “Different Culture”: The Paradox of Double Subject Positions and the Pedagogy of the Privileged. The Review of Education/pedagogy/cultural Studies, 37(1), 71–89. https://doi.org/10.1080/10714413.2015.988539 DOI: https://doi.org/10.1080/10714413.2015.988539
Doerr, N. M. (2017). Learning as othering: narratives of learning, construction of difference and the discourse of immersion in study abroad. Intercultural Education (London, England), 28(1), 90–103. https://doi.org/10.1080/14675986.2017.1288482 DOI: https://doi.org/10.1080/14675986.2017.1288482
Doerr, N. M. (2020). 'Global competence' of minority immigrant students: Hierarchy of experience and ideology of global competence in study abroad. Discourse, 41(1), 83-97. https://doi.org/10.1080/01596306.2018.1462147 DOI: https://doi.org/10.1080/01596306.2018.1462147
Doyle, D. (2009). Holistic assessment and the study abroad experience. Frontiers: The Interdisciplinary Journal of Study Abroad, 18, 143-155. https://doi.org/10.36366/frontiers.v18i1.258 DOI: https://doi.org/10.36366/frontiers.v18i1.258
Engle, L. (2013). The rewards of qualitative assessment appropriate to study abroad. Frontiers: The Interdisciplinary Journal of Study Abroad, 22, 111-126. https://doi.org/10.36366/frontiers.v22i1.321 DOI: https://doi.org/10.36366/frontiers.v22i1.321
Ewell, P. T. (2002). An emerging scholarship: A brief history of assessment. In T. W. Banta & Associates (Eds.), Building a scholarship of assessment (pp. 3–25). Jossey-Bass.
Fantini, A. (2009). Assessing intercultural competence: Issues and tools. In D. Deardorff (Ed.), (pp. 456-476). Sage. DOI: https://doi.org/10.4135/9781071872987.n27
Foucault, M. (2005 [1966]). The order of things: An archaeology of the human sciences. Routledge.
Fuller, M. B., Skidmore, S. T., Bustamante, R. M., & Holzweiss, P. C. (2016). Empirically exploring higher education cultures of assessment. Review of Higher Education, 39(3), 395-429. https://doi.org/10.1353/rhe.2016.0022 DOI: https://doi.org/10.1353/rhe.2016.0022
Hadis, B. F. (2005). Gauging the impact of study abroad: how to overcome the limitations of a single-cell design. Assessment and Evaluation in Higher Education, 30(1), 3–19. https://doi.org/10.1080/0260293042003243869 DOI: https://doi.org/10.1080/0260293042003243869
Hall, E. T. (1973). The silent language. Anchor.
Hall, E. T., & Whyte, W. F. (1960). Intercultural communication: A guide to men of action. Human Organization, 19(1), 5-12. https://doi.org/10.17730/humo.19.1.6432l3505p206231 DOI: https://doi.org/10.17730/humo.19.1.6432l3505p206231
Hammer, M. R. (2012). The intercultural development inventory: A new frontier in assessment and development of intercultural competence. In M. Vande Berg, R. M. Paige & K. H. Lou (Eds.), Student learning abroad: What our students are learning, what they're not, and what we can do about it (pp. 115-136). Stylus.
Hammer, M. R., Bennett, M. J., & Wiseman, R. (2003). The intercultural development inventory: A measure of intercultural sensitivity. International Journal of Intercultural Relations, 27(4), 421-443. https://doi.org/10.1016/S0147-1767(03)00032-4 DOI: https://doi.org/10.1016/S0147-1767(03)00032-4
Harris, M. (1976). History and Significance of the EMIC/ETIC Distinction. Annual Review of Anthropology, 5(1), 329–350. https://doi.org/10.1146/annurev.an.05.100176.001553 DOI: https://doi.org/10.1146/annurev.an.05.100176.001553
Rogers, E.M., W.B. Hart, Y. Miike. (2002). Edward T. hall and the history of intercultural communication: The United States and Japan. Keio Communication Review, (24), 3-26.
Hartle, T. (2012). Accreditation and the public interest: Can accreditors continue to play a central role in public policy? Planning for Higher Education, 40(3), 16-21.
Hofstede, G. (2001). Culture's consequences : Comparing values, behaviors, institutions and organizations across nations (2nd ed.). Sage Publications.
Holmes, D. S. (2007). Intercultural communication and dialogue process: An attempt at clarification and synthesis. IDM: International Society of Diversity Management, 1.
Holzweiss, P. C., Bustamante, R., & Fuller, M. B. (2016). Institutional cultures of assessment: A qualitative study of administrator perspectives. Journal of Assessment and Institutional Effectiveness, 6(1), 1-27. https://doi.org/10.5325/jasseinsteffe.6.1.0001 DOI: https://doi.org/10.5325/jasseinsteffe.6.1.1
Intercultural Development Inventory. (2021). The roadmap to intercultural competence using the IDI. https://idiinventory.com/generalinformation/12-reasons-for-using-the-idi/
Intercultural Development Inventory. (2022). The roadmap to intercultural competence using the IDI: IDI validation research. https://idiinventory.com/idi-validation/
Javidan, M., Hough, L., & Bullough, A. (2010). Conceptualizing and measuring global mindset®: Development of the global mindset inventory. Technical Report. Thunderbird School of Global Management. DOI: https://doi.org/10.1108/S1535-1203(2011)0000006005
Kauffmann, N. L., & Kuh, G. D. (1984). The impact of study abroad on personal development of college students. ERIC Clearinghouse.
Kehl, K., & Morris, J. (2007). Differences in Global-Mindedness between Short-Term and Semester-Long Study Abroad Participants. Frontiers (Boston, Mass.), 15(1), 67–80. https://doi.org/10.36366/frontiers.v15i1.217 DOI: https://doi.org/10.36366/frontiers.v15i1.217
Keuth, H. (2015). The positivist dispute in German sociology: A scientific or a political controversy? Journal of Classical Sociology: JCS, 15(2), 154–169. https://doi.org/10.1177/1468795X14567825 DOI: https://doi.org/10.1177/1468795X14567825
Koester, J., & Lustig, M. W. (2015). Intercultural communication competence: Theory, measurement, and application. International Journal of Intercultural Relations, 48, 20-21. https://doi.org/10.1016/j.ijintrel.2015.03.006 DOI: https://doi.org/10.1016/j.ijintrel.2015.03.006
Lee, A. (2017). Teaching interculturally: A framework for integrating disciplinary knowledge and intercultural development. Stylus Publishing, LLC.
Lee, J., & Negrelli, K. (2022). Two countries, one goal: A mixed-methods study of the development of study abroad sojourners’ cultural identification in japan and korea. Frontiers: The Interdisciplinary Journal of Study Abroad, 34(1), 131-162. https://doi.org/10.36366/frontiers.v34i1.601 DOI: https://doi.org/10.36366/frontiers.v34i1.601
Levy, O., Beechler, S., Taylor, S., & Boyacigiller, N. A. (2007). What we talk about when we talk about 'global mindset': Managerial cognition in multinational corporations. Journal of International Business Studies, 38(2), 231-258. https://doi.org/10.1057/palgrave.jibs.8400265 DOI: https://doi.org/10.1057/palgrave.jibs.8400265
Maurer, R. (2019). Do we really need change management plans? The Journal for Quality and Participation, 42(1), 1-2.
Oravec, J. A. (2019). The "dark side" of academics? emerging issues in the gaming and manipulation of metrics in higher education. Review of Higher Education, 42(3), 859-877. http://doi.org/10.1353/rhe.2019.0022 DOI: https://doi.org/10.1353/rhe.2019.0022
Patterson, T. C. (2020). A social history of anthropology in the United States. Routledge. DOI: https://doi.org/10.4324/9781003084358
Porter, T. M. (2020 [1995]). Trust in numbers. Trust in numbers. Princeton University Press. DOI: https://doi.org/10.23943/princeton/9780691208411.001.0001
Price, D. H. (2016). Cold war anthropology: The CIA, the Pentagon, and the growth of dual use anthropology. Durham and London: Duke University Press. DOI: https://doi.org/10.1215/9780822374381
Raaper, R. (2016). Academic perceptions of higher education assessment processes in neoliberal academia. Critical Studies in Education, 57(2), 175-190. https://doi.org/10.1080/17508487.2015.1019901 DOI: https://doi.org/10.1080/17508487.2015.1019901
Roy, P., Wandschneider, E., & Steglitz, I. (2014). Assessing education abroad outcomes: A review of the BEVI, IDI, and GPI. White Paper. Michigan State University Office of Study Abroad.
Salisbury, M. H., An, B. P., & Pascarella, E. T. (2013). The effect of study abroad on intercultural competence among undergraduate college students. Journal of Student Affairs Research and Practice, 50(1), 1-20. https://doi.org/10.1515/jsarp-2013-0001 DOI: https://doi.org/10.1515/jsarp-2013-0001
Savicki, V., Brewer, E., & Whalen, B. (2015). Assessing study abroad: Theory, tools, and practice. Stylus Publishing, LLC.
Schenker, T. (2019). Fostering global competence through short-term study abroad. Frontiers: The Interdisciplinary Journal of Study Abroad, 31(2), 139-157. https://doi.org/10.36366/frontiers.v31i2.459 DOI: https://doi.org/10.36366/frontiers.v31i2.459
Shore, C. (2008). Audit culture and illiberal governance: Universities and the politics of accountability. Anthropological Theory, 8(3), 278-298. https://doi.org/10.1177/1463499608093815 DOI: https://doi.org/10.1177/1463499608093815
Shore, C. (2010). Beyond the multiversity: Neoliberalism and the rise of the schizophrenic university. Social Anthropology, 18(1), 15-29. https://doi.org/10.1111/j.1469-8676.2009.00094.x DOI: https://doi.org/10.1111/j.1469-8676.2009.00094.x
Shore, C., & Wright, S. (2000). Coercive accountability: The rise of audit culture in higher education. In M. Strathern (Ed.), Audit cultures (pp. 69-101). Routledge.
Slaughter, S. (2014). Academic capitalism in the age of globalization JHU Press.
Slaughter, S., & Leslie, L. L. (1997). Academic capitalism: Politics, policies, and the entrepreneurial university. Johns Hopkins University Press.
Slaughter, S., Slaughter, S. A., & Rhoades, G. (2004). Academic capitalism and the new economy: Markets, state, and higher education. Johns Hopkins University Press.
Spenader, A. J., & Retka, P. (2015). The role of pedagogical variables in intercultural development: A study of faculty-led programs. Frontiers: The Interdisciplinary Journal of Study Abroad, 25, 20-36. https://doi.org/10.36366/frontiers.v25i1.342 DOI: https://doi.org/10.36366/frontiers.v25i1.342
Spitzberg, B., & Chagnon, G. (2009). Conceptualizing intercultural communication competence. In D. Deardorff (Ed.), (pp. 2-52). Sage. DOI: https://doi.org/10.4135/9781071872987.n1
Stanny, C. J. (2018). Putting assessment into action: Evolving from a culture of assessment to a culture of improvement: Putting assessment into action. New Directions for Teaching and Learning, 2018(155), 113-116. https://doi.org/10.1002/tl.20310 DOI: https://doi.org/10.1002/tl.20310
Taylor, P., Parker, K., Fry, R., Cohn, D., Wang, W., Velasco, G., & Dockterman, D. (2015). Is college worth it? college presidents, public assess value, quality and mission of higher education. Pew Research Center. https://www.pewresearch.org/social-trends/2011/05/15/is-college-worth-it/
Tonkin, H., & Quiroga, D. (2004). A qualitative approach to the assessment of international service-learning. Frontiers: The Interdisciplinary Journal of Study Abroad, 10, 131-149. https://doi.org/10.36366/frontiers.v10i1.138 DOI: https://doi.org/10.36366/frontiers.v10i1.138
Tooker, D. E. (2012). Space and the production of cultural difference among the akha prior to globalization. Space and the production of cultural difference among the Akha prior to globalization. Amsterdam University Press. DOI: https://doi.org/10.1017/9789048514380
Vande Berg, M., Paige, R. M., & Lou, K. H. (2012). Student learning abroad: Paradigms and assumptions. In M. Vande Berg, R. M. Paige & K. H. Lou (Eds.), Student learning abroad: What our students are learning, what they're not, and what we can do about it (pp. 3-28). Stylus.
Wandschneider, E., Pysarchik, D. T., Sternberger, L. G., Ma, W., Acheson, K., Baltensperger, B., Good, R., Brubaker, B., Baldwin, T., Nishitani, H., Wang, F., Reisweber, J., & Hart, V. (2015). The Forum BEVI Project: Applications and Implications for International, Multicultural, and Transformative Learning. Frontiers: The Interdisciplinary Journal of Study Abroad, 25(1), 150–228. https://doi.org/10.36366/frontiers.v25i1.350 DOI: https://doi.org/10.36366/frontiers.v25i1.350
Watson, J. R., & Wolfel, R. L. (2015). The intersection of language and culture in study abroad: Assessment and analysis of study abroad outcomes. Frontiers: The Interdisciplinary Journal of Study Abroad, 25(1), 57-72. https://doi.org/10.36366/frontiers.v25i1.345 DOI: https://doi.org/10.36366/frontiers.v25i1.345
Weber, M. (2016 [1921]). The types of legitimate domination. Social theory re-wired (pp. 270-286). Routledge.
Wiley, A. (2016). Intercultural development inventory (IDI): Independent review. ACS Ventures, LLC.
Williams, C. (2006). The epistemology of cultural competence. Families in Society: The Journal of Contemporary Social Services, 87(2), 209-220. https://doi.org/10.1606/1044-3894.3514 DOI: https://doi.org/10.1606/1044-3894.3514
Williams, T. R. (2005). Exploring the impact of study abroad on students’ intercultural communication skills: Adaptability and sensitivity. Journal of Studies in International Education, 9(4), 356-371. https://doi.org/10.1177/1028315305277681 DOI: https://doi.org/10.1177/1028315305277681
Wright, S., & Shore, C. (2017). Death of the public university?: Uncertain futures for higher education in the knowledge economy. Berghahn Books. DOI: https://doi.org/10.2307/j.ctvw04bj2
Zemach-Bersin, T. (2010). Selling the world: Study abroad marketing and the privatization of global citizenship. The handbook of practice and research in study abroad (pp. 325-342) Routledge. DOI: https://doi.org/10.4324/9780203876640-27
ZoomInfo Technologies. (2022). Who is intercultural development inventory. Retrieved from https://www.zoominfo.com/
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Gareth Barkin, Lauren Collin
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
