Methodology & Sample
The researchers used a mixed-methods approach. They collected a wealth of both qualitative and quantitative data from UWC stakeholders across surveys, interviews, and site visit observations.
It should be noted that the study's data collection procedures took place from 2018-2021. Accordingly, the study was affected by the COVID-19 pandemic, including cancellation of four anticipated campus visits (14 completed prior to March 2020) and adjustments to methods (e.g. remote video observations at 2 schools in place of in-person observations).
Surveys
The researchers designed a set of online surveys for UWC students and alumni to complete. This captured longitudinal data from students that could be used to generate growth models of change over time:
- 2020 graduating students completed the survey four times during their enrollment in the IBDP. The majority of the student data contained in this report pertains to the 2020 graduating cohort.
- 2021 graduating students completed the survey three times during their enrollment in the IBDP.
In order to capture cross-sectional data from the alumni pool, all alumni across the UWC movement were invited to complete a primary survey. A smaller number of alumni were also offered two additional optional survey formats:
- An open-ended questionnaire with in-depth questions regarding an intercultural encounter and its effects.
- A journey mapping activity involving the creation of a visual representation of the effect of UWC on their life paths.
A sample of 13 non-UWC schools was recruited to take part in the study. They were asked to complete personalised versions of the student and alumni surveys as a condition of participation in the research.
The data gathered from surveys represented a wealth of both quantitative and qualitative information about the UWC experience and its potential effects.
Interviews
The researchers conducted semi-structured interviews with UWC students, alumni, and staff.
- About 15-20 UWC students were chosen at random from each school's 2020 graduating cohort. A portion of these students were interviewed remotely; the remainder were interviewed in-person on school site visits.
- Several hundred alumni were invited to take part in an interview from those who had volunteered via the primary survey. A purposeful sampling method was used in order to ensure representation of schools and generational cohorts. The vast majority were interviewed remotely.
- A targeted sample of staff members were chosen at each school. This sample included teachers from a range of subject areas, high-level administrators, and individuals in key student-facing roles, such as pastoral care, university guidance, and student life staff. Most of these interviews were conducted during site visits to school campuses.
Site Visits and Observations
The researchers visited 14 of the 18 UWC schools and colleges over the course of this study for intensive site visits. At least two members of the research team visited these campuses over the course of the 2018-2019 and 2019-2020 academic years. Campus visits lasted between 5-7 days each.
Interviews and Observations
- 720 semi-structured interviews (297 students, 209 alumni, 214 staff)
- 286 environments, activities and events observations
Interviews and observations were not conducted at participating non-UWC schools.
Sample
Alumni Surveys
- 6,894 alumni surveys
- 116 intercultural encounters
- 37 journey maps
More recent UWC alumni were overrepresented in survey responses, with over 40% of participants having graduated from their school since 2010.
Figures indicate the number of usable surveys as opposed to overall participation (i.e., participants who did not provide consent or provided largely incomplete survey submissions, for example, were dropped from the data).
Student Surveys
- UWC schools:
- 2,168 - 2020 graduates
- 2,266 - 2021 graduates
Non-UWC schools:
- 948 - 2020 graduates
- 882 - 2021 graduates
Measures and Methodology
Quantitative
Student and alumni surveys included a variety of quantitative question types, with a focus on validated psychometric scale items. Scales assessed attitudes and behaviours including open-mindedness, ethical reasoning, diversity and multiculturalism, social justice, and empathy.
Additionally, quantitative questions focused on the following topics:
- School experiences, including classroom climate, sources of stress, and sense of autonomy
- Social impact and effects on the United Nations' Sustainable Development Goals
- Extensive demographic information
Field observation protocols completed during observations were also designed to capture quantitative information regarding the presence or absence of certain elements of classroom climate (e.g., “mention of assessments”), teacher pedagogy (e.g., “teacher asks students what they want to learn”), and student behaviour (e.g., “students ask questions”).
Multiple forms of quantitative analysis were used to investigate results, including structural equation path modelling, multilevel modelling, latent cluster and profile analyses, and latent growth curve modelling. To ensure validity and reliability of results, we also performed confirmatory factor analysis (CFA), exploratory factor analysis (EFA), and measurement invariance testing across groups and time.
Qualitative
Qualitative data were gathered through a number of open-ended survey questions, through in-depth Interviews, and via observations. All interviews were transcribed and thematically coded in an iterative process that ensured reliability.
Surveys also included qualitative question types (e.g. free response questions about social impact and meaningful activities). Data generated from these questions were analysed using coding schemes based on emergent themes; data outputs from these schemes were then quantitatively analysed. During observations, detailed notes were recorded. Field notes enabled the research team to capture detailed examples of practice.
For more detailed information on the methodology and sample of the study, read pages 7-16 of the report Educational Experiences and Outcomes at the United World Colleges (UWC): An Investigation of Impact.