Tools - New Traditions: Student Interview Protocol

John Moore
Department of Chemistry
University of Wisconsin-Madison
1101 University Ave.
Madison WI 53706
email: jwmoore@chem.wisc.edu
(608) 262-5154

Susan Millar
LEAD Center
University of Wisconsin-Madison
1402 University Ave.
Madison, WI 53706
email: smillar@engr.wisc.edu
(608) 265-5943

Debby Penberthy
LEAD Center
University of Wisconsin-Madison
1402 University Ave.
Madison, WI 53706
email: penberth@engr.wisc.edu
(608) 265-5927

Introduction:
In the middle of the Spring semester 1997, LEAD researchers interviewed a sample of 14 students from the UW-Madison's Topic-oriented Approach to Chemistry 104 (2^nd semester general chemistry) using an open-ended interview protocol designed to elicit information on their experiences in the course overall and with particular course elements. Students were identified using a "quasi" random process: that is, student names were generated using a random number to produce a list of students to contact. However, the actual list of interviewees was modified somewhat to ensure that the resulting group was similar to the make-up of the entire class and was comprised of at least one student of each of the eight TAs. The proportions of those interviewed were roughly similar (given the interview sample size of 14) to that of the entire lecture, with respect to gender and ethnic categories, incoming ability indicators, and academic rank.

Conducting the interviews
The LEAD Center interview protocols were created on the basis of specific research questions. These research questions were often designed so as to determine the degree to which the course professor's learning goals were achieved and to examine the nature of the students' learning process. Thus, if a potential user of these instruments has different student learning goals, some of the interview questions may be less appropriate than others.

All of the LEAD Center interview protocols included here are open-ended. In contrast to most surveys, open-ended interview protocols are designed to allow the respondent to speak freely about their experiences without being limited by predetermined response categories. Potential users of these instruments should keep in mind that the questions served as a guide to a "conversation" with the interview respondent; the questions were not asked verbatim or in the same order for each interview, as the interviewee was allowed to guide the conversation to some degree.

Interviews were conducted by LEAD Center researchers. In all cases these researchers were considered "third party" in that they were not members of the academic departments through which the course or program under evaluation was taught. Thus, the identity of student interviewees was not revealed to the course instructors, and students were informed that this was the case before agreeing to participate. This was important because it allowed the student interviewees to feel that their opinions would not affect their relations with their course instructors. In cases where no third party evaluator is available or desirable, we recommend that the interviews be conducted by someone who has no control over the students' grades. One possibility is for two professors to interview each others' students.

Analyzing the interviews
Sample results based on the use of LEAD Center interview protocols were obtained by analysis of verbatim transcripts of the audio-taped interviews. Although it is not necessary to transcribe the interviews, this does allow for more in-depth analysis of subtle themes. It is possible for the interviewer simply to take notes during the interviews and base their analysis on these notes.

The LEAD Center analysis process consists of developing inductive, thematic codes for the students' experiences. Generally, the researchers first read all of the transcripts. Then, they develop a rough coding scheme based on themes which appear in multiple interviews. Using this scheme, each individual transcript is coded. Then, interview excerpts are cut and pasted into an analysis document so that statements which are alike can be examined side by side. (Alternatively, a qualitative research software package may be used to sort and organize interview themes.) The coding scheme is gradually refined so that it represents a listing of the umbrella themes that capture the range of experiences discussed in the interviews. The analysis process is quantitative only in the sense that rough counts of the number of interviewees who spoke about a particular type of experience are developed. It differs from the traditional definition of quantitative analysis in that it does not tell one how many of the participants had a particular experience, but only how many chose to talk about it in the interview. Again, the interviewees are engaged in conversation, and therefore, not all issues are touched upon in each interview. Thus, though a small number of students may discuss an issue, the issue might be important in the experiences of a larger number or percentage of students.