Today I am flying to Indonesia to conduct interviews of staff in a company in Sukabumi, in the highlands of West Java. The Asian Productivity Organization provided them with training in 5S and kaizen a few years ago. My aim is to discover and evaluate the impacts of the training through interviews. After a couple of days I will fly to Thailand and do the same in Phitsanulok in Northern Thailand near the Laotian border.
As a knowledge-pull activity, the interview is the art and science of asking questions. The most unrepentant knowledge-pushers are professor-lecturers-cum-bloggers (like me!) and the best cure for them is to do interviews (ouch!).
From a previous blog post (“L14- Voicing”), the unproductive manners of voicing are: lack of respect, being judgmental and acting as if you know better, and talking more than listening. The interviewer is forced to unlearn these unproductive habits of voicing. The interviewer is also forced to learn the art of asking questions and interviewing.
What are some productive ways of interviewing?
- The presumptions of the interviewer hide underneath every question he asks — presumptions which may not always be shared by the respondent. Hence the first duty of the interviewer is to reduce his presumptions to a bare minimum. This is the reason why it is important to start with open-ended questions, or “grand tour” questions (see Step 1 in the diagram below), before moving on to more structured questions.
- A good open-ended question is the invitation to storytelling (e.g. “Tell us about it/them” in Steps 1-3 in the diagram below) which is a good device to elicit narration of experiences from a respondent in her own terms and using her own assumptions to interpret her experiences. The interviewer just listens, instead of asking questions which are often loaded — knowingly or unknowingly — by his own assumptions. The story teller’s experiences and her interpretation of those experiences reveal her mental models (see “12 Types of Learning”).
- In general, the cultural, technical and other assumptions of the interviewer are not the same as those of the respondent. And so the duty of the interviewer is to discover the assumptions of the respondent. He can then develop his next questions “on the spot” following the assumptions, categories and mental models/structures of the respondent — not his own. This is the reason why many anthropologists and ethnographers shy away from structured questionnaires: these research tools insinuate the researcher’s own assumptions onto the respondents’ answers. The casualties are scientific validity and cultural fairness.
- Interviews are expensive in terms of time, expertise and money. Open-ended interviews cannot generate the kind of data needed to make useful generalizations. In other words, open-ended interviews are good for hypothesis building, but structured questionnaires are better for hypothesis testing. If questionnaires are designed AFTER open-ended interviews and ACCORDING to mental models/structures of the respondent population, then structured questionnaires offer opportunities for generalizations that open-ended methods alone cannot offer. The two are complementary, but a structured questionnaire/survey should be a LATTER step (see Step 6 in the diagram below) in the research process.
The same principle applies in designing a questionnaire: more open-ended questions should precede the more structured questions. For some research purposes, it is not advisable to tell the respondent what is the purpose of a questionnaire until after she is done answering. A good technique for avoiding or postponing some “smart” respondents becoming “test-wise” (i.e. they guess the purpose of the questionnaire and that guess consciously influences their answers) is to start with more open-ended questions.
Structure is “a fact of research life” because having research objectives means adopting a research structure. In this particular interview, my aim is impact evaluation of projects. The table above is oriented towards that aim. For my field interviews in five Asian countries, the sponsor and user of the impact evaluation is the Asian Productivity Organization or APO.
The range of concerns in impact evaluations typically includes (starting from the most immediate): outputs of a project, results on the larger organizational context of the project, side effects, and broader social outcomes. Attribution is easiest to claim on outputs and most difficult on social outcomes. Because interviews are at the individual level and social outcomes are at the macro level, interview is not a suitable tool for delineating social outcomes. The interviewer can still ask outcome questions, but he must remember that the answers he will get are highly interpretive and contextual on the part of the respondent. Hence, the correct sequence in asking such questions in an interview is to start with concrete and immediate outputs, and then move towards the more macro questions and their interpretive and contextual answers (Steps 2 and 3, followed by Steps 4 and 5 in the diagram below).
I will appreciate comments and improvements from other interviewers, or other impact evaluators (I am practicing knowledge-pulling!).
Note that there are embedded links in this blog post. They show up as colored text. While pressing “Ctrl” click on any link to create a new tab to reach the webpages pointed to.
Tags: 5S, Asian Productivity Organization, asking questions, generalizability, hypothesis building, hypothesis testing, impact evaluation, interviewing, kaizen, knowledge management, knowledge pull, knowledge push, manners of voicing, open-ended question, output, side effect, social outcome, storytelling, structured survey