© Please note: Mark Thogmartin owns the copyright for the dissertation below. You may download and print one copy for educational purposes only. These pages may not be duplicated, distributed, redistributed or republished in any manner without express permission from the author.
Introduction
The methodology used in any study is necessarily dependent
on the purpose of the study. Patton (1990) states: "Purpose
is the controlling force in research. Decisions about design,
measurement, analysis, and reporting all flow from purpose. Therefore,
the first step in a research process is getting clear about purpose"
(p. 150).
In discussing the marriage of purpose and process (methodology),
Wolcott (1992) comments that
research purpose is the only basis on which decisions about process
can be made; the clearer the purpose, the clearer the ways to
achieve it. . . . Stripped of eloquence that can be added later,
this key feature of qualitative (and any other) research begins
with the phrase, "The purpose of this study is . . . "
The fewer the words needed, the better; wordiness is a dead giveaway
to an ill-formed, or at least not-yet-formed idea. (p. 7)
Therefore, the type of research used in a study is primarily dependent
on the purpose of the study or what the researcher hopes to achieve.
Patton discusses five basic types of research. These are+
1. basic research to contribute to fundamental knowledge and theory
2. applied research to illumine a societal concern
3. summative evaluation to determine program effectiveness
4. formative evaluation to improve a program
5. action research to solve a problem. (Patton, 1990, p. 150)
The purpose of this study is to provide feedback to those responsible
for the coordination of the Seventh-day Adventist North American
Division Tutoring and Mentoring effort that is a response to President
Clinton's America Reads initiative. Through this formative evaluation,
specific recommendations are given which, I hope, will help in
the ongoing development of this and similar programs. More generally,
the compilation of references from the existing literature about
subjects related to the effort will be valuable to those interested
in providing literacy tutoring help to young readers.
Because formative studies aim to improve a specific program, the
findings or recommendations may or may not be applicable to other
situations. "The purpose is to improve human intervention
within a specific set of activities at a specific time for a specific
group of people" (Patton, 1990, p. 156). Qualitative methods
are particularly suited for this kind of study since the focus
is limited. Subject numbers are usually small, with case studies
being a preferred mode of investigation.
Features of Qualitative Research Generally and in This Study
Eisner's Characteristics of Qualitative Studies
In defining the features of qualitative modes of inquiry, I use a six-point description offered by Eisner (1991). As I offer his descriptions, I apply them to this study specifically and explain why they are appropriate given the purpose offered above.
Field Focused
First, qualitative studies are field focused. By this term,
Eisner implies that the researcher goes to the people and places
where the events of interest to the researcher are happening.
This also implies that qualitative studies are nonmanipulative,
that is, objects and situations are studied intact instead of
in an artificial, experimental environment.
In this study, I was in the field in two different capacities.
The initial Tutoring and Mentoring training event was held in
late August 1997 at the North American Division Headquarters in
Silver Spring, Maryland. I attended this conference in a dual-role
capacity as both a co-presenter and as a participant observer
with a research goal in mind.
Participant observation is an important tool for the qualitative
researcher. The challenge for the participant observer is "to
seek the essence of the life of the observed, to sum up, to find
a central unifying principle" (Bruyn, 1966, p. 316). My participation,
as mentioned before, was in a role different from those being
trained since I was also acting as a trainer. But I was still
able to fulfill the purpose of participant observation, which
is "to describe the setting that was observed, the activities
that took place in that setting, the people who participated in
those activities, and the meanings of what was observed from the
perspective of those observed" (Patton, 1990, p. 202). These
observations were documented with field notes, which were added
to after I reviewed the entire training event by videotape (provided
to me by one of the trainees). My time with the trainees was brief,
as was their training experience. Therefore, the "depth"
of the encounter was limited because of time factors, but it did
parallel the experience of the participants. This is the essence
of participant observation.
Another field-based qualitative method used to gather data is
by performing case studies of participants who are of particular
interest to the researcher. My choices of follow-up subjects were
based on a number of factors including their level of implementation
of the program, scheduling concerns, and their particular backgrounds.
Merriam (1988) identifies four essential characteristics of case
study research which may help to define it in this context:
Particularistic. Case studies focus on a particular phenomenon,
situation, event, or program.
Descriptive. The end product of a case study is a rich "thick" description of the phenomenon under study.
Heuristic. Case studies illuminate the reader's understanding of the phenomenon under study.
Inductive. Case studies rely on inductive reasoning in that the generalizations made as a result of studying the phenomenon emerge from the examination of the data collected in context. (pp. 11-12)
Like Merriam, I view these characteristics not as "types"
of case study research, but as threads that weave in and out of
each case that I investigated. I discuss why I chose to study
the particular individuals that I did later in this chapter.
Interviews with the case study participants were tape recorded.
I used an interview schedule not as a script, but as a general
guide (Appendix A). Knapp (1997) makes a case for this type of
open-ended interviewing, stating that having a "shared agenda"
is not only the most ethical way to interview people, but also
the most effective since it allows for serendipitous responses
to questions not thought of by the interviewer. The interview
guide I used was based on the framework of critical components
described in chapter 2. I took some field notes during my on-site
visits, but interviews provided the bulk of information gleaned
from case study participants.
The Self as Instrument
A second characteristic of qualitative inquiry as defined by Eisner
(1991) relates to the self as instrument. Eisner states that the
important features in any study do not simply announce themselves.
"Researchers must see what is to be seen, given some frame
of reference and some set of intentions. The self is the instrument
that engages the situation and makes sense of it" (pp. 33-34).
He cautions that this "appreciation for personal insight
as a source of meaning does not provide a license for freedom"
(p. 35). Evidence and reasons for interpreting things the way
they do must be provided by researchers. Peshkin (1988) comments
that subjectivity is inevitable in any study, whether qualitative
or quantitative, and that researchers should consciously seek
it out during their entire study. Ideally they should let readers
know of their own biases while they seek to control it. This will
help to keep their work from becoming "autobiographical."
Whenever possible in this study, I let the participants speak
for themselves. Yet my interpretation of their comments was based
in the framework of critical components of effective tutoring
programs that I assembled from the literature. Borman, LeCompte,
and Goetz (1986) suggest that this combination of description
with established frameworks derived from the literature can prevent
qualitative studies from producing conclusions that some might
call "trivial." They state:
In our opinion, simple, flat description that does not create
linkages with substantial conceptual and theoretical literature
is not good ethnography; neither is research that does not examine
sociohistorical context for explanations of what is going on.
The recursive nature of analysis in qualitative and ethnographic
research is designed to help the investigator build constructs
and integrate them with existing results from the research literature,
to create linkages among the classes of phenomena observed in
the research site, to build constructs and integrate them with
the existing research literature, and to generate explanations
for what has been found. (p. 49)
I had the liberty to act as an editor of the comments made
by participants, choosing what to include or not include. How
can the reader be assured that my renderings of their worlds are
accurate? I accomplished this in two important ways. The first
is that I solicited feedback from the primary case study informants.
Miles and Huberman (1984a) contend that "a good explanation
deserves attention from the people whose behavior it is about.
Getting feedback from the informants . . . has particular confirmatory
power" (p. 28). After completing my analysis of their tutoring
programs, I sent an explanation of my critical components matrix
to my primary case study informants along with the material I
had written based on my interviews and visits with them. Their
feedback was critical in shaping the final draft of my report.
Another way that my conclusions were verified was through the
use of multiple data sources. This data triangulation (Denzin,
1978) helps to strengthen the analysis presented at the end of
the study. My data were collected through interviews, observation,
surveys, and journals from a number of participants over a 9-month
period. The findings that I present in this study were not considered
unless they could be confirmed by at least two different sources.
I used two open-ended surveys during the study. The first was
simply a large, blank index card where the subjects were asked
to give written responses to several different questions. The
data from these cards are analyzed and reported in chapter 4.
The second instrument was somewhat more formal. Obviously, the
participants had/have backgrounds that vary markedly from each
other. Many have had little or no experience as instructors in
any kind of teaching-learning situation, so they came to this
situation with various levels of concern about what they have
volunteered to do. Fuller (1969) hypothesized that the concerns
teachers have progress through several stages as they are learning
new innovations. Initially, their concerns are with "self"
issues: "How will this affect me?" The next level is
concerned with the "task," or "What is this innovation
all about?" The final level of concern has to do with "impact,"
or "How is this affecting my pupils?" A team from the
Research and Development Center for Teacher Education (Hall, George,
& Rutherford, 1977) expanded upon Fuller's work by breaking
down the concerns into seven different stages: awareness, informational,
personal, management, consequence, collaboration, and refocusing.
They developed the Stages of Concern (SoC) Questionnaire to measure
the continuum of attitudes toward an innovation that evolve in
a fairly natural sequence. With this tool, I analyzed the concerns
of these volunteers as they were completing their initial training
session (Appendix A). This instrument is described more fully
in chapter 4.
As a final source of data I kept a journal of my telephone and
E-mail conversations that I had with the participants. Because
of the difficulty in recording phone conversations, I took notes
during my talks and I wrote as much as I could recall immediately
as we were finished. I also wrote my own thoughts about my continuing
work in this journal. This helped me to formulate my ideas as
the study progressed.
Interpretive Character
The interpretive character of qualitative research is the third
feature mentioned by Eisner (1991). The term interpretive has
two meanings in the context of qualitative inquiry. The first
pertains to questions of why. Why do people respond the ways they
do in given situations? Why does an approach work in one location
and not in another? To answer these questions the researcher either
has to use constructs from the social sciences or he/she must
create new theory for consideration.
Interpretation also has to do with matters of meaning. This is
related to the why questions discussed above, but it goes deeper
into areas of motive and experience. Rather than simply giving
an account of the more obvious reasons for peoples' actions, interpretation
in this context "penetrates the surface. Qualitative inquirers
. . . aim beneath manifest behavior to the meaning events have
for those who experience them" (Eisner, 1991, p. 35).
Perhaps Eisner's two questions above are a slightly more poetic
way of describing what other qualitative theorists might call
hypothesis development (Glaser & Strauss, 1967) or conclusion
drawing (Miles & Huberman, 1984a). Experimental studies begin
with a hypothesis which is either verified or not, while nonexperimental
studies may generate hypotheses from data that have been systematically
gathered.
Most often in this study, I let the participants speak for themselves
while offering just enough interpretation to set their comments
within a context. But, especially when discussing my findings
in the final chapters, I delve into the shared experiences and
the meanings behind the actions (or absence of action) of the
participants. I explain the reasons, based on their own accounts
and my own observations, for their eventual ability or inability
to begin tutoring projects, citing evidence from my records.
The Use of Expressive Language
The use of expressive language is the fourth characteristic of
qualitative research discussed by Eisner (1991). The detached,
mechanistic voice found in many journals is not usually a feature
of qualitative studies. Eisner calls the use of first-person singular
and the utilization of metaphor and descriptive, even poetic,
language the "signature" of the qualitative researcher.
He asks, "Why take the heart out of the situations we are
trying to help readers understand?" (p. 37)
Miles and Huberman (1984a) suggest that making metaphors has the
capability of pulling great quantities of fragmented data together
into a one single descriptive image. Yet they confess that their
general style of reporting does lean toward the "realist/positivist
side" describing themselves as "right-wing qualitative
researchers" (p. 23). This illustrates that there are wide
variations even among qualitative theorists as to the personal
poetic flair they give their work. Tesch (1990) presents a continuum
of qualitative research interests with science-like methods on
one end of the spectrum and art-like methods on the other.
I appreciate and take the liberty to use descriptive language
and first-person singular voice. I agree with Eisner when he asserts
that the detached voice used in many reports can be a deceptive
way to imply detachment and objectivity. Yet my style does tend
to be less metaphoric than others. I try to tell it like it is,
and I use poetic language when it seems useful.
Attention to Particulars
A fifth feature of qualitative studies, according to Eisner (1991),
is their attention to particulars. The "flavor" of the
particular situation is valued at least as much as aggregated
data from multiple subjects. This flavor is maintained "by
sensitivity to what might be called the aesthetic features of
the case" and an "awareness of its distinctiveness"
(p. 38).
Case study research, which I use in this study, allows for discovery
of these distinctive and aesthetic features. Through interviewing
and observation the unique characteristics of each case can be
discovered and mined, uncovering a wealth of information valuable
to the researcher and reader. Merriam (1988) agrees:
Investigators use a case study design in order to gain an in-depth
understanding of the situation and its meaning for those involved.
The interest is in process rather than outcomes, in context rather
than a specific variable, in discovery rather than a confirmation.
(p. xii)
The cases I investigate are drawn from a larger group of people
who were the first trainees in a particular tutoring and mentoring
program. More people have and will become involved in the effort,
but I decided to limit my focus to this initial cohort, choosing
to tell their stories in an in-depth fashion rather than extending
to the larger group.
Coherence, Insight, and Instrumental Utility
Eisner's sixth and final characteristic of qualitative studies
"pertains to the criteria for judging their success. Qualitative
research becomes believable because of its "coherence, insight,
and instrumental utility" (1991, p. 39). By these terms,
Eisner means that qualitative studies use persuasion rather than
statistical "proofs" of cause and effect or association.
He points to the field of law as a model for such work. Lawyers
utilize the theaters of courtrooms to convince juries of the plausibility
of their case. So it is with qualitative work. It is the "weight"
of the evidence, the coherence of the facts presented, and the
cogency of the interpretation that convinces the reader of the
explanations given.
The purpose of this study is primarily descriptive as opposed
to experimental. "The aim of descriptive research is to examine
events or phenomena" (Merriam, 1988, p. 7). This report is
intended to enlighten those who are intricately involved in the
program under study. It is my goal to highlight the strengths
and weaknesses of the approach undertaken by the organizers of
the Adventist Tutoring and Mentoring initiative. My need to convince,
to be plausible in my explanations, and to provide insight is
for the purpose of providing some amount of guidance to the decision-makers
in this effort.
Perhaps a comment should be made regarding what Eisner called
instrumental utility. I see this as a measure of the study's usefulness
to not only the target group, but also to those outside of the
study's immediate context. Eisner (1991), in defining instrumental
utility as a study's usefulness, says that it can be useful in
two primary ways. It can help us to comprehend a situation that
would otherwise be confusingthe usefulness of comprehension.
A study can also help in anticipating the futurethe usefulness
of anticipation. This anticipation can be provided by studies
that figuratively act as predictors, maps, or guides.
Design of the Study
Data Collection
Phases of Data Collection
The data for this study were collected in four general phases.
I began collecting data during the first training session held
in Silver Spring, Maryland, during the final week of August 1997.
The SMILIES portion of the training took place during the last
2 days of the session. During this time Dr. Shirley Freed led
the training and I assisted her. We solicited several written
responses from the participants that were open-ended answers to
questions having to do with specific parts of the training. I
administered the open-ended Stages of Concern (SoC) survey (Hall
et al., 1977) during the final hour of the training (Appendix
A). I also took observation field notes during the training event,
and I spent some time informally getting to know the participants.
Chapter 4 is an analysis of this SMILIES training session.
In the second phase of data collection from September to December
1997, I took notes as I talked with many of the participants by
telephone to assess their progress in establishing tutoring sites
in their respective locations. It became obvious during this time
that most were experiencing some degree of frustration or hesitancy,
even though they still maintained a general enthusiasm for the
initiative. It was also during this time that I learned of the
successful establishment of projects in Boston and the Los Angeles
area by two of the participants. I became interested in their
stories and determined to make on-site visits to them in the future.
The third phase of data collection from January through March
1998 involved my visits, observations, and interviews with these
two participantsArnold in Los Angeles and Nydia in Bostonand
with some of their trainees. These visits took place in February
and March, 1998. In each location I tape-recorded my interviews
and collected documents related to their efforts. Also, at the
end of January, project coordinator Sandra Brown held a teleconference
with several of the trainees from the August 1997 training session.
I took part in this conference and took notes as it progressed.
In the final phase of data collection I contacted as many of the
initial trainees by telephone as possible for a final interview,
during which I took notes. We discussed what progress, if any,
had been made in their implementation efforts, and I gave them
an opportunity to express whatever frustrations or hopes that
they had related to the initiative.
The schedule of data collection (see Table 3) will help the reader
to develop a sense of the timing and nature of the contact between
the participants and me. I did attempt to contact each of the
participants at least four or five times, and I had difficulty
connecting with a few. There were three "couples" at
the trainingtwo husband and wife teams and one mother and
daughter team. In each couple, the wife or mother acted as the
spokesperson for the pair. All names of participants on the schedule
and in this report, with the exceptions of my two case study subjects,
are pseudonyms.
TABLE 3
SCHEDULE OF DATA COLLECTION
TABLE 3
Name |
Phase 1 August 1997 |
Phase 2 Sept. - Dec. 1997 |
Phase 3 Jan. - March |
Phase 4 April - May 1998 |
Monica | O Q Q | T | T | |
Irene | O Q Q | T | ||
Margie | O Q | |||
Susan | O Q Q Telephone | T T | T | |
Thomas | O Q | |||
Connie | O Q Q Telephone | T | C | T |
Paula | O Q Q Telephone | |||
Dawn | O Q Q Telephone | T | T | |
Berta | O Q Q Telephone | |||
Lee | O Q Q Telephone | T | ||
Carla | O Q Q Telephone | T | T | |
Amy | O Q Q Telephone | T | ||
Terry | O Q Q Telephone | C | T | |
Sherry | O Q Q Telephone | T | C | T |
Howard | O Q | |||
Nydia | O Q | E T E E E | E E E V | E E T |
Tasha | O Q Q Telephone | C | T | |
Ruth | O Q Q Telephone | T | C | T |
Rose | O Q Q Telephone | T | T | |
Patty | O Q | |||
Edward | O Q Q Telephone | T | T | |
Arnold | O Q Q Telephone | T T T | C E V | E E |
Ella | O Q Q Telephone | T | T | |
June | O Q Q Telephone | |||
Lindsey | O Q Q | T | T | |
Note.
C = Teleconference; E = E-mail; O = Observation; Q = Questionnaire; V = Site Visit;
T = Telephone Conversation.
Selection of Cases
As mentioned above, two participants, Arnold Trujillo and Nydia,
Mendez (they requested that I use their actual names), were immediately
successful in initiating tutoring projects. Their cases represent
very different scenariosArnold is a full-time church leader
working with volunteers of college age or younger. The tutoring
projects under his supervision are operating from churches. Tutoring
takes place outside of the school day. Nydia, on the other hand,
is a principal at a public elementary school in Boston. Her volunteers
are mothers who tutor at the school during the regular school
day in full cooperation with the children's teachers. I decided
that these two cases warranted special attention because of the
immediate success of the overseers and because of their unique
settings and operating parameters. No other projects initiated
by the August trainees were in operation during the 1997-1998
school year.
Chapter 7 contains a discussion of the other participants and
their situations synthesized from data I collected in all four
phases. Certainly, each trainee has a unique story to tell as
to why he or she experienced frustration or paralysis in trying
to establish a program. Rather than simply categorizing their
frustrations, I have chosen to use the more "art-like"
(Tesch, 1990) form of depiction known as the collective story
(Richardson, 1990). "The collective story displays an individual's
story by narrativizing the experiences of the social category
to which the individual belongs, rather than by telling the particular
individual's story " (p. 25). The social category, in this
case, is composed of the trainees who, for various reasons, were
unable or unwilling to establish a tutoring site in their home
locations. Thus, this chapter tells composite stories that are
based on telephone interviews I had with these people.
Denzin (1994) boldly asserts that "in the social sciences,
there is only interpretation. Nothing speaks for itself"
(p. 500). He classifies research writing as either "productive"
or "expressive" (pp. 504-505). My writing previous to
chapter 7 is more "productive" in that I seek to present
the objective realities experienced by the participants. Using
the classic qualitative tools of observation and interview, my
findings are categorized and classified, usually according to
preexisting schemes. But in chapter 7, I choose to use the "expressive"narrative
research tool of telling collective stories because, in Denison's
(1996) words,
stories show instead of tell; they are less author-centered; they
allow the reader to interpret and make meaning, thus recognizing
that the text has no universal or general claim to authority;
and, they effectively communicate what has been learned. (p. 352)
Rinehart (1998) uses different terms to describe types of qualitative/ethnographic
writing. He defines "academic ethnography" as the now
traditional sort of qualitative work that attempts to "capture
the experience so that readers may more nearly approach the primary
experience" (p. 203). This parallels my "productive"
writing explained above. Collective stories fit under Rinehart's
"fictional ethnography" where "the fundamental
thrust . . . is to get at both the affective feel of the experience
and the cognitive truth of it" (p. 204).
My goal, in doing this case study research, it to communicate
the lived experiences of the participants to the reader as effectively
as possible. It is hoped that the successes and frustrations of
the first trainees may prove to be instructive and enlightening
to the project organizers and to all those interested in the successful
implementation of tutoring programs.
The Data File
I built a data file containing all pertinent information gathered
during all four phases of data collection. This file includes
observation field notes, completed questionnaires, interview transcriptions,
notes taken during telephone conversations, E-mail correspondences,
and journal reflections. It also includes some printed material
supplied to me by the case study informants and by the project
coordinatorvision statements, resumes, flyers, and other
correspondence. After I compiled all the data into the file, I
organized it into four parts:
1. Questionnaires and other material from the August, 1997 Tutoring
and Mentoring training session, including information supplied
to me by the project coordinators.
2. Transcriptions, notes, and other material relating to Arnold's
case study.
3. Transcriptions, notes, and other material relating to Nydia's
case study.
4. Telephone conversation notes taken while talking with the other
participants, including matrices and other classification tools
that I developed to aid me in my analysis of the data.
Organizing the data in this way not only makes sense categorically,
it also results in a somewhat chronologically sequenced file.
When referencing data file information, I make the citation (Data
File, p. #) in parentheses.
Data Analysis
The case study data were interpreted using a conceptual framework
(Miles & Huberman, 1984a) suggested by the literature (Borman
et al., 1986) pertaining to successful literacy tutoring programs.
As I read and reread the transcriptions from interviews, I coded
the data according to the framework I developed, which is explained
in chapter 2. Glaser and Strauss (1967) warned that using borrowed
classification schemes may hinder the generation of new categories.
Keeping this in mind, I also allowed new categories to emerge
as I read. As I carried the framework into the next case, I again
coded the data, continuing to watch for the emergence of potential
new categories. This "speculation," according to Merriam
(1988), is "the key to developing theory in a qualitative
study" (p. 141). Merriam cites Goetz and LeCompte (1984),
who say that speculation involves
playing with ideas probabilistically. It permits the investigator
to go beyond the data and make guesses about what will happen
in the future, based on what has been learned in the past about
constructs and linkages among them and on comparisons between
that knowledge and what is presently known about the same phenomena.
These guesses are projections about how confidently the relationships
found or explanations developed can be expected to obtain in the
future. (p. 173)
This is exactly the process I used in making "speculations"
about the factors that combine to promote or frustrate the development
of literacy tutoring programs. I also evaluate the status of the
existing tutoring projects using a similar set of factors.
Yet, if this study is to have any measure of instrumental utility
(Eisner, 1991), I must present my "speculations" in
a manner that is cohesive and understandable. My evaluations of
existing programs are made in the light of the framework of critical
components I developed as I compiled information from the literature.
This framework is straightforward and almost speaks for itself.
When telling the collective stories of those who experienced frustration
in establishing tutoring sites, I chose to organize these experiences
according to themes and types that emerged as I studied the data
and people involved. Eisner (1991) defines themes as "recurring
messages construed from the events observed" (p. 189). The
utilization of themes in qualitative data analysis is a classic
strategy described by Miles and Huberman (1984b) as one that is
somewhat instinctive to humans. "The human mind finds patterns
so quickly and easily that it needs no how-to advice. Patterns
just happen,' almost too quickly" (p. 216).
Summary
The method used in this study is a qualitative case study approach
utilizing the classic tools of observation, questionnaires, and
interviews, and the narrative tool of collective storytelling.
Participants in a tutoring and mentoring initiative were followed
for a school year as they experienced varying degrees of success
in establishing tutoring projects. The findings in this study
were verified through triangulation and member checks. The end
result is a description of what factors contributed to implementation
or non-implementation of tutoring projects by the trainees.