Carnegie Mellon RAIRE Award
Results and Discussion from the Schommer Epistemological Survey
Jennifer White, Senior, Psychology
As part of our evaluation efforts, we also sought to measure how, if
at all, the experience of research changed students' mental models
for thinking about learning and the nature of knowledge. For example,
by the end of the research experience would students typically alter
their attitudes toward the role of failure and uncertainty in the
learning process? We predicted that students' epistemological
frameworks would, in fact, become more sophisticated (that is more
willing to fail, less certain of absolute truths, more independent,
etc.) as a result of incorporating research into learning.
We used the Schommer Epistemological Survey, developed by Marlene
Schommer of Wichita State University, to measure students' beliefs
about learning and knowledge. This survey breaks epistemological
beliefs into the following categories:
- Seek single answers
- Avoid integration
- Avoid ambiguity
- Knowledge is certain
- Depend on authority
- Don't criticize authority
- Ability to learn is innate
- Can't learn how to learn
- Success in unrelated to hard work
- Learn the first time
- Learning is quick
- Concentrated effort is a waste of time
Each category reflects the least sophisticated way of thinking about
that aspect of learning. For example, while a student with a less
sophisticated epistemological framework would "avoid
integration" of knowledge, a sophisticated student would embrace
integration of knowledge. The survey itself consists of 63 statements
which students must rate on a scale of 1 to 5 according to how much
they agree with each statement. The statements include such things as
"How much a person gets out of school mostly depends on the
quality of the teacher," and "If a person tries too hard to
understand a problem, they will most likely just end up being
confused." The ratings reflect the level of sophistication of
students' beliefs about learning and the statements are
counterbalanced such that in half of the statements, a low number
denotes sophistication while in the other half, a high number
indicates sophistication. We also included a cover letter where each
student was asked to report his or her grade level and past research
experience. The students' reports of their own past research
experience which was later ranked on a scale from 1-5 and categorized,
allowed us to investigate whether students who had already done more
research were more sophisticated in their thinking than students who
had not participated in research. We also tested whether students in
later years of college were more epistemologically sophisticated than
younger students with less college experience.
We had originally planned to give the survey in a pretest/posttest
format. In the fall of 1997, we administered the test to students in
Eric Grotzinger's Freshman Seminar, Linda Kauffman's
Experimental Biology Lab, Susan Finger's Rapid Prototyping class,
and students receiving Small Undergraduate Research Grants to conduct
independent research projects. The sample included 71 students who
were mostly freshman and juniors. At the end of the semester we
retested only the Freshman Seminar and SURG students.
To analyze the data, we first conducted a descriptive analysis and
then used an ANOVA to look for systematic trends in the data. The
independent variables were prior research experience and year in
school. The dependent variable was epistemological sophistication as
measured by survey responses. These tests were performed for each of
the 13 epistemological categories and then overall by collapsing all
categories together. We hypothesized that students in higher grade
levels and students with past research experience would show a higher
degree of epistemological sophistication than younger students and
those with no past research experience. However, results showed no
significant differences overall between groups separated by research
experience (F(5,70) = 1.04, p = .403) or by year in school (F(3,70) =
1.42, p= .245). Freshmen were typically just as sophisticated in
their thinking as were seniors; likewise, students who had never
participated in research responded similarly to those who had
extensive research experience. In 11 of the 13 categories, we found
no significant differences between groups, but in two categories,
there were significant differences. In the "Success is unrelated
to hard work" category, the between-groups variance was
significant (F(3,70) = 3.45, p = .008). However, the difference is in
the opposite direction of what was expected. Students with none to
little research experience reported more sophisticated responses than
students with a great deal of research experience. When separated by
grade level, however, there is no significant difference among groups
in this category. Additionally, in the "Concentrated effort is a
waste of time category" the between groups difference was
statistically significant for across both grade level (F(3,70) = 4.73,
p = .005) and research experience (F(3,70) = 5.22, p < .005). Once
again, though, the difference in each group is in the opposite
direction as was predicted. Due to the unequal number of respondents
within groups in each category (e.g. high numbers of freshman and
juniors, very low numbers of seniors and also high numbers of students
with little research experience and very low numbers of students with
much research experience), these differences, while significant, are
probably not valid or useful. Due to the lack of interesting results
and variation between respondents, we did not administer the survey a
second time to all students, as originally planned.
The lack of significant results may be accounted for by several
factors. First, the survey may not have been an effective measure of
epistemological sophistication. There was very little variation in
the responses, so the survey may not have been sensitive enough to
detect subtle differences between students. Also, the questions were
rather transparent as to what they were trying to measure which may
have changed the way some students responded. Furthermore, some
questions seemed to measure things unrelated to epistemology. For
example, one statement says, "Whenever I encounter a difficult
problem in life, I consult with my parents." A sophisticated
answer would disagree with this, but perhaps the statement does not
measure epistemology at all, but rather a student's relationship
with his or her parents. Finally, the Schommer survey tries to
measure beliefs that it may be hard to introspect about. What someone
reports they believe is not always the same belief that his actions
demonstrate. A survey such as this may not be able to assess
students' true attitudes toward learning and the nature of
knowledge.
[Previous Page]
[Main Report Page]
sfinger@ri.cmu.edu
|