Carnegie Mellon   RAIRE Award

Results and Discussion from the Schommer Epistemological Survey
Jennifer White, Senior, Psychology

As part of our evaluation efforts, we also sought to measure how, if at all, the experience of research changed students' mental models for thinking about learning and the nature of knowledge. For example, by the end of the research experience would students typically alter their attitudes toward the role of failure and uncertainty in the learning process? We predicted that students' epistemological frameworks would, in fact, become more sophisticated (that is more willing to fail, less certain of absolute truths, more independent, etc.) as a result of incorporating research into learning.

We used the Schommer Epistemological Survey, developed by Marlene Schommer of Wichita State University, to measure students' beliefs about learning and knowledge. This survey breaks epistemological beliefs into the following categories:

  1. Seek single answers
  2. Avoid integration
  3. Avoid ambiguity
  4. Knowledge is certain
  5. Depend on authority
  6. Don't criticize authority
  7. Ability to learn is innate
  8. Can't learn how to learn
  9. Success in unrelated to hard work
  10. Learn the first time
  11. Learning is quick
  12. Concentrated effort is a waste of time
Each category reflects the least sophisticated way of thinking about that aspect of learning. For example, while a student with a less sophisticated epistemological framework would "avoid integration" of knowledge, a sophisticated student would embrace integration of knowledge. The survey itself consists of 63 statements which students must rate on a scale of 1 to 5 according to how much they agree with each statement. The statements include such things as "How much a person gets out of school mostly depends on the quality of the teacher," and "If a person tries too hard to understand a problem, they will most likely just end up being confused." The ratings reflect the level of sophistication of students' beliefs about learning and the statements are counterbalanced such that in half of the statements, a low number denotes sophistication while in the other half, a high number indicates sophistication. We also included a cover letter where each student was asked to report his or her grade level and past research experience. The students' reports of their own past research experience which was later ranked on a scale from 1-5 and categorized, allowed us to investigate whether students who had already done more research were more sophisticated in their thinking than students who had not participated in research. We also tested whether students in later years of college were more epistemologically sophisticated than younger students with less college experience.

We had originally planned to give the survey in a pretest/posttest format. In the fall of 1997, we administered the test to students in Eric Grotzinger's Freshman Seminar, Linda Kauffman's Experimental Biology Lab, Susan Finger's Rapid Prototyping class, and students receiving Small Undergraduate Research Grants to conduct independent research projects. The sample included 71 students who were mostly freshman and juniors. At the end of the semester we retested only the Freshman Seminar and SURG students.

To analyze the data, we first conducted a descriptive analysis and then used an ANOVA to look for systematic trends in the data. The independent variables were prior research experience and year in school. The dependent variable was epistemological sophistication as measured by survey responses. These tests were performed for each of the 13 epistemological categories and then overall by collapsing all categories together. We hypothesized that students in higher grade levels and students with past research experience would show a higher degree of epistemological sophistication than younger students and those with no past research experience. However, results showed no significant differences overall between groups separated by research experience (F(5,70) = 1.04, p = .403) or by year in school (F(3,70) = 1.42, p= .245). Freshmen were typically just as sophisticated in their thinking as were seniors; likewise, students who had never participated in research responded similarly to those who had extensive research experience. In 11 of the 13 categories, we found no significant differences between groups, but in two categories, there were significant differences. In the "Success is unrelated to hard work" category, the between-groups variance was significant (F(3,70) = 3.45, p = .008). However, the difference is in the opposite direction of what was expected. Students with none to little research experience reported more sophisticated responses than students with a great deal of research experience. When separated by grade level, however, there is no significant difference among groups in this category. Additionally, in the "Concentrated effort is a waste of time category" the between groups difference was statistically significant for across both grade level (F(3,70) = 4.73, p = .005) and research experience (F(3,70) = 5.22, p < .005). Once again, though, the difference in each group is in the opposite direction as was predicted. Due to the unequal number of respondents within groups in each category (e.g. high numbers of freshman and juniors, very low numbers of seniors and also high numbers of students with little research experience and very low numbers of students with much research experience), these differences, while significant, are probably not valid or useful. Due to the lack of interesting results and variation between respondents, we did not administer the survey a second time to all students, as originally planned.

The lack of significant results may be accounted for by several factors. First, the survey may not have been an effective measure of epistemological sophistication. There was very little variation in the responses, so the survey may not have been sensitive enough to detect subtle differences between students. Also, the questions were rather transparent as to what they were trying to measure which may have changed the way some students responded. Furthermore, some questions seemed to measure things unrelated to epistemology. For example, one statement says, "Whenever I encounter a difficult problem in life, I consult with my parents." A sophisticated answer would disagree with this, but perhaps the statement does not measure epistemology at all, but rather a student's relationship with his or her parents. Finally, the Schommer survey tries to measure beliefs that it may be hard to introspect about. What someone reports they believe is not always the same belief that his actions demonstrate. A survey such as this may not be able to assess students' true attitudes toward learning and the nature of knowledge.


[Previous Page] [Main Report Page]

sfinger@ri.cmu.edu