Shifting epistemic levels in introductory students’ lab reports • Scientific writing has the potential to improve student thinking (Quitadamo & Kurtz 2007), but less is known about how and under what conditions. • Libarkin & Ording (2012) showed that, with repeated practice, non-majors improve data presentation, include more course content, and write clearer conclusions, but that posing hypotheses and connecting to a broader scientific purpose are more difficult to improve. • In order for writing to support reasoning, it must involve more sophisticated engagement with scientific knowledge. • We conjecture that success will depend on how students understand the purpose of writing a lab report - as a practice of making and justifying claims rather than an exercise in reporting on experimental outcomes. • In this work, we ask: How does shifting the way we frame and evaluate student lab reports impact the epistemic sophistication of their writing? Context: Second semester introductory biology course with 3 hr./week lab. Here we analyze changes in the first lab report of the semester over two years Data: Discussion sections from student lab reports (taught by the same graduate student TA) from 2014 baseline (N=24) and 2015 re-framed labs (N=19) Analysis: 1. Text categorized as either knowledge claim, evidence, reasoning or other. 2. Coding scheme developed to assign epistemic levels to each category (adapted from Kelly & Takao, 2002) with higher epistemic levels (more sophisticated) defined as: • theoretically general knowledge claims • use and evaluation of multiple forms of evidence • mechanistic reasoning that evaluates strength and generality of knowledge claims 3. All discussion sections from both years coded by single coder (reliability analysis not yet complete) Epistemic Level Examples of coded text 2014 (N=24) 2015 (N=19) Knowledge Claims 0. No claim, re-statement of results “it was hypothesized that the E938 strain of E. coli (Mut- Lac-) would have more colonies on the LB agar + rif plates than the E939 strain.” 13 0 1. Strain with damaged DNA repair will have higher mutation rate “the hypothesis [was] that as the efficiency of DNA repair mechanisms increases the mutation rate decreases” 8 1 2. Mutations can be beneficial “The survival and adaptation of the mutants in this experiment demonstrates how mutations can be beneficial and neutral, not simply detrimental to a species’ survival.” 3 6 3. Fitness associated with mutation rate depends on environment “the experiment supported the hypothesis in demonstrating that having a higher mutation rate in harsh/non-typical environments is favorable to an organism’s fitness” 0 12 Sources of Evidence 1. Single source (growth in novel environments) “the E938 strain showing a much higher percentage of colonies with 1 or more red papillae than the E939 strain (Figure 3) ” 23 7 2. Compares two sources (growth in novel and neutral environments) “…under standard conditions in the LB agar plates, both strains had similar numbers of colonies…more mutant colonies grew on rifampicin than wildtype colonies” 1 11 3. Compares and evaluates multiple sources (literature or simulation output) “If these results hold true in nature, then why is DNA repair so pervasive? …further studies should be done to explore the long-term survival of bacteria with higher mutation rates.” 0 1 agar + antibiotic (rif) nutrient agar WT strain (E939) Mutant strain (E938) Reasoning 0. No causal reasoning provided “it can be deduced that the E938 strain’s status as mut- allows it to better survive.” 2 1 1. Broken DNA repair causes mutation “E938, since it cannot repair mutations, had a greater chance of mutating” 16 1 2. Increased mutation rate causes fitness advantage “the ability to digest lactose gives the bacteria another source of nutrition…. this should give the mutant strain a reproductive advantage” 5 9 3. Fitness advantage depends on environment “high mutation rate seems to help individuals grow in antibiotics where mutation is required to help the organism to survive.... [In a] typical laboratory environment, it seems that more mutations are detrimental.” 1 8 • Shifts in epistemic level cannot easily be explained by differences in incoming ability or inherent skill of the TA. Nor do they come from extensive instruction (this was the first lab report). • We suspect the shifts represent a difference in how students understood the purpose of the lab and the write- up, which we plan to investigate in further study. • The shifts were achieved by changing the framing of the lab experiment – not the protocol itself, which is a low- cost, high-impact way to reform learning in introductory labs. Julia S. Gouvea & Cynthia F.C. Hill Departments of Education & Biology, Tufts University, Medford, MA “For Part 1, it was hypothesized that the E938 strain of E. coli (Mut-) would have more colonies on the LB agar + rif plates than the E939 strain. … The results confirm these hypotheses by showing that E938 had a much larger average number of colonies on the LB + rif plates than E939. … These results can be explained because E938 lacks a functional DNA repair mechanism therefore mutations occur more frequently. ” 0 1 1 “[G]rowth on LB nutrient for the two strains does not have observable difference…sufficient to suggest that the strain’s high mutation rate does not have any considerably detrimental effect on survival in a normal, benign environment. … On media that contain rifampicin, however, the number of E938 colonies … is almost 5 times as high.... More colonies are observed because, with a higher mutation rate, E938 will have more individuals that have developed resistance to rifampicin and survived. … [I]t is possible for high mutation rate to give populations a higher ground in the evolutionary process…. Nevertheless, this is still inconclusive as to whether individuals with higher mutation rate are always preferred.” 3 2 3 Typical discussion sec/on in 2014: Typical discussion sec/on in 2015: Epistemic sophis/ca/on of discussion sec/ons shi;s from one year to the next with students making higher level claims, using more sources of evidence and more complex reasoning to support claims. Background & Mo/va/on Re‐framing the lab experience Methods Summary & Implica/ons Guidelines for writing the lab report discussion section References Kelly, G. J., & Takao, A. (2002). Epistemic levels in argument: An analysis of university oceanography students’ use of evidence in wriFng. Science Educa+on, 86(3), 314–342. Libarkin, J., & Ording, G. (2012). The uFlity of wriFng assignments in undergraduate bioscience. CBE Life Sciences Educa+on, 11(1), 39–46. Quitadamo, I. J., & Kurtz, M. J. (2007). Learning to Improve: Using WriFng to Increase CriFcal Thinking Performance in General EducaFon Biology. CBE ‐ Life Sciences Educa+on, 6, 140–154. In the original lab, students read pre- lab materials and took a quiz. In the re-framed lab, students began with group discussion of benefits/ costs of high/low mutation rate. Both years students conducted the same experiment: Two strains of E. coli that differ in mutation rate (E938 has sub-optimal DNA repair) were grown in benign (nutrient agar) and novel environments (e.g. antibiotic). Original Lab (2014) Reframed Lab (2015) You should include the following: • One-sentence summary of what you found • A statement of how your results relate to your hypothesis • If your results are unexpected, offer some explanations • Suggestions for future experiments that should be conducted “What does the experiment tell you about some of the questions raised in this lab, and what does the experiment not tell you?” “You will be graded for the logical flow and for evidence of your own thinking.” “Don’t be afraid to say things that are “incorrect,” but be sure to fully explain your thinking.” Acknowledgements We thank Dr. Colin Orians for thoughts on re‐framing labs, Dr. Michael Grossi for help in implemenFng labs, and the students and graduate students who allowed us to collect and analyze their work.