Current science assessments typically present a series of isolated fact-based questions, poorly representing the complexity of how real-world science is constructed. The National Research Council asserts that this needs to change to reflect a more authentic model of science practice. We strongly concur and suggest that good science assessments need to consist of several key factors: integration of science content with scientific inquiry, contextualization of questions, efficiency of grading and statistical validity and reliability. Through our Situated Assessment using Virtual Environments for Science Content and inquiry (SAVE Science) research project, we have developed an immersive virtual environment to assess middle school children’s understanding of science content and processes that they have been taught through typical classroom instruction. In the virtual environment, participants complete a problem-based assessment by exploring a game world, interacting with computer-based characters and objects, collecting and analyzing possible clues to the assessment problem. Students can solve the problems situated in the virtual environment in multiple ways; many of these are equally correct while others uncover misconceptions regarding inference-making. In this paper, we discuss stage one in the design and assessment of our project, focusing on our design strategies for integrating content and inquiry assessment and on early implementation results. We conclude that immersive virtual environments do offer the potential for creating effective science assessments based on our framework and that we need to consider engagement as part of the framework.
References
[1]
National Research Council. Classroom Assessment and the National Science Education Standards; The National Academies Press: Washington, DC, USA, 2001.
[2]
National Research Council. A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas; The National Academies Press: Washington, DC, USA, 2012.
[3]
National Research Council. Inquiry and the National Science Education Standards: A Guide for Teaching and Learning; The National Academies Press: Washington, DC, USA, 2000.
[4]
Bybee, R. Teaching Science as Inquiry. In Inquiring into Inquiry Learning and Teaching in Science; Minstrel, J., Van Zee, E.H., Eds.; American Association for the Advancement of Science (AAAS): Washington, DC, USA, 2000. Chapter 3; pp. 20–46.
[5]
Dewey, J. Democracy and Education (First free press paperback 1966 ed.); Macmillan Company: New York, USA, 1944.
[6]
National Research Council. National Science Education Standards: Observe, Interact, Change, Learn; The National Academies Press: Washington, DC, USA, 1996.
[7]
Rutherford, F.J. Vital Connections: Children, Books, and Science. In Vital Connections: Children, Science, and Books; Saul, W., Jagusch, S.A., Eds.; Library of Congress: Washington, DC, USA, 1991; pp. 21–30.
[8]
Li, J.; Klahr, D. The psychology of scientific thinking: Implications for science teaching and learning. In Teaching science in the 21st century, 1st ed.; Rhoton, J., Shane, P., Eds.; National Science Teachers Association: Arlington, VA, USA, 2006; pp. 307–327.
[9]
Massachusetts Department of Education. Massachusetts Science and Technology/Engineering Curriculum Framework [Electronic Version]. 2006. Available online: http://www.doe.mass.edu/frameworks/scitech/1006.pdf (accessed on 15 January 2008).
[10]
Anderson, R. Reforming science teaching: What research says about inquiry. J. Sci. Teach. Educ. 2002, 13, 1–12, doi:10.1023/A:1015171124982.
[11]
Gibson, H.; Chase, C. Longitudinal impact of an inquiry-based science program on middle school students’ attitudes toward science. Sci. Educ. 2002, 86, 693–705, doi:10.1002/sce.10039.
[12]
Savage, L.; Ketelhut, D.J.; Varnum, S.; Stull, J. Raising Interest in Science Careers through Informal After-School Experiences; Paper presented at the National Association for Research in Science Teaching: Philadelphia, PA, USA, 2010.
[13]
Leonard, W.H.; Speziale, B.J.; Penick, J.E. Performance assessment of a standards-based high school biology curriculum. Am. Biol. Teach. 2001, 63, 310–316.
[14]
Alberts, B. Some Thoughts of a Scientist on Inquiry. In Inquiring into Inquiry Learning and Teaching in Science; Minstrel, J., Van Zee, E.H., Eds.; American Association for the Advancement of Science: Washington, DC, USA, 2000. Chapter 1; pp. 3–13.
[15]
Blanchard, M.; Sutherland, S.; Osborne, J.; Sampson, V.; Annetta, L.; Granger, E. Is inquiry possible in light of accountability? A quantitative comparison of the relative effectiveness of guided inquiry and verification laboratory instruction. Sci. Educ. 2010, 94, 577–616, doi:10.1002/sce.20390.
[16]
Marx, R.; Blumenfeld, P.; Krajcik, J.; Fishman, B.; Soloway, E.; Geier, R.; Tal, R. Inquiry-based science in the middle grades: Assessment of learning in urban systemic reform. J. Res. Sci. Teach. 2004, 41, 1063–1080, doi:10.1002/tea.20039.
[17]
Tai, R.; Liu, C.; Maltese, A.; Fan, X. CAREER CHOICE: Enhanced: Planning early for careers in science. Science 2006, 312, 1143–1144, doi:10.1126/science.1128690.
[18]
Jorgenson, O.; Vanosdall, R. The death of science: What we risk in our rush towards standardized testing and the three R’s. Phi Delta Kappan 2002, 83, 601–605.
[19]
Nelson, B.; Ketelhut, D.J. Designing for real-world inquiry in virtual environments. Educ. Psychol. Rev. 2007, 19, 265–283, doi:10.1007/s10648-007-9048-1.
[20]
Carnegie Corporation. The Opportunity Equation: Transforming Mathematics and Science Education for Citizenship and the Global Economy; Carnegie Corporation of New York: New York, NY, USA, 2009.
[21]
Krajcik, J.S.; McNeil, K.L.; Reiser, B.J. Learning-goals-driven design model: Developing curriculum materials that align with national standards and incorporate project-based pedagogy. Sci. Educ. 2007, 92, 1–32.
[22]
Lave, J.; Wenger, E. Situated Learning: Legitimate Peripheral Participation; Cambridge University Press: New York, NY, USA, 1991.
[23]
Brown, J.S.; Collins, A.; Duguid, P. Situated cognition and the culture of learning. Educ. Res. 1989, 18, 32–42.
[24]
Songer, N.; Wenk, A. Measuring the Development of Complex Reasoning in Science. In Paper presented at the American Education Research Association (AERA) Annual Meeting, Chicago, 25 April 2003.
[25]
Michael, J. Conceptual assessment in the biological sciences: A National Science Foundation sponsored workshop. Adv. Physiol. Educ. 2007, 31, 389–391.
[26]
Resnick, L.B.; Resnick, D.P. Assessing the Thinking Curriculum: New Tools for Educational Reform. In Changing Assessments: Alternative Views of Aptitude, Achievement, and Instruction; Gifford, B., O’Connor, M., Eds.; Kluwer Academic Publishers: Norwell, MA, USA, 1992; pp. 37–75.
[27]
Southerland, S.A.; Smith, L.K.; Sowell, S.P.; Kittleson, J.M. Resisting unlearning: Understanding science education’s response to the United States’ national accountability movement. Rev. Res. Educ. 2007, 31, 45–77.
[28]
National Research Council. America’s Lab Report: Investigations in High School Science; National Academies Press: Washington, DC, USA, 2005.
[29]
Harlow, A.; Jones, A. Why students answer TIMSS science test items the way they do. Res. Sci. Ed. 2004, 34, 221–238, doi:10.1023/B:RISE.0000033761.79449.56.
[30]
Shavelson, R.J.; Baxter, G.P. What we’ve learned about assessing hands-on science. Educ. Leadersh. 1992, 49, 20–25.
[31]
Behrens, J.T.; Frezzo, D.; Mislevy, R.; Kroopnick, M.; Wise, D. Structural, Functional, and Semiotic Symmetries in Simulation-Based Games and Assessments. In Assessment of Problem Solving Using Simulations; Baker, E., Dickieson, J., Wulfeck, W., O’Neil, H., Eds.; Lawrence Erlbaum Associates: New York, NY, USA, 2007.
[32]
Stecher, B.M.; Klein, S.P. The cost of science performance assessments in largescale testing programs. Educational Evaluation and Policy Analysis 1997, 19, 1–14.
[33]
National Assessment Governing Board (NAGB). Science Framework for the 2009 National Assessment of Educational Progress; NAGB, U.S. Department of Education: Washington, DC, USA, 2008.
[34]
Barab, S.; Arici, A.; Jackson, C. Eat your vegetables and do your homework: A design based investigation of enjoyment and meaning in learning. Educ. Technol. 2005, 45, 15–20.
[35]
Nelson, B. Exploring the use of individualized, reflective guidance in an educational multi-user virtual environment. J. Sci. Educ. Technol. 2007, 16, 83–97, doi:10.1007/s10956-006-9039-x.
[36]
Nelson, B.; Erlandson, B.; Denham, A. Global channels for learning and assessment in complex game environments. Br. J. Educ. Technol. 2011, 42, 88–100, doi:10.1111/j.1467-8535.2009.01016.x.
[37]
Steele, M. Teaching science to middle school students with learning problems. Sci. Scope 2005, 29, 50–51.
[38]
Ketelhut, D.J. The impact of student self-efficacy on scientific inquiry skills: An exploratory investigation in river city, a multi-user virtual environment. J. Sci. Educ. Technol. 2007, 16, 99–111, doi:10.1007/s10956-006-9038-y.
[39]
Shute, V.J.; Ventura, M.; Bauer, M.I.; Zapata-Rivera, D. Melding the Power of Serious Games and Embedded Assessment to Monitor and Foster Learning: Flow and Grow. In The Social Science of Serious Games: Theories and Applications; Ritterfeld, U., Cody, M.J., Vorderer, P., Eds.; Routledge/LEA: Philadephia, PA, USA, 2009. Chapter 18; pp. 295–321.
[40]
Clark, D.; Nelson, B.; Sengupta, P.; D’Angelo, C. Rethinking Science Learning through Digital Games and Simulations: Genres, Examples, and Evidence. An NAS Commissioned Paper. Available online: http://www7.nationalacademies.org/bose/Clark_Gaming_CommissionedPaper.pdf (accessed on 7 October 2009).
[41]
Ketelhut, D.J.; Dede, C. Alternative Assessments of Students’ Understanding of Scientific Inquiry via a Multi-User Virtual Environment. In Invited Paper Presented at the Distributed Learning and Collaboration (DLAC-II) Symposium, Singapore, Singapore, 11 June 2007.
[42]
The Commonwealth of Pennsylvania. Pennsylvania System of State Assessment. 2011. Available online: http://www.portal.state.pa.us/portal/server.pt/community/pennsylvania_system_of_school_assessment_(pssa)/8757/resource_materials/507610 (accessed on 30 December 2011).
[43]
Du Bay, W. the Principles of Readability. 2004. Available online: http://www.nald.ca/library/research/readab/readab.pdf (accessed on 21 March 2013).
[44]
Creswell, J.W. Qualitative Inquiry and Research Design; Sage: Thousand Oaks, CA, USA, 1998.
