This study, conducted as part of the University of Cambridge MEd program at the College of Education, examines threats to validity in assessment practices within design and engineering education through a case study of the Competitive Product Development (CPD) module. Grounded in Messick’s (1996) construct validity framework and Crooks et al.’ (1996) chain model, the research identifies critical weaknesses in assessment design, communication, and scoring processes. Data derived from student feedback and module evaluation reveal that unclear task communication, misalignment between intended learning outcomes (ILOs) and marking criteria, and overly holistic scoring methods undermine assessment validity. To address these issues, the study proposes targeted strategies, including clearer task instructions, analytical scoring rubrics, transparent marking weights, and stronger alignment of assessment criteria with learning objectives. The revised assessment framework incorporates detailed rubrics and analytic marking approaches to capture essential qualities of task performance while enhancing fairness and consistency. Findings highlight that improving assessment validity not only strengthens the reliability and credibility of academic evaluation but also supports student engagement, motivation, and learning outcomes. The paper concludes with recommendations for integrating validity-focused assessment design into institutional policy and professional practice within higher education.
References
[1]
Black, P., Harrison, C., Hodgen, J., Marshall, B., & Serret, N. (2010). Validity in Teachers’ Summative Assessments. Assessment in Education: Principles, Policy & Practice, 17, 215-232. https://doi.org/10.1080/09695941003696016
[2]
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The Concept of Validity in Psychometric Testing. Psychological Review, 111, 1061-1071. https://doi.org/10.1037/0033-295x.111.4.1061
[3]
Brown, J. D., Ferrara, S., & Bell, J. F. (1997). Validity Considerations in Using the National Educational Development Assessment System with State Education Improvement Efforts. Journal of Personnel Evaluation in Education, 11, 351-358.
[4]
Chang, C. H., & Seow, T. (2018). Geographical Education that Matters—A Critical Discussion of Consequential Validity in Assessment of School Geography. Geographical Education (Online), 31, 31-40.
[5]
Cole, N. S., & Moss, P. A. (1989). Bias in Test Use. In R. L. Linn (Ed.), Educational Measurement (3rd ed., pp. 443-507). American Council on Education/Macmillan.
[6]
Crooks, T. J., Kane, M. T., & Cohen, A. S. (1996). Threats to the Valid Use of Assessments. Assessment in Education: Principles, Policy & Practice, 3, 265-286. https://doi.org/10.1080/0969594960030302
[7]
Downing, S. M. (2003). Validity: On the Meaningful Interpretation of Assessment Data. Medical Education, 37, 830-837. https://doi.org/10.1046/j.1365-2923.2003.01594.x
[8]
Frederiksen, J. R., & Collins, A. (1989). A Systems Approach to Educational Testing. Educational Researcher, 18, 27-32. https://doi.org/10.3102/0013189x018009027
[9]
Gearhart, M., Herman, J., Baker, E., & Linn, R. (1995). Development of a Multidimensional Construct for Scoring Narrative Writing. Educational Assessment, 3, 309-333.
[10]
Gibbs, G., & Simpson, C. (2004). Conditions under Which Assessment Supports Students’ Learning. Learning and Teaching in Higher Education (LATHE), 1, 3-31.
[11]
Haertel, E. (1984). An Application of Latent Class Models to Assessment Data. Applied Psychological Measurement, 8, 333-346. https://doi.org/10.1177/014662168400800311
[12]
Heather, F., Knight, S., & MacKenzie, S. (2009). A Handbook for Teaching and Learning in Higher Education: Enhancing Academic Practice (3rd ed.). Routledge.
[13]
Kane, M. T. (2001). Current Concerns in Validity Theory. Journal of Educational Measurement, 38, 319-342. https://doi.org/10.1111/j.1745-3984.2001.tb01130.x
[14]
Linn, R. L. (1991). Evaluating the Validity of Assessments: The Consequences of Use. Educational Measurement: Issues and Practice, 16, 14-16. https://doi.org/10.1111/j.1745-3992.1997.tb00587.x
[15]
MacMillan, J. H. (2004). Educational Research: Fundamentals for the Consumer (4th ed.). Pearson Education.
[16]
Mansell, W., James, M., & Assessment Reform Group (2009). Assessment in Schools, Fit for Purpose? A Commentary by the Teaching and Learning Research Program. ESRC, TLRP.
[17]
Messick, S. (1989). Meaning and Values in Test Validation: The Science and Ethics of Assessment. Educational Researcher, 18, 5-11. https://doi.org/10.3102/0013189x018002005
[18]
Messick, S. (1995). Standards of Validity and the Validity of Standards in Performance Assessment. Educational Measurement: Issues and Practice, 14, 5-8. https://doi.org/10.1111/j.1745-3992.1995.tb00881.x
[19]
Messick, S. (1996). Validity. In R. J. Linn (Ed.), Educational Measurement (pp. 13-103). American Council on Education/Macmillan.
[20]
Moss, P. A. (1995). Themes and Variations in Validity Theory. Educational Measurement: Issues and Practice, 14, 5-13. https://doi.org/10.1111/j.1745-3992.1995.tb00854.x
[21]
Newton, P. E. (2010). The Multiple Purposes of Assessment. In E. Baker, B. McGaw, & P. Peterson (Eds.), International Encyclopedia of Education (pp. 392-396). Elsevier. https://doi.org/10.1016/b978-0-08-044894-7.00355-9
[22]
Olds, B. M., Moskal, B. M., & Miller, R. L. (2005). Assessment in Engineering Education: Evolution, Approaches and Future Collaborations. Journal of Engineering Education, 94, 13-25. https://doi.org/10.1002/j.2168-9830.2005.tb00826.x
[23]
Quality Assurance Agency for Higher Education (2006). Code of Practice for the Assurance of Academic Quality and Standards in Higher Education (2nd ed.). The Quality Assurance Agency for Higher Education. https://dera.ioe.ac.uk/9713/2/COP_AOS.pdf
[24]
Race, P. (2001). The Lecturer’s Toolkit: A Practical Guide to Learning, Teaching & Assessing (2nd ed.). Kogan Page.
[25]
Shepard, L. A. (1993). Evaluating Test Validity. In R. L. Linn (Ed.), Review of Research in Education (Vol. 19, pp. 405-450). American Educational Research Association. https://doi.org/10.2307/1167347
[26]
Sireci, S. G. (2009). Packing and Unpacking Sources of Validity Evidence: History Repeats Itself Again. In R. W. Lissitz (Ed.), The Concept of Validity: Revisions, New Directions, and Applications (pp. 19-37). Information Age Publishing.
[27]
Voltz, D., Sims, M., & Nelson, B. (2010). Connecting Teachers, Students and Standards: Strategies for Success in Diverse and Inclusive Classrooms. Association for Supervision and Curriculum Development.
[28]
Wiliam, D. (2008). Quality in Assessment. In S. Swaffield (Ed.), Unlocking Assessment (Chapter 8, pp. 123-137). Routledge.
