Formative Assessment Design Challenge Updated

For the past four years, I have been working on a research project investigating a system of technology-enhanced accessible classroom assessment tasks called ONPAR. ONPAR assessments are designed to provide teachers and students with diagnostic feedback about students' progress towards the learning goals set out in the Next Generation Science Standards (NGSS) for middle school.

I am not an assessment expert by any measure, however, in my role providing professional development to teachers piloting ONPAR with their students, I gained a fair amount of insight into accessible design and classroom assessment. I also had an opportunity to hear directly from teachers what they liked about our assessments, what they did not like, and what features they thought would make ONPAR easier to use in real classrooms. Two improvements that came up several times during the study were: 1) the ability for teachers to select which question clusters to use or not use within a task, and 2) diagnostic feedback that links directly to actions that can be taken by teachers and/or students as a result of the assessment.

At the same time our research team was developing and testing the ONPAR system of assessments, another group was busy creating a full set of high-quality open-resource curriculum materials for middle school science aligned with NGSS called OpenSciEd. This group, made up of experts from universities, non-profits, and state agencies, has made a big splash in the science education community by offering their fully adaptable NGSS-aligned curriculum materials online completely free of charge. This price point, along with their standards-based approach to instruction and assessment, has resulted in numerous school districts adopting the OpenSciEd units as they become available.

For my CEP 813 Formative Assessment Design project, I would like to take what I learned from working on the ONPAR project and add new insights gained in the MSU course to create an assessment that (a) aligns with an OpenSciEd unit not well supported by the current ONPAR suite of materials (sound waves), and (b) addresses the two improvements requested by teachers in the ONPAR pilot: flexibility and actionable feedback. 

The purpose of my assessment will be to provide middle school teachers and their students with informative and actionable feedback about students' understanding of key concepts identified in the NGSS related to the topic of sound waves. It will also be designed to align with the scope and sequence described in the OpenSciEd 8th grade sound waves unit, but be flexible enough to use with other curricula or with younger students (fifth or sixth grade rather than eight). Unlike other assessments, where accessibility and equity are often afterthoughts, I plan to tackle these considerations from the outset in an effort to create an assessment that can be effectively used by most middle school students without the need for additional modification.

One major challenge I will have is creating assessments with the qualities mentioned above without the benefit of programming skills. ONPAR achieves much of its accessibility and all of its automated diagnostic feedback with the support of professional programmers. My version will need to rely on easy-to-use digital productivity tools available to me as part of the university community or free for everyone online and usable by me as a less-then-tech-savvy boomer. After some looking into technology tools, I think applying the "kiss" principle (keep it simple stupid) is probably best. Most teachers have access to Google docs and are comfortable using them, so anything I make using Google tools will probably not require much of a learning curve. Plus, Google Forms includes some neat features that make it easy to create and modify assessments that are at least partially auto-scored and allow students to respond to more open-ended questions in a variety of formats. See my blog post on Using Google Classroom to Create Formative Assessments for more details.   

Ideally, I would like to create a collection of assessments, or an "item bank", that would provide teachers with a choice of NGSS-aligned question clusters which could be used individually or strung together in groups to create longer assessments. Rubrics would be used for scoring individual items and suggest feedback for students. Content assessed and recommended actions would align with a subset of lessons within the OpenSciEd sound waves unit, but would also be generic enough to use with other curricula. Actionable feedback will be emphasized over scores as a way to encourage teachers and students to use the assessments formatively. 

Fairness is at the heart of the design of ONPAR assessments, so after nearly 5 years working on a University research project that designs and tests accessible assessments, and then taking a deeper dive into accessibility and assessment bias in CEP813, I see unfair assessment practices everywhere: Content tests in science or even math that include long, confusing paragraphs of information for students to sift through before they get to the actual question; Electronic learning platforms that add decorations and doodads that distract rather than enhance; Contexts designed to engage certain segments of the population that are more likely to offend than engage; and assessments that privilege the ways of knowing valued in the dominant school culture at the expense of everyone else.  

With this "challenge" I will do my best to create assessments that are as accessible as possible to different kinds of learners and structured in a way that allows for administration to be fair and bias-free. However, as Starck et. al. write in their 2020 article Teachers are people too: Examining the Racial Bias of Teachers Compared to Other Adults, " current research shows that teachers' racial attitudes largely reflect those held within their broader society."  They go on to conclude that "teachers need just as much support in contending with their bias the population at large".  To that end, here are some resources I am using to address my own implicit biases. Perhaps you might find them useful too!

Resources and Links 

•  This excellent resource from the Southern Poverty Law Center called Learning for Justice  (formerly "Teaching Tolerance").
•  The Talking about Race project of the National Museum of African American History and Culture
• This summary of Starck et. al.'s research about teachers' racial bias on Brookings.edu.
• This opportunity to test your implicit bias at Harvard's Project Implicit.
• The New York Times' 1619 Project.

Click here to see Formative Assessment Design document in progress. 

Image by Startup Stock Photos from Pexels.com