In my first blog post “What is authentic assessment and why use authentic assessments in your class,” I talked about how an authentic assessment (AA) differs from traditional assessments on the five dimensions. In this blog, I am sharing one of my “authentic” assessment assignments, Create Your Own Problem Set wherein students work in groups in phase 1 to create a solvable problem of a topic each group chooses, and then work individually to solve a problem created by other groups in phase 2. The desired outcome of this experiment is to provide students the opportunities to apply course materials in various contexts as well as to create a sense of ownership for students by taking a more active role in learning. I will start with where this idea came from, and wrap up by sharing how I characterize it on those five dimensions. I will also share some of tips if you are interested in using the same in your class.
Where did this Idea Come from?
I find students are increasingly having difficulty solving problems when taking exams. Many students complain that the instructor “throws a curve ball” on exams when in fact, the problems are slightly modified versions of the problem contexts found at the end of the assigned textbook chapter. In addition, more and more students lack the capability to apply models they learn in course materials to real-world problems. For students, this lack of applying capability makes students question “why are we learning this model when we cannot use it at all,” causing them to become disengaged. For me, the instructor, I believe that if a student cannot apply, then the student doesn’t really learn it—a point I made in my first mini blog too.
To address this concern, I wanted students to take an active role to think about how to apply, as opposed to a mere solving a problem that I (or the textbook authors) have created, then equate ability of solving a canned problem to ability of applying. Toward that, students should take the role of an instructor and/or a textbook authors who designed those practice problems. And that is where this “Create Your Own Problem Set” idea came from.
The “Create Your Own Problem Set”: What it is and Tips for Instructors
The “Create Your Own Problem Set” is a two-phase assignment. In phase 1, groups of approximate four student choose a topic from a list of topics to create the problem set; first come first serve. To ensure all topics are chosen, instructors may limit the number of groups for each topic. In such case, instructors should also ensure the level of difficulty across problems of the same topic and points allocation is comparable. The phase 1 task is to create a five-point problem along with the answer key that details the step-by-step solutions to the problem created by the group. The answer key should include information as to how partial credit is awarded at each step, along with the rationale as to why the partial credit should be awarded. Rubrics in this phase should focus on the completeness of the information needed to solve the problem set and the accuracy of the answer key to the created problem. By providing the information that the problem is solvable, the likelihood that students understand the assumptions is increased. This is something that many students tend to overlook but is important in real-world application. Similarly, the accurate answer key requirement demonstrates students’ understanding of the problem they created. To encourage creativity, and to address the complaint of “curve ball,” a small portion of the points awarded is based on the originality of the problem context: changes to numbers to the chapter-end and/or in-class practice problems means no originality, whereas application to a not-so-obvious context exhibits originality. In essence, students “study” together by creating a problem and answer key. The requirement of discussing rationale of how these 5 points should be allocated allows students to think about the relative importance or difficulty level of each step in solving a problem, and hopefully, students will then pay more attention to those steps when they themselves are utilizing a given tool in future applications.
Phase 2 is an individual assignment. Each student within each team is required to solve a different problem created by the other teams in the class. To help students learn a different topic, instructors may limit the individual problem to be solved to problems on topics different from ones’ own phase-1 problem. Similar to chapter-end problems, this is an individual assignment where problem(s) to be solved are created by other students as the only difference.
As aforementioned, in cases where multiple groups create a problem set on the same topic/tool, the instructor needs to ensure the difficulty level and points allocation are consistent across problems. The faculty should also make necessary modifications to the problem statements where the instructor sees the fits. As such, it is recommended to have a one to two weeks gap between the due date of the phase 1 and the available date of the phase 2. The final point allocations should also be made clear in the problem sets released to students for phase 2.
Linking to the Authentic Assessment “Scale”
Authentic assessments differ from traditional assessment on five dimensions. Below is how I would rate this Create Your Own Problem Set on each of the five dimensions. Ratings can be subjective, but overall, I do think the Create Your Own Problem Set should be ranked highly toward the authentic assessment. Among the five dimensions, I rate the “real-life” and “direct evidence” relatively closer to traditional assessments, compared to the other three dimensions. I rate the “real-life” dimension lower (or closer to traditional assessments) because students are most likely to refer to the textbook and/or in-class examples, and find an alternative context to frame the problem. To complement to this shortcoming, I have students to do multi-phase term project that requires them to find an organization, and apply at least two different tools, one in each phase, to solve that real-world problems for that organization they study.
As to the “direct evidence” dimension, I rate it relatively lower but not the lowest among the five dimensions because phase 1 of this assignment is group based; as such, I cannot completely assess whether everyone in that group demonstrates the same level of mastery of the topic. This is part of the consideration that this assignment combines both group-level and individual- level, so that at least students’ work in phase 2 allows the instructor to assess student learning individually.
Another benefit of this assignment is that it helps uphold integrity, particularly for online courses and exams—since each time the problem is new and unique, it is easy for instructors to detect when those problems are posted on Chegg.com or the like. Finally, I would like to mention that while I’ve experimented this approach in a purely quantitative tool course, I believe the same can be applied in more content oriented courses. In fact, that is my next step— to test this idea, I’ve asked students to create their own discussion questions based on articles I posed for threaded discussions. Similarly, an instructor can ask students to make problem statements of topic(s) students choose.
Extended reading: Make it Stick: The Science of Successful Learning by Peter C. Brown, Henry L. Roediger, III, and Mark A. McDaniel underlines the pedagogical, theoretical reasoning of this “Create Your Own Problem Set” assignment.
Related YouTube video: it’s more from students’ viewpoint about how to study and/or prepare for exams which is a tradition assessment