Students can now compare their reviews with those of others

Once the review phase of an assignment is concluded, students can now compare the reviews they wrote with the reviews other students wrote for the same assignments.

By comparing their reviews with those by others, students can discover any reviewing errors they might have made, or any aspects of the submissions they might have overlooked.  Students also learn by seeing which advice and feedback other students gave in the same circumstances.

Here are two screenshots (of a fake assignment we use for testing), showing how this feature looks like.

Click to enlarge

Click to enlarge

We are considering introducing an option that lets students view the reviews written by others on the same submissions also during the review process, so that students can compare their observations with those of others, and come up with more balanced reviews that reflect a broader consensus.  This would make the review phase a collaborative phase with many discussions (not unlike the workings of a conference program committee).  This would be an option, so that instructors would turn it on or off depending on their wishes.

Students would see reviews written by others only once they turn in a review themselves, and we plan to store the initial review and grade.  This should act as a disincentive for turning in initially a blank review, and using the comments written by others to come up with the final review.  Other disincentives towards such lazy behavior could be built into our statistical analysis tool that we use for assigning grades.

We would love to hear your thoughts on this.