Saturday, March 21, 2015

Getting reviews, not stress

Students can learn much from feedback from their peers -- but sometimes, the grades that come with such feedback can be a cause for undue stress.  Often, what is desired is peer feedback -- without the judgement component.

For instance, a typical way in which CrowdGrader is used is to provide students feedback on their work -- be it an essay, an Android application, or a lab report.  The students can then use the feedback to improve their work, and resubmit it.  When CrowdGrader is used in this way, the grades that are associated with reviews can be a spurious source of stress: students are often better off focusing on improving their work than on the grades.  Nevertheless, having some grades can be useful to the teacher to get an idea of class progress.

Teachers can now check the "Hide review grades" box in assignments:

This has the effect of showing students only the final instructor-assigned grade.  Teachers and their assistants have access to the full information.  Presto, feedback without judgement!

Towards supporting submit-feedback-resubmit

The above change is one in a series of planned changes that aim at supporting a submit-feedback-resubmit workflow.  In this workflow, students will be able to submit a draft of their work, and receive feedback (but no grades) on their draft.  Students would then be able to improve their work in light of the peer feedback, and re-submit it for evaluation. 

Implementing the option to hide review grades is a first step.  We are soon going to allow to clone an assignment together with the reviewer assignment.  In this way, instructors can create two assignments for a given homework.  In the first one, students get feedback (but no grades) on their preliminary work.  Students would then be able to improve their work in view of the received feedback, and re-submit it to the second assignment.  In this second assignment, their work would be assigned to the same reviewers who wrote the initial feedback.  These reviewers would be able to judge how well the feedback has been taken into account, and they would assign an overall grade to the work.  This is similar to what happens in scientific conferences with an author-response phase. 

We plan to provide support for submit-feedback-resubmit in less than a month.

Sunday, March 8, 2015

Giving more information and control to instructors

Today we made two changes to CrowdGrader that we hope will give instructors better control of the grades, and students more assurance that their work is properly evaluated.

Average grade, and grade difference, for every submission

First, the list of all submissions now includes both the average grade received by a submission, and the delta between the highest and lowest grades.  You can sort on either column, thus having quick access to the students who got the lowest average grade, and those who got the most widely differing grades.

This is how this looks on a sample assignment, and on a real assignment (with student identities hidden for privacy):

Average grade and grade difference
for a sample assignment.
Average grade and grade difference
for a real assignment.

Instructors can now also grade submissions

The other change, which goes hand in hand with the previous one, is that instructors can now grade submissions, if they wish.  The option is offered when viewing the details of a submission, and here is how it looks: 

If instructors select Review and grade this submission, they are led to a page where they enter a review, and a grade, for the submission, using an interface that is the same as the one used by regular reviewers.  When the assignment grades are computed, CrowdGrader gives precedence to the teacher grade over the student-assigned grades. 

The result: more information, and more control

These two changes together give more information and more control to instructors.  Instructors can easily see which submissions received the most widely different (and thus, more unreliable) grades, and they can easily re-grade a subset of the submissions, ensuring that they receive proper grades. 

Furthermore, in case where students give widely exaggerated grades, the instructors can easily intervene and assign proper grades to the submissions. 

Since instructor-assigned grades take precedence over student-assigned grades, this also creates a powerful incentive for students to be fair in their grading: students whose grade differs markedly from the instructor-assigned grade will receive a low review grade