Students can now see what contributes to their grade

CrowdGrader now lets students see which information is used in computing their grade, and which information is disregarded as unreliable.  Here are two examples, taken from an actual assignment:

CrowdGrader indicates
which review grades were
considered low-confidence.

CrowdGrader classified the -2 feedback as possible
tit-for-tat, and discarded it from the computation of
the feedback grade.    

How it works

In CrowdGrader, the grade a student receives for an assignment is the sum of three components:

1. The submission grade, computed from the grades other students gave to the submission.
2. The accuracy grade, which measures how precise the student was in giving grades.
3. The feedback grade, which depends on the feedback the student received for the reviews she or he wrote. 

Submission grade.  When computing the submission grade, not all reviews are considered equal: CrowdGrader uses a reputation system to give more weight to the input of more credible reviewers. CrowdGrader now lets students know which reviews were considered low confidence, and given less weight in the computation of the submission grade.

Feedback grade.  The feedback a review receives can range from -2 (very unhelpful or bogus) to +2 (very helpful).  CrowdGrader uses a sophisticated algorithm to determine when a negative feedback might have been given as a tit-for-tat retribution for a low grade.  Such negative feedbacks are not considered in the computation of the feedback grade. 

Instructors should compute the grades once all feedback is in

When CrowdGrader computes the grades, it classifies each feedback as either legitimate or possible tit-for-tat. Once grades are computed, students can no longer modify their feedback: this ensures that the feedback they see is the same that was used to compute the grades.  For this reason, it is important that instructors compute grades only once all the feedback is in.