Review Board Management: Rank Rounds

The Rank round type option makes it easy for reviewers to establish application rank as a part of your review board scoring. In this article, we will review:

  1. How to create a rank round
  2. How reviewers will view and rank applications
  3. How to review scores for a rank round

Create a rank round

When creating a new review round, select Rank as your round type.

You will establish which applications will be in the round, who will be reviewing the applications, and how the applications will be distributed to reviewers.

Once the above is complete, you will establish the scorecard for the round. The scorecard for a rank round consists of a table of the applications assigned to reviewers where they will move the applications up and down to establish rank.

On the Build Scorecard page, you have the option to move applicant data into the table for quick review by reviewers. For example, if reviewers are ranking based primarily on a video uploaded to their application, you may consider adding that video to the rank table.

To do so, check the box next to the application question in the left table.

You can update the label of the reviewer column to make it easier for reviewers to read. You can also move columns around using the up/down arrows.

Note: If any video, image, or document questions are added to the Reviewer Columns table, the reviewer will be able to click a link to view the video, image, or document within the application. This makes for easy review and rank.

Additionally, you can allow reviewers to add optional comments on applications by toggling the "Allow Optional Reviewer Comments" on.

You can preview the scorecard to confirm it looks as you wish by clicking the 'Preview Scorecard' button in the bottom navigation bar.

Reviewer scoring

To score a rank round, reviewers will move applications up and down on a list to establish an overall rank.

Reviewers can do this by dragging an application up or down, or by using the small arrow icons to the right of an application.

Any applicant data that was added to the table will be visible as a column in the rank table. Reviewers can also choose to view the entire application by clicking on the eye icon next to the rank arrows.

If comments are enabled for the round, reviewers can leave comments on applications by clicking the message icon to the right of an application. These comments are visible to the admin of the round on the score report.

Reviewers can save their current ranking to come back at anytime prior to the deadline. Or, if ready, they can submit ranking by clicking the 'Submit Ranking' button in the bottom navigation bar.

Reviewers can re-submit rank as often as necessary prior to the deadline date. The updated rank will be reflected in your score report.

Reviewing rank scores

Applications reviewed in a rank round will receive an average rank score. This number is the average rank the application received between all reviewers it was assigned to. Sorting this number from lowest to highest on your Manage Round page or within a score report will show you the highest ranked applicant across all reviewers.

Additionally, you can export a score report for rank rounds. The score report will display the following information:

  • Applicant Application Code, First Name, Last Name, and Email Address
  • Reviewer First Name, Last Name, and Email Address
  • The status of scoring per application per reviewer
  • The rank of each application, by individual reviewer
  • The average rank for an application for a round across all reviewers
  • The overall total score for an application across all rounds (rank does not impact this score)
  • The overall average score for an application across all rounds (rank does not impact this score)