Review Board Management: Build Round Scorecard
Whether the round is a rank or scorecard round, you will need to establish how the reviewer in your round will be scoring. This is done on the Build Scorecard page.
In this article, you will learn how to:
Add Round Instructions:
For both round types, start by adding round instructions from the round Settings page. Click the gear icon to access the settings at any time. Round instructions will appear to reviewers at the top of the scorecard.
Scorecard Round:
The scorecard is a group of questions that your reviewers will answer while reviewing each application. To begin building your scorecard drag a question type from the left-hand Elements menu into one of your scorecard’s sections.
Available Question Types:
Text
- Short Input – This will provide a short text answer box.
- Long Input – This will provide a longer text answer box, often used for comment boxes. You can add a word count to this question type.
- Text – This is text displayed on the scorecard, often used for adding additional instructions within a section.
Contact
- Email Address - This will provide an answer box that restricts the end users from typing anything outside of a typical email format: (email@domain.com).
- Phone Number – This will provide an answer box that restricts end users from typing anything outside of a phone number format: (XXX) XXX XXXX.
Date
- Date Picker – Displays a calendar for the end user to select a date. You can make this a date range, where the reviewer will select a range of dates (for example years in college).
Currency
- Currency– This will provide a small text box that restricts the user to type only numbers. The format displayed upon entering will always be $XX.XX.
Number
- Number – Prevents reviewer from adding anything but a numeric value to the response.
Choice
- Dropdown – This displays a dropdown list. This defaults to only allow one response to be selected by the reviewer.
- You can make any dropdown multi-select by clicking the gear icon on the question type and toggling on multi-select. A minimum and maximum number of selections can be set in the question properties.
- Point values can be added to this question type when selecting the “Custom” dropdown by toggling on “Point Value”.
- Single Select– Displays each answer option next to a radio button. The end user is only able to select 1 of the answer options.
- Point values can be added to this question type by toggling on “Point Value”.
- Multiple Select – Displays each answer option next to a checkbox. The end user can select multiple answer options.
- Point values can be added to this question type by toggling on “Point Value”.
- Additionally, minimum and maximum selections can be set in the question properties.
Build your scorecard
To begin building your scorecard, add a section title by clicking on Add Section Title.
Note – Section Titles are not required, but if you want to leave this blank be sure to clear “Add Section Title”. If you do not want score questions grouped by a section, skip this step and continue to 2.
A section description can be added by clicking “Add sublabel or context”. This is not required and if not entered, will not show to the reviewer.
To add questions to that section, drag and drop your desired question type from the left side Elements list into the section. There is no limit to the number of questions within a section.
Add your question title by selecting“Type a question” and updating to your question text. If you would like to add a description or sublabel to the question, select “Add sublabel or context”. This may be used to display a question weight to a reviewer.
Questions will default as “Required”, but you can make a question optional by toggling off “Required”.
If you have similar questions across your entire scorecard, you can clone a scorecard question by clicking the clone icon. It will clone the question and all of its settings directly beneath the original question.
For some question types like Dropdown, there are additional settings available (like the ability to multi-select options from the list). To access these settings, click on the gear icon on the question.
The settings available here depend on the question type. For short input text fields like the example shown here, there is an option to add placeholder text.
Choice Question Types
The most used question type for a scorecard, choice question types give you the option to add point values to your scorecard so that your reviewers’ responses equal a certain point total. The choice question types are Dropdown, Single Select, and Multi-Select.
Dropdown
Dropdown question types display a list of questions where the user can select one option. When you add a dropdown question to your scorecard you have the option to build a custom list or use one of Kaleidoscope’s pre-defined lists.
Using a pre-defined list allows you to:
- Add additional answer options by clicking (+ Add Option)
- Remove an answer option by clicking on the small X on the right of the answer option.
- Edit an answer option by clicking on the small pencil icon next to the option.
Using a custom list allows you to create a list unique to your program and scorecard. You can enter as many question options as needed.
Single Select
Single Select question types display a list of responses, created by you, with radio buttons. Only one response can be selected by a reviewer for this question type.
Multiple Select
Multiple Select question types display a list of responses, created by you, with checkboxes. More than one response can be selected by a reviewer for this question type. You may also add a minimum and maximum selection option in the question settings.
Add Point Values to Choice Question Types
Add point values to each answer option by toggling the point value to the right. Point values are active when the bar turns green and an additional column for point values appears next to each answer option.
Points can be added in the number input box next to the answer option.
- In the example below the points entered here are telling us:
- When a reviewer selects “Strongly Agree”, 5 points will be added to the total score of the application being reviewed.
- If a reviewer selected “Neutral”, then 3 points will be added to the total score of the application being reviewed.
You also have the option to add a weight the point total added to each question response. For example, by adding a 200% weight to the above question, “Strongly Agree” is now worth 10 points.
To add weight to a question, click the settings gear on the question and toggle on “Question Weight”and enter the desired question weight percentage.
Score Sharing & Reviewer Notes
If you would like reviewers to collaborate on scores, you can opt to turn on score sharing for questions on the scorecard. Turning on score sharing will allow reviewers scoring the same application in a round to view the responses given by each other to that scorecard question for that application.
To turn on score sharing for a question, click the settings gear on the question. Toggle the “Score sharing” option on (green).
When turned on, reviewers will be able to click a button on the scorecard to review responses made by other reviewers in the round.
To share reviewer notes, add a long text question type to your scorecard and turn on score sharing. Invite your reviewers to add notes for other reviewers in this long text field.
Preview Scorecard
Once you have completed your reviewer scorecard, preview the scorecard to confirm it looks and acts as you wish.
Click the “Preview Scorecard” button at any time to view and test your scorecard. Point values will tally as you select options in the scorecard preview.
Rank Round
On a rank round, you will establish what application answers you would like your reviewers to focus on when ranking applicants against one another.
To select which application answers you would like reviewers to rank on, check the box next to the application question in the left table.
Click the arrow to transfer the question from the left table to the right table - the Reviewer Columns table.
You can update the label of the reviewer column to make it easier for reviewers to read. You can also move columns around using the up/down arrows.
Note: If any video, image, or document questions are added to the Reviewer Columns table, the reviewer will be able to click a link to view the video, image, or document within the application. This makes for easy review and rank.
Finally, you can allow reviewers to add optional comments on applications by toggling the "Allow Optional Reviewer Comments" on.
Preview Scorecard
Once you have completed your reviewer scorecard, preview the scorecard to confirm it looks as you wish.
Click the “Preview Scorecard” button at any time to view your scorecard.