GOapply Reviewer Admin Guide
GOapply applications can be reviewed by both internal and external reviewers using the GOapply review features. These features allow the user to create and manage specific review groups for particular phases of their submitted applications. In addition, GOapply Users can be made reviewers and can be added to as many review groups, as necessary.
The GOapply review features are divided into several components, which can be found on the left-hand navigation pane in the GOapply workspace.
OVERVIEW OF GOAPPLY REVIEWER COMPONENTS
GOapply Reviewer is made up of many of the same entities as GOapply. Users, Applications, and Status Tracking are all employed in Reviewer in addition to several Reviewer-specific entities. These include Reviewers, Review Groups, Review Group Applications, and Reviewer Request Score. Below is a high-level overview of Reviewer entities; we will dive deeper into these topics later in this document.
Your reviewers will register for a GOapply account independently. This process is the same as applicant registration, which we will walk through step-by-step later in this document.
From the Application, you can see which Review Groups it is attached to.
Status Tracking records are what informs Reviewer which phases/applications are ready for review.
Reviewers are all GOapply Users that have the Reviewer or Applicant and Reviewer role assigned.
Reviewers will not have access to any applications to review until they have been added to a review group.
Review Group Applications use the GOapply Status Tracking record to display the application for the reviewer. If the GOapply Status Tracking record does not exist or does not contain a PDF application, the application will not display correctly in the reviewer dashboard.
GOapply Admins can upload an alternative file to be displayed for the reviewer. In the Status Tracking Record, there is an upload field called Reviewer File. Here is where you would upload what you want displayed to the reviewer if it is different from the submitted PDF. This file will override the submitted PDF.
Reviewer Request Score records allow you to see the score that a reviewer gave. These records also contain the Reviewer Form Response json file. This json file is a string of code; it is similar to the Form Response in GOapply Status Tracking records.
In all areas of akoyaGO CRM, you have a system dashboard called GOapply Application Review that tracks applications that need to be assigned to a review group, applications with a conflict of interest, applications with scores from review groups, and active reviewer request scores.
Now that you know how the GOapply Reviewer entities work together, let’s get started creating Review Groups and adding Reviewers.
Each review group consists of general info (including the application and phase), settings to determine what reviewers can see, the reviewers themselves, and review group applications. All of these components are necessary for reviewers to have everything they need.
The following steps detail the process to create a completed review group.
1. From the Review Group entity, click “New Review Group.”
The New Review Group form will load.
2. Name your review group.
3. Select the application and phase that the review group will be assigned to.
4. Set a review group deadline in the Review By field. Once the Review By date has passed, the review group will no longer appear in the reviewer’s list in GOapply.
5. Click Save. Upon save, the Review Form Definition field will populate with its unique string of code and lock to prevent accidental deletion. To download the Review Form Definition, click the link in the field.
Click the Settings tab of the newly created Review Group.
The Settings tab contains various customization options for each review group. These settings involve the information you wish to share with the reviewers. All but Anonymous Review will default to Yes. Anonymous Review will default to No. Scoring Type will default to Sum (total score).
Show Other Reviewers Comments: If Yes, the Reviewer Comments button will be visible on the application review form. If set to No, the Reviewer Comments button will not be visible on the application review form. Please note that if Show Other Reviewers Comments is set to No, the Show Reviewer Weighted Score setting is rendered null, and the Public setting in Form Builder (page 10 of this guide) will also be null.
If Show Other Reviewers Comments is Yes:
Show Mark for Conflict of Interest: If set to Yes, the Conflict of Interest checkbox will appear on the Reviewer’s review form. If set to No, the Conflict of Interest checkbox will not be visible on that form.
Anonymous Review: Select Yes if you want applicants to appear anonymous. If Yes is selected, the applicant’s name will be removed from the PDF that the reviewer sees. The reviewer PDF will be stored in a field called Reviewer File in the GOapply Status Tracking record.
Show Reviewer Weighted Score: If Yes, reviewers will be able to see how other reviewers scored the application. It renders as the far right column in the Reviewer Comments. If No, the weighted score column is not visible.
Scoring Type: Choose Sum or Average.
- If Sum (total score), each question will be equally weighted, and the reviewer score will be the total of the reviewer’s answers. With sum selected, you can still set the question’s value in the Choices section of properties in Form Editor. See Set Scoring section of this document for more information.
- If Average, you will be able to set weight of each question in that question’s properties, akoyaGO section. More information can be found in the Question Properties section of this document.
Two flows exist to duplicate a review group. To run the flow, select the review group, click Flows, and choose either Duplicate Review Group or Duplicate Review Group and Reviewers.
Duplicate Review Group: Duplicates application, phase, review by date and time, review form definition, review form and all properties, all settings, show staff comments and attachments setting, and staff notes
Duplicate Review Group and Reviewers: Duplicates application, phase, review by date and time, settings, show staff comments and attachments setting, staff notes, reviewers
Please note the staff notes and staff attachments are NOT duplicated with these flows.
Reviewers are the GOapply users that will be evaluating the applications in their review groups. Reviewers must be added to each review group they are reviewing applications for. In order to be added to a Review Group, the user(s) must have the Reviewer or Applicant and Reviewer role.
From the Reviewers tab of the review group, click New Reviewer in the All Reviewers subgrid.
The New Reviewer form will populate with the review group information. Review Complete and Lead Reviewer will default to No.
Select a GOapply User in the Reviewer field. If the user is the lead reviewer, make that designation in their reviewer record. Lead Reviewers will be able to see all reviewers' comments regardless of the Show Reviewers Comments setting in the review group settings. Please note that the intended functionality is to have one lead reviewer per review group. If you try to set another reviewer as lead reviewer, it will override the previous lead reviewer.
Review Complete: this field will automatically flip to Yes when the Reviewer has completed reviewing all the applications in all the review groups that they are assigned to. It will default to No upon implementation, so if you are in the middle of a review cycle on 8/4/2022, you will need to manually flip Review Complete to Yes for that cycle.
The Reviewers tab of the Review Group record are three subgrids: All Reviewers, In Progress, and Complete.
- All Reviewers shows the Reviewers assigned to the group and includes a Review Complete column that will read Yes or No.
- In Progress shows the Reviewers who have started the review process but have not submitted all their assigned reviews.
- Complete shows the Reviewers who have completed all their assigned reviews.
The Staff Attachments and Comments tab in the Review Group record allows for staff notes that apply to ALL applications for that review group. Attachments can also be added here as public documents for the review group.
These notes and attachments are displayed above the review group applications.
Review forms are displayed for the reviewer side by side with the application. Question types include a text field, a radio button, a drop down, a long text comment, and a rich text editor. For example, below you’ll see the submitted application (or whatever is in the Reviewer file tab of the Status Tracking record) on the left, and the reviewer’s questionnaire on the right. At the bottom of the page they can save a draft or submit.
From the review group, navigate to the Form Builder tab. Select Launch Form Editor.
The Form Editor will load in a separate tab.
Unlike the form editor in GOapply, the form editor in Reviewer is only able to support one page.
Please ensure you design your form to account for this. You will be familiar with this Form Editor; it is the same one you use to create your GOapply Applications. On the right-hand side, you will see the 5 criteria types described above. Drag and drop the question types into the form to add.
There are five types of question: a text field, a radio button, a drop down, a long text comment, and a rich text editor. Please note that only Radiogroup and Drop-Down field types can have score values assigned.
Single Input: a text or number response field. These fields cannot be used to calculate scores.
Radiogroup: a single selection radio button style multiple choice. These responses can have score values assigned.
Drop Down: a drop-down list of responses. These responses can have score values assigned.
Comment: a long text or number response field. These fields cannot be used to calculate scores.
Editor: a long text or number response field, with rich text formatting. These fields cannot be used to calculate scores.
If the question type is a type used to calculate reviewer scores, the scoring must be included in the choices for that question.
Each question field has a properties menu. Open properties by clicking on the question and selecting Properties from the far right.
Common settings for each criteria include:
Required: Required setting can be found in the General dropdown in the form of a toggle called “Is Required?” If on, these criteria must be completed for the reviewer to mark the review as “complete.” Required criteria will be denoted with an asterisk in the review form. Required will default to no.
Weight: Weight can be found under akoyaGO. This functionality is only available if the Scoring Type in the Review Group’s setting is set to Average. If the criteria are averaged, enter the weight of the score. Note: if averaging is used, the weights of all criteria must total 100.00.
Public: This setting can be found in the akoyaGO dropdown. If checked, the criteria responses will be shared with the review group in the Reviewer Comments tab. Note: the review group setting Show Other Reviewers Comments must also be turned on for this to appear. Lead Reviewer will be able to see all reviewer responses.
Scoring can be set on the dropdown or radiogroup question types only.
1. In the Properties for the scoring field, drop down Choices
2. For Value, enter the option, then two colons, then the numerical value of that choice. The proper format for this formula is no spaces. For example, Agree::80. In the Text field, simply enter the option that you want the reviewer to see. In this example, you’d just put Agree. Repeat this process for each option in the drop down.
If your Review Group settings designate the scoring type as Average, you will see a property called Weight under the akoyaGO section. Here is where you will set the weight of each scored question. Weight is only available for drop down and radiogroup question types.
Now that you have created your review group, designated reviewers, and created the review form, it is time to add the submissions for the reviewers to review.
This is done through the entity called Review Group Applications. In that entity is a view called “GOapply Reviewer”. Through this view, you can manipulate the applications that a reviewer sees. For example, deactivating a record in that view would remove it from the list of applications that the reviewer sees.
Next, you’ll assign the submitted applications to the review group.
Review Group Applications is an entity in akoyaGO CRM and uses Status Tracking records to determine which submissions get shown to the reviewers.
Bulk add submissions from Requests entity:
You will only want to add applications this way if you have more than 1 review group reviewing the set of applications. This will allow you to select a set of Request from a view and add them to a review group.
Step 1: Copy the name of the Review Group because you will need to enter it correctly in a workflow dialog.
Step 2: You can go to an individual Request record or a view to choose multiple to add
Step 3: Go to Flow > GOapply Add Request to Review Group
Step 4: Click Continue in the first box that pops up
Step 5: In the next window paste in the name of the Review Group and click Continue.
1. Starting from the submitted Status Tracking record, click on the Related tab, and choose Review Group Applications.
2. Click +New Review Group Applications
3. The New Review Group Application form will open and autofill the Request and Status Tracking field. All you need to do is assign the Review Group. Save.
Now, the submission(s) you just added will appear in the review group’s dashboard in GOapply. Repeat this process for each submission that the review group needs to review.
Reviewer responses are recorded in the Review Responses entity, Reviewer Request Score entity, in the Reviewer Scores tab of the Review Group Application entity, and in the Responses tab of the Reviewer record.
Review Responses is an entity that stores every response from each reviewer. For example, if your reviewer form has five questions, every time a reviewer submits the form, five reviewer response records will be created.
Inside a Review Response record, you will see:
- Question: The name of the question
- Criteria: GOapply Version 2 Reviewer does not use Criteria, so that field will be blank for the time being.
- Response: The choice selected or the comment entered by the reviewer
- Reviewer: A lookup to the Reviewer record
- Review Group: A lookup to the Review Group record
- Status Tracking: A lookup to the associated Status Tracking record
- Public: Whether this question was Public or not (a property in Form Editor)
- Score: What the associated score was for the choice made by the reviewer
- Weight: The weight that this question carries
- Conflict of Interest: Whether or not this submission was a conflict of interest
REVIEWER REQUEST SCORE
Reviewer Request Score records contain:
- Reviewer: A lookup to the Reviewer record
- Review Group Request: A lookup to the Review Group Application record
- Reviewer Form Response: A reviewer form response json file
- Score: The score attributed based on the reviewer’s answer to the question
Reviewer responses and scores can also be seen from within the Reviewer record.
In the Review Groups entity, you have a report called GOapply Review Responses that you can run to see each reviewer’s responses for each application assigned to their group.