A Reviewer’s role in a PICO Portal project refers to a person who is responsible for Abstract Screening, Full-Text Review, and Data Extraction. A Judicator’s role in a PICO Portal project refers to a person who is responsible for adjudicating conflicting decisions on citations and providing the final consensus when extracted data items differ. The responsibilities and capabilities the user has depends on the specific role(s) to which the user has been assigned.
See Invite Members To Your Project for more information on User Roles.
Project Dashboard – Manage Projects #
When a Reviewer or Judicator logs in to PICO Portal, the Manage Projects dashboard appears and provides a list of their projects with their various metrics, such as the project name, the number of files imported, the creation date and due date, project lead name, and the number of team members.
Project Dashboard – Main Dashboard [Owner or Co-Owner] #
When an Owner or Co-Owner selects a project, they are redirected to a main project dashboard. By default the project appears with the following information:
- Project number and title: Displays the project number assigned by PICO Portal, as well as the title of the project.
- Project Description: Displays the description of the project, as entered by the project owner.
- Project Files: Project-specific or related files, such as the protocol, can be uploaded and viewed here.
- Help: Displays helpful information regarding PICO Portal, including a step-by-step introduction to some of the main elements of the Dashboard, a direct link to the docs portal (Wiki), and a direct link to the contact page.
- Abstract Screening: This displays the number of articles imported, the articles pending screening, as well as the number of articles that have been Included and Excluded by all team members. It also displays information regarding Pending Conflicts, Duplicates that need to be reviewed, and Supplemental Records that have been identified and need to be reviewed. Owners and Co-Owners can click on the number under Pending Screening to begin Abstract Screening, or the number beside Pending Conflict, Duplicates, and Supplemental Records to review these articles.
- Full-Text Review: This displays the number of articles pending review, the number of articles that have been Included and Excluded by all team members. It also displays information regarding Pending Conflicts and Reports not Retrieved. Owners and Co-Owners can click on the number under Pending Review to begin Full-Text Review, or the number beside Pending Conflict and Reports not Retrieved to review these articles.
- Risk of Bias: This displays the number of articles pending Risk of Bias assessment, as well as the number of articles that have had their assessment completed. It also displays the number of completed risk of bias assessments that are In Review and need to be reviewed by the Judicator.
- Data Extraction: This displays the number of articles pending Data Extraction, as well as the number of articles that have their data extraction completed. It also displays the number of completed data extractions that are In Review and need to be reviewed by the Judicator.
Additional information available to the Owner or Co-Owner:
- Task Workflow: Single Review or Dual Independent Review.
- Task Due Date: Listed on each aspect of the project, if enabled.
- Abstract Screening/Full-Text Review Metrics: Both display additional information regarding the total number of articles screened per hour, the total hours spent screening/reviewing, and the total estimated hours needed to reach completion of the screening/reviewing process.
- ARCTIC Notifications: ARCTIC is the AI engine used by PICO Portal. This lists the various automatic updates that the system runs on the project, including identifying duplicates, creating tag highlights, and exporting files to EndNote.
- Team Member Metrics: Displays each team member’s total articles screened so far, their total hours spent on the project as a whole, their average accuracy, and their last login.
Note: If you are the Owner, or are assigned the role of Co-Owner, you are able to toggle the Reviewer dashboard ON or OFF as needed in the top right corner of the project dashboard. However, it is not necessary to toggle the Reviewer dashboard to ON if you want to screen/review. If you would like to screen/review, click on the number under Pending Screening or Pending Review. If you click Close in the top right corner of the screening window, you will be automatically directed to the Reviewer dashboard. To get back to the Owner or Co-Owner dashboard, click on Close again in the top right corner.
Project Dashboard – My Articles [Judicator] #
When a Judicator selects a project, they are redirected to the My Articles dashboard. By default the project appears with the following information:
- Project number and title: Displays the project number assigned by PICO Portal, as well as the title of the project.
- Project Description: Displays the description of the project, as entered by the project owner.
- Project Files: Project-specific or related files, such as the protocol, can be uploaded and viewed here.
- Help: Displays helpful information regarding PICO Portal, including a step-by-step introduction to some of the main elements of the Dashboard, a direct link to the docs portal (Wiki), and a direct link to the contact page.
- Abstract Screening/Full-Text Review – My Adjudication: This displays the number of pending conflicts, the number of articles you have adjudicated (My Adjudication), as well as the number of articles you have Included and Excluded based on the adjudicated decision. Judicators can click on the number beside My Pending Conflict to begin the adjudication process.
- Risk of Bias and Data Extraction: The pending conflicts needing to be adjudicated for Risk of Bias and Data Extraction will display as Pending Judication, with the number of articles you have adjudicated displaying as My Articles.
Project Dashboard – My Articles [Reviewer or Principal Reviewer] #
A Principal Reviewer is a special role for a Reviewer that is required to look at all records if the relevant flow is selected in setup (e.g., Dual Review with Principal Reviewer). For example, in a set of 500 citations with 6 people on a team with dual review, if one person is a Principal Reviewer, that person’s input/decision is required on all 500 citations, and the other members will contribute to the “second” decision on each citation as they screen on a first-come basis.
If there are multiple Principal Reviewers, only one of them is required to look at a citation. This is a role reserved for content experts or situations where one or more people may be “leading” the screening and want to have a say on each citation.
The role is also applicable to the Risk of Bias and Data Extraction stages if the relevant project setup is selected (e.g., Single review with Quality Check). In this case, the Principal Reviewer will be the Judicator.
When a Reviewer selects a project, they are redirected to the My Articles dashboard. By default the project appears with the following information:
- Project number and title: Displays the project number assigned by PICO Portal, as well as the title of the project.
- Project Description: Displays the description of the project, as entered by the project owner.
- Project Files: Project-specific or related files, such as the protocol, can be uploaded and viewed here.
- Help: Displays helpful information regarding PICO Portal, including a step-by-step introduction to some of the main elements of the Dashboard, a direct link to the docs portal (Wiki), and a direct link to the contact page.
- Abstract Screening/Full-Text Review: This displays the number of articles pending screening/review, the number of articles you have screened/reviewed, as well as the number of articles you have Included and Excluded. Reviewers can click on the number beside Pending Screening to begin Abstract Screening, or Pending Review to begin Full-Text Review.
See Screening Page Elements for more information.
Depending on the role assigned and the permissions granted, the dashboard may appear differently from project-to-project:
- If you are a Reviewer, PICO Portal will display the citations that are pending review.
- If you are a Judicator, PICO Portal will display conflicting citations for final consensus.
- When you are both a Reviewer and a Judicator, PICO Portal will display the citations that are pending review and the conflicting citations for final consensus. When a user with a dual role starts adjudicating conflicting citations, the user will see the other Reviewer’s name even though the option to hide the Reviewer’s name is enabled. This is because the user has to contact the other Reviewer to discuss the selections and make a final consensus.
- If you are a Principal Reviewer, PICO Portal will display the citations that are pending review. A Principal Reviewer is required to review all of the citations in the project.
- If you are the project Owner or Co-Owner, PICO Portal will first assign the role of a Reviewer and then as a Judicator after all the citations are reviewed. If the workflow requires a Principal Reviewer, then the Owner and the Co-Owner are assigned the role of a Principal Reviewer and not of a Reviewer.
- If you are assigned the role of Co-Owner, you are able to toggle the Reviewer dashboard ON or OFF as needed in the top right corner of the project dashboard.
Additional information may be displayed depending on which study components are active in the project setup. These can be toggled ON or OFF at any time during the project:
- Risk of Bias: This displays the number of articles Pending Risk of Bias assessment, as well as the number of articles you have completed. Reviewers can click on the number beside Pending Risk of Bias to start the Risk of Bias assessments.
- Data Extraction: This displays the number of articles Pending Data Extraction, as well as the number of articles you have completed. Reviewers can click on the number beside Pending Data Extraction to start the Data Extraction process.
- Team Members: Displays a list of team members along with their roles and their last login information. It also provides different metrics of each team member, however, these metrics are not available to all team members.
- Duplicates: You can review the duplicate citations and mark them as duplicate or not duplicate using this screen.
See Manage Project for more information on Duplicates.
Other metrics that may be available:
- Task workflow: Single Review or Dual Independent Review.
- Your personal total hours spent screening.
- Your average number of articles screened per hour.
- Your average accuracy in screening.
See Screening Page Elements and Extract Data for more information
My Reviews #
My Reviews displays a list of all the citations that you have reviewed or adjudicated. You can sort the list and click the citation to view the details.
To access your citations, click on the number beside My Screening/My Articles/My Reviews/My Adjudication and scroll down to see the list. The citations can also be searched by CiteID, DOI, Author, Title, and Source. You can also view your citations based on whether they have been Included or Excluded by clicking on the number associated with Included (green) or Excluded (red).
See View My Articles for more information.
To go back to the Manage Projects dashboard, click on Projects in the top left corner of the screen beside the PICO Portal logo, or Close in the top right corner.
