Template for NTP Powerpoint Presentations (PPTx)
Systematic Review in the Environmental Health Sciences: Current Tools and Software Vickie R. Walker Office of Health Assessment and Translation National Institute of Environmental Health Sciences (NIEHS)
NTP Office of Health Assessment and Translation
• Conducts literature-based evaluations to assess the evidence that environmental chemicals, physical substances, or mixtures cause adverse health effects • Evaluation format depends on purpose and extent of the evidence (e.g., scoping or full systematic reviews) • Serves as an environmental health resource to the public and to regulatory and health agencies
NTP Monographs NTP Reports
Workshops
Use Of Software Tools In Systematic Review Process
• Problem Formulation • Search & Select • Data Extraction
• Study Quality • Synthesize Evidence • Develop Conclusions
Problem Formulation How to Sort Through the Evidence for Decision Making
Goal: To develop an addressable systematic review question by surveying the extent, range, and nature of research activity for a given topic.
Key Considerations: – – – –
What are the evidence streams? What types of studies are published? Did the study report an effect or not? What is the quality of the studies?
Exposures & Outcomes
• Categorize studies using filters • Article type • Exposure • Evidence stream • Health outcomes
Aspirin Motrin NSAIDS Tylenol acetaminophen ibuprofen indomethacin naproxen paracetamol [No Tag]
• Priority rank studies • Visualization of filters
Available for free (http://www.sciome.com/swift-review/)
Hematological Reproductive and and Immune Developmental 2610 3476 4 7 146 376 1 10 148 511 173 543 1024 3188 101 249 146 457 52 136
Active Exploration of Evidence “Pockets” Identify studies of specific exposures, outcome or evidence stream
OUTCOME
CHEMICAL
EVIDENCE
Visualization Options
Priority Ranking Feature Confirm Relevance for Exposure and Health Outcome(s) of Interest
Manual Identification +
Search and Select: Screening Studies Goal: Identify relevant studies related to the focused systematic review question. Screening is completed in two phases, title and abstract screening and full text review using inclusion/exclusion criteria.
Key Considerations: – – – – –
Transparency in methods Data management Multiple reviewers Screening conflict resolution features Tracking Progress
• Web-based tool • Features the ability to have multiple reviewers and conflict resolution management • Active learning continually updates the document prioritization model in response to feedback from the user • Unlike standard machine learning approaches, active learning does not require a large training set • Suggests when screener can stop based on desired recall level (e.g., 95%)
Customizable Screening Questions Active Machine Learning
Tracking Progress Visuals to help illustrate progress and results
Screening
• Web-based systematic review tool • Features multiple reviewers and conflict resolution • Features built in real time reports including PRISMA, which means that you know your study’s precise status at all times. • Flexible screening form creation
https://www.evidencepartners.com/products/distillersr-systematic-review-software/
Customizable Screening Questions
Tracking Progress
Screening & Data Extraction
• Web-based content management system designed to create, store, share and display data results
• Provides a workplace for multiple users on a single project • Features various modules: – – – –
Study Management Screening and Review Data Extraction and visualization (Epidemiological, animal and in vitro) Risk Of Bias
Available for free https://hawcproject.org/
Search and Select: Screening
Screening Visualizations
WHO/IARC Monograph 112
Animal Data Extraction Transparent and publicly available Experimental protocol and dose regimen Endpoint Summary
n
Animal Data Visualization
Risk of Bias Analysis Individual study risk of bias
Summary of risk of bias by metric
Heatmap risk of bias for study collection
Summary • A systematic review and evidence integration framework – Increases transparency and objectivity in literature evaluations
– Methods are available to address environmental health questions • OHAT Approach and Handbook (https://ntp.niehs.nih.gov/go/38673)
• Tools add transparency to multiple steps – Problem Formulation, Screening, Data extraction and Risk of Bias – Tools ultimately contribute to transparency in the review
• Example - NTP monographs • Website https://ntp.niehs.nih.gov/go/749926 • Data in HAWC https://hawcproject.org/assessment/57
Thank You
NTP Office of Health Assessment and Translation
• Conducts literature-based evaluations to assess the evidence that environmental chemicals, physical substances, or mixtures cause adverse health effects • Evaluation format depends on purpose and extent of the evidence (e.g., scoping or full systematic reviews) • Serves as an environmental health resource to the public and to regulatory and health agencies
NTP Monographs NTP Reports
Workshops
Use Of Software Tools In Systematic Review Process
• Problem Formulation • Search & Select • Data Extraction
• Study Quality • Synthesize Evidence • Develop Conclusions
Problem Formulation How to Sort Through the Evidence for Decision Making
Goal: To develop an addressable systematic review question by surveying the extent, range, and nature of research activity for a given topic.
Key Considerations: – – – –
What are the evidence streams? What types of studies are published? Did the study report an effect or not? What is the quality of the studies?
Exposures & Outcomes
• Categorize studies using filters • Article type • Exposure • Evidence stream • Health outcomes
Aspirin Motrin NSAIDS Tylenol acetaminophen ibuprofen indomethacin naproxen paracetamol [No Tag]
• Priority rank studies • Visualization of filters
Available for free (http://www.sciome.com/swift-review/)
Hematological Reproductive and and Immune Developmental 2610 3476 4 7 146 376 1 10 148 511 173 543 1024 3188 101 249 146 457 52 136
Active Exploration of Evidence “Pockets” Identify studies of specific exposures, outcome or evidence stream
OUTCOME
CHEMICAL
EVIDENCE
Visualization Options
Priority Ranking Feature Confirm Relevance for Exposure and Health Outcome(s) of Interest
Manual Identification +
Search and Select: Screening Studies Goal: Identify relevant studies related to the focused systematic review question. Screening is completed in two phases, title and abstract screening and full text review using inclusion/exclusion criteria.
Key Considerations: – – – – –
Transparency in methods Data management Multiple reviewers Screening conflict resolution features Tracking Progress
• Web-based tool • Features the ability to have multiple reviewers and conflict resolution management • Active learning continually updates the document prioritization model in response to feedback from the user • Unlike standard machine learning approaches, active learning does not require a large training set • Suggests when screener can stop based on desired recall level (e.g., 95%)
Customizable Screening Questions Active Machine Learning
Tracking Progress Visuals to help illustrate progress and results
Screening
• Web-based systematic review tool • Features multiple reviewers and conflict resolution • Features built in real time reports including PRISMA, which means that you know your study’s precise status at all times. • Flexible screening form creation
https://www.evidencepartners.com/products/distillersr-systematic-review-software/
Customizable Screening Questions
Tracking Progress
Screening & Data Extraction
• Web-based content management system designed to create, store, share and display data results
• Provides a workplace for multiple users on a single project • Features various modules: – – – –
Study Management Screening and Review Data Extraction and visualization (Epidemiological, animal and in vitro) Risk Of Bias
Available for free https://hawcproject.org/
Search and Select: Screening
Screening Visualizations
WHO/IARC Monograph 112
Animal Data Extraction Transparent and publicly available Experimental protocol and dose regimen Endpoint Summary
n
Animal Data Visualization
Risk of Bias Analysis Individual study risk of bias
Summary of risk of bias by metric
Heatmap risk of bias for study collection
Summary • A systematic review and evidence integration framework – Increases transparency and objectivity in literature evaluations
– Methods are available to address environmental health questions • OHAT Approach and Handbook (https://ntp.niehs.nih.gov/go/38673)
• Tools add transparency to multiple steps – Problem Formulation, Screening, Data extraction and Risk of Bias – Tools ultimately contribute to transparency in the review
• Example - NTP monographs • Website https://ntp.niehs.nih.gov/go/749926 • Data in HAWC https://hawcproject.org/assessment/57
Thank You
