TOXMAP - GIS Gateway to Environmental Health Databases
TOXMAP - GIS Gateway to Environmental Health Databases 2005 ESRI Health User Conference October 25, 2005 http://toxmap.nlm.nih.gov
National Institutes of Health, U.S. Dept. of Health and Human Services
Authors • Colette Hochstein, D.M.D., MLS National Library of Medicine (NLM) TOXMAP Project Officer • Darren Gemoets Aquilent, Inc. Senior Software Engineer
National Library of Medicine Located at the National Institutes of Health in Bethesda, Maryland
Toxicology and Environmental Health Program (TEHIP) …provide core toxicology and environmental health information resources and services
Toxics Release Inventory (TRI)
Iterative Approach
Single chemical, single year
Trends – single chemical
Population Overlay
TOXMAP Architecture • Architecture defined before ArcIMS chosen • ArcIMS best fit • Standard browser • Server designed to run on Windows, Solaris, or Linux • Java & Open Source • Web services for growth
Evaluation: Focus Groups
What is a Focus Group? • A carefully planned discussion • A data collection procedure • With 7 to 12 people and a skilled interviewer • Informal sharing of ideas/perceptions
Benefits of Focus Groups • • • • • •
Quick identification Time efficient, cost effective Comfortable format Builds rapport Enables observation Written transcript
However… • Focus groups assess only what customers say they do • There are often major differences between what people say and what they do • Focus groups should not be used as the only source of usability data • They should not be used to measure interface usability
Uses of Focus Groups • • • • • • • • •
Decision making Product/program development Evaluation of user satisfaction Planning and goal setting Needs assessment Quality improvements Organizational learning Policy making and testing Research tool
When? Before, during, or after …. • Before: Needs Assessment, asset analysis, “climate survey,” planning, and pilot testing • During : Formative evaluation, feedback, monitoring, reporting, process evaluation • After: Summative evaluation, outcome evaluation, feedback
Who? • What type of people would give you the information you want? • Who are the target audiences? • What characteristics should the participants have? • How do you identify information-rich people?
How? • Do it yourself
• Use a professional moderator
Our First Approach Teleconference format: • Two sessions with professional colleagues • Participants reviewed a web-based “TOXMAP Tour” before the sessions • Followed a discussion guide we developed ourselves • Recorded the sessions (ourselves/audio)
Challenges Time • We made all contacts ourselves • We developed all related resources ourselves • We had to practice (we’re not professional moderators)
Expertise • We contacted those we already knew • We asked questions we thought would give us the info we wanted • We got an audio transcript only (not written)
One Year Later… • We had acquired NIH evaluation funds to study TOXMAP • We had experience with professionally moderated web-based focus groups that had been conducted on several of our other resources • We began the process of planning professionally moderated focus groups
Our Objectives
•
To assess the strengths and weaknesses of the TOXMAP web site
•
To better understand how respondents navigated TOXMAP
•
To gauge reactions to potential features of TOXMAP that were presented in mock-up format
Our Participants • Two groups of 9–12 each (professionals and concerned citizens) • Balance of adult men and women • Employed in the environmental, occupational, industrial, and public health and information fields • Actively participate in community or national activities related to environmental health
We Screened Potential Participants
We Used a “Tour”
We Gave Homework!
We Used Mock-Ups
We Had a Professional Moderator • Followed a preplanned script of specific issues and set goals for the type of information to be gathered. • Kept the discussion on track without inhibiting the flow of ideas and comments. • Ensured that all group members contribute to the discussion and must avoid letting one participant's opinions dominate. • Wrote a short report summing up the prevailing mood in the group, illustrated with a few quotes.
We Read the Report
We Studied the Logs
We “Mapped Out” Our Goals
We Determined Our Next Steps….
Multiple chemicals displayed on a single map
Superfund/NPL sites
Occupation Maps
Male Lung Cancer data with… TRI Sites
TRI Chromium Sites
Sample Timeline • Determine your objectives Conduct the Focus Group! • Decide •proposal Draft will discussion give you the guide • Obtainwho from selected information (moderator you want orparticipants client) consultant or• Remind develop your own outlining all topicstool; to be objectives • Web-based and timing • Develop aproposal recruitment • Reviewcovered for:thescreening teleconference, etc. associated withinformation each questionnaire (provides used • Timing • Sessions normally held to select participants) • Cost • Prepare materials to use at 6 & 8pm and last for • Details (location, where the during the session one hour and forty-five • Begin participant recruitment study will occur, recruitment minutes • Select a general moderator and a specifications, topics • Determine what/if incentive can/will be to berecorder covered) • Record the session offered
Informal
Formal
Less expensive More control over selection of participants
More expensive Less control over selection of participants
Smaller potential participant Larger potential participant base base More paperwork Less paperwork (moderator) Formal report depends on YOUR notes
Prompt, professional formal report
May be viewed as less objective by participants (because YOU are conducting)
Often viewed as more objective- the moderator is neutral
Focus Groups Have Helped Direct Our Growth
For More Information … Colette Hochstein [email protected] General Questions [email protected]
http://toxmap.nlm.nih.gov
National Institutes of Health, U.S. Dept. of Health and Human Services
Authors • Colette Hochstein, D.M.D., MLS National Library of Medicine (NLM) TOXMAP Project Officer • Darren Gemoets Aquilent, Inc. Senior Software Engineer
National Library of Medicine Located at the National Institutes of Health in Bethesda, Maryland
Toxicology and Environmental Health Program (TEHIP) …provide core toxicology and environmental health information resources and services
Toxics Release Inventory (TRI)
Iterative Approach
Single chemical, single year
Trends – single chemical
Population Overlay
TOXMAP Architecture • Architecture defined before ArcIMS chosen • ArcIMS best fit • Standard browser • Server designed to run on Windows, Solaris, or Linux • Java & Open Source • Web services for growth
Evaluation: Focus Groups
What is a Focus Group? • A carefully planned discussion • A data collection procedure • With 7 to 12 people and a skilled interviewer • Informal sharing of ideas/perceptions
Benefits of Focus Groups • • • • • •
Quick identification Time efficient, cost effective Comfortable format Builds rapport Enables observation Written transcript
However… • Focus groups assess only what customers say they do • There are often major differences between what people say and what they do • Focus groups should not be used as the only source of usability data • They should not be used to measure interface usability
Uses of Focus Groups • • • • • • • • •
Decision making Product/program development Evaluation of user satisfaction Planning and goal setting Needs assessment Quality improvements Organizational learning Policy making and testing Research tool
When? Before, during, or after …. • Before: Needs Assessment, asset analysis, “climate survey,” planning, and pilot testing • During : Formative evaluation, feedback, monitoring, reporting, process evaluation • After: Summative evaluation, outcome evaluation, feedback
Who? • What type of people would give you the information you want? • Who are the target audiences? • What characteristics should the participants have? • How do you identify information-rich people?
How? • Do it yourself
• Use a professional moderator
Our First Approach Teleconference format: • Two sessions with professional colleagues • Participants reviewed a web-based “TOXMAP Tour” before the sessions • Followed a discussion guide we developed ourselves • Recorded the sessions (ourselves/audio)
Challenges Time • We made all contacts ourselves • We developed all related resources ourselves • We had to practice (we’re not professional moderators)
Expertise • We contacted those we already knew • We asked questions we thought would give us the info we wanted • We got an audio transcript only (not written)
One Year Later… • We had acquired NIH evaluation funds to study TOXMAP • We had experience with professionally moderated web-based focus groups that had been conducted on several of our other resources • We began the process of planning professionally moderated focus groups
Our Objectives
•
To assess the strengths and weaknesses of the TOXMAP web site
•
To better understand how respondents navigated TOXMAP
•
To gauge reactions to potential features of TOXMAP that were presented in mock-up format
Our Participants • Two groups of 9–12 each (professionals and concerned citizens) • Balance of adult men and women • Employed in the environmental, occupational, industrial, and public health and information fields • Actively participate in community or national activities related to environmental health
We Screened Potential Participants
We Used a “Tour”
We Gave Homework!
We Used Mock-Ups
We Had a Professional Moderator • Followed a preplanned script of specific issues and set goals for the type of information to be gathered. • Kept the discussion on track without inhibiting the flow of ideas and comments. • Ensured that all group members contribute to the discussion and must avoid letting one participant's opinions dominate. • Wrote a short report summing up the prevailing mood in the group, illustrated with a few quotes.
We Read the Report
We Studied the Logs
We “Mapped Out” Our Goals
We Determined Our Next Steps….
Multiple chemicals displayed on a single map
Superfund/NPL sites
Occupation Maps
Male Lung Cancer data with… TRI Sites
TRI Chromium Sites
Sample Timeline • Determine your objectives Conduct the Focus Group! • Decide •proposal Draft will discussion give you the guide • Obtainwho from selected information (moderator you want orparticipants client) consultant or• Remind develop your own outlining all topicstool; to be objectives • Web-based and timing • Develop aproposal recruitment • Reviewcovered for:thescreening teleconference, etc. associated withinformation each questionnaire (provides used • Timing • Sessions normally held to select participants) • Cost • Prepare materials to use at 6 & 8pm and last for • Details (location, where the during the session one hour and forty-five • Begin participant recruitment study will occur, recruitment minutes • Select a general moderator and a specifications, topics • Determine what/if incentive can/will be to berecorder covered) • Record the session offered
Informal
Formal
Less expensive More control over selection of participants
More expensive Less control over selection of participants
Smaller potential participant Larger potential participant base base More paperwork Less paperwork (moderator) Formal report depends on YOUR notes
Prompt, professional formal report
May be viewed as less objective by participants (because YOU are conducting)
Often viewed as more objective- the moderator is neutral
Focus Groups Have Helped Direct Our Growth
For More Information … Colette Hochstein [email protected] General Questions [email protected]
http://toxmap.nlm.nih.gov
