UX Research Study
Working with a team to conduct discovery stage UX research on the National Weather Service website and prepare a usability report with recommended improvements.
Project Type
Usability Research
Student Project
Date
Nov 2024
1 Month
Role
UX Researcher
Background
The National Weather Service (NWS) is a trusted source of accurate weather, water, and climate data.
The mission of the NWS is to provide this data to support impact-based decision support services for the protection of life and property and enhancement of the national economy.
National Weather Service Website Photos
Project Challenge
“How might we evaluate and recommend changes to the National Weather Service’s website to improve usability and enhance overall user satisfaction with its features?”
Decisions and Outcomes:
-
Determine if any changes are needed to the design or structure of the NWS website to ensure users can easily locate and understand key weather information.
-
Identify the highest-priority usability issues with the website
As a result of this research, our team will:
-
Compile data and create a research presentation
-
Create a list of what is working well on the National Weather Service site
-
Create a list of recommended improvements to the National Weather Service site
Research Study
Business and Research Questions
Does the NWS website have good immediate usability? What issues are there?
-
What are users’ first impressions of the NWS website?
-
How easily can users find critical weather information, like current forecasts and alerts, on their first visit?
-
Do users encounter any navigation difficulties while searching for specific weather-related data? If so, where do they experience confusion?
Can users use the NWS website to accurately interpret and understand key information about their local region, including visual indicators or alert levels?
-
Can users interpret and understand key weather information on the NWS website, such as alerts and advisories?
-
How well do users understand the site’s visual indicators (e.g., alert levels, maps), and do these visuals aid comprehension?
-
Do users find the language and visuals used on the site clear and helpful for understanding the necessary information?
Can users efficiently use the NWS website to locate critical weather information, such as current forecasts, warnings, and alerts?
-
How quickly can users locate their target weather information (e.g., current forecasts, warnings, alerts) on the site?
-
Do users find it easy to navigate the site to locate weather information specific to their location?
-
Does it take users an excessive amount of time to locate critical information, or are they able to access it efficiently?
Methodology
-
Moderated usability study
-
Quantitative measures for time on task, frequency of issues, task success, and Likert scale
-
Fielded with representative prospective users
-
Sample size: 20 users, new users of site
-
Fieldwork sessions completed in the state of Washington
Users
The target audience for this study includes young adult Americans who regularly access weather information, ranging from those who need critical, real-time updates about severe weather to casual users checking daily forecasts.
Results and Insights
Task Descriptions
Task 1: Find and report the low temperature for tomorrow evening
-
Requires user to navigate to local weather
-
This information is not provided on the home screen
Task 2: Find and report if it was raining in the Seattle city area on the day you were born
-
Requires user to navigate to past weather data
-
This step requires several web pages and actions from the user (including entering data)
Task 3: Find and report an active warning or alert in a region within the United States
-
Main function of the weather.gov site
-
Should be relatively easy because of the alert map on the homepage
Task 4: Find and report one safety action someone could take to be prepared for this type of warning or alert
-
We expected users to navigate to the safety part of the website
-
This task also serves the main function of the website
Time on Task
User Quote: “I feel like if I had just googled it, I would have found it quicker”
Insights:
-
Users had a wide range of time requirements when completing each task
-
Taking the median (excludes outliers), participants took anywhere from 36 seconds to 3 minutes to complete a task
-
Task 2 (past weather data) was the most time consuming, and task 3B (finding safety data) was the least
-
Task 3A helped Task 3B go faster; participants were often able to find safety information in the weather alert text
The tasks take much longer than any other typical method users to accomplish these tasks using services besides this site. Task 2 was consistently the most complicated and time consuming, an indicator of structural issues.
Likert Scale
User Quote: “It’s awful”
After completing tasks, we had the participants answer these two questions using the Likert Scale rating:
Question 1: I am satisfied with the ease of the task in this scenario
Question 2: I am satisfied with the amount of time it took to complete the task in this scenario
Insights:
-
Task 1 and Task 2 were more difficult and more time consuming for users
-
Task 1 was “relatively easy”
-
Task 2 was more complicated
-
Task 3A and 3B received higher than average marks, but still not high satisfaction
The level of satisfaction is below the middle point for Task #1 and Task #2. Task #3, which explores the main point of the website, did score better. However, since it highlights the main purpose of the website, it should ideally achieve a higher score, such as a 6 or 7.
Task Success
User Quote: “It’s not sending me where I want to go!”
Insights:
-
Nearly all participants were able to find the low temperature information (Task 1)
-
Only slightly over half were able to access past weather data (Task 2)
-
About 3/4 of participants were able to accurately report weather alerts and safety (Task 3)
Eventually the majority of users were able to accomplish the task. However, the fact that there was any failure at all means there is room for improvement. It’s notable that task #2 had a 40% failure rate which means that there are significant navigation and approachability issues.
Issues
User Quote: “This is very interesting info...the way that you get there is a problem”
Insights:
-
Out of 31 unique issues, we identified 9 major themes
-
Top issue themes include: navigation issues, being confused by format of information, and not seeing information
-
Even though only 2 users encountered broken links, that’s 10% of participants
All of the recurring issues are about interacting with the website itself, this means that the basis of the problem is structural issues, not the content itself.
Recommendations
Recommendation #1
Simplified Website with Expert Version: Balancing Accessibility and Depth
Provide a simplified website that highlights essential information aligned with the mission statement. Include a clickable portal for knowledgeable users to access a more detailed, expert-level view.
Key Insights:
-
Appeals to Diverse Users: A streamlined interface caters to general users, while the expert version supports advanced needs, improving overall satisfaction.
-
Boosts Confidence: Simplifying the main view makes critical information more accessible and visually appealing, enhancing user trust and confidence.
Data Points: Poor user opinion on ease of use and time (T1 & T2); All three tasks had failures
Recommendation #2
Improved Map Function: Enhancing Navigation and Usability
Replace reliance on zip codes or longitude/latitude with a direct map-based navigation system, enabling users to interactively search and navigate specific areas.
Key Insights:
-
Addressing Confusion: Users struggle with zip codes and longitude/latitude searches; a map-based system reduces cognitive load and aligns with familiar navigation patterns.
-
Leveraging Familiarity: Most users are familiar with Apple or Google Maps. Adopting similar map-first functionality aligns with Jakob’s Law: users prefer familiar interfaces.
-
Advancing the Mission: Enhanced map navigation improves access to critical weather information, supporting NWS's goal of better accessibility.
Data Points: 15/20 users encounters navigation issues; Higher satisfaction with using the map in Task 3
Recommendation #3
Restructure Website: Streamlining Navigation for Efficiency
Reduce the number of nested pages to optimize the main user flow, ensuring navigation is limited to a single or double level for quicker and more intuitive access to key information.
Key Insights:
-
Simplified Navigation: Fewer nested pages reduce user confusion and streamline the journey to critical information.
-
Optimized User Flow: A shallow navigation hierarchy prioritizes essential content, helping users achieve their goals faster.
-
Improved Usability: Simplified structures align with best practices for intuitive design, improving overall site efficiency and user satisfaction.
Data points: Users tended to take more time, especially for Task 2 (required multiple pages); Recurring issue of users getting in loops
Recommendation #4
AI Assistant for Weather Information: Personalizing User Support
Introduce an AI assistant to greet users with a “What can we help you with today?” prompt. This assistant can guide users to relevant information based on their needs, balancing critical weather updates with everyday inquiries like rain forecasts.
Key Insights:
-
Enhanced Focus: An AI assistant helps users in disaster zones quickly locate urgent information without being overwhelmed by excessive details.
-
User-Centric Design: Shifting from an information-heavy homepage to an interactive experience improves clarity and aligns with modern user expectations for intuitive support.
Data points: Users had higher times (36s to 3mins) to find information; Users also having issue of going in loops to find information
Recommendation #5
Prioritize Access to Safety Information: Streamlining Critical Updates
Ensure all pages referencing a weather advisory link directly to the advisory for that specific weather type, consolidating key information and increasing visibility of reputable safety updates.
Key Insights:
-
Improved Information Flow: Direct links to advisories streamline navigation, ensuring users quickly access critical safety updates.
-
Highlighting Trusted Resources: Consolidating advisory information reinforces the credibility of NWS resources and ensures important details are seen.
-
User Behavior Insights: Testing revealed that 85% of participants did not access safety section and relied on plain HTML advisory details, highlighting the need for more intuitive access the full extent of safety content.
Data Points: Task 3 took a combined time of 1.5mins when users were not under pressure