Project Overview
Course: ISM 360 – User-Centered Evaluation
Roles: Researcher, Testing Moderator, Testing Observer
Group: Jenica de Leon, Patty Taylor, Rakesh Kosuri
Research Goals:
- To understand the users of the Ventra mobile app
- To identify current usability issues and make recommendations
- To improve the interaction between the system and the user
| Who? | Why? | How? |
|---|---|---|
| Our target audience was moderate to frequent users of the Ventra app who were over 18, college students, and / or working professionals. | The goal of this research was to capture the user’s perspective while examining the strengths and weaknesses of the existing app’s usability. | By evaluating users through interviews and usability tests, we were able to observe the results and make recommendations for improvement. |
User Research
To learn about the users, my group conducted interviews and analyzed the results. Each group member interviewed and observed 2 users, resulting in a total of 6 interviews / observations.
Interviews
Our team interviewed 6 users of the Ventra mobile app. The participants consisted of 2 students from varying colleges, 3 working professionals, and 1 participant that was considered both a student and a working professional.
First, we asked screener questions to make sure we were interviewing participants within our target audience. Once they’ve qualified, we presented each participant with a consent form that explains the purpose of the study and to inform them that their participation is completely voluntary. After they have signed or verbally approved the form, we proceeded with our interview protocol. We asked them a list of open-ended questions so that we can understand their thoughts and learn the tasks that they do with the Ventra app.
Data Analysis
After the interviews, we analyzed and sorted the data we have gathered as a group by creating an affinity diagram to form insights about the users. Each of us wrote our individual findings on post-it notes, and then we clustered them into 5 categories: concerns, frequency, positives, use, and wants.
PERSONAS
3 personas from our data:
- Aaron Smith – “Frequent Professional User”
- James Miller – “Frequent Student User”
- Sally Johnson – “Infrequent User”
TASK ANALYSIS
Based on our interviews, we found out that some users experienced ease, while others experienced difficulty when purchasing train tickets from the app. An example of a difficulty is how users felt like there were too many ‘clicks’ when going through the process of purchasing their tickets. Our team created a task analysis diagram to walkthrough the process of this specific task, and to be able to understand what the users go through when completing the task.
The diagram shows how a user purchases a Metra ticket using the Ventra mobile app.
Usability Testing
As final part of our research, my group conducted usability testing to observe any emerging patterns within our participants and their interaction with the Ventra app. Similar to our interviews, my group used a recruitment and screening process to find participants for our testing sessions. These sessions took place at the Usability Testing Lab inside DePaul’s CDM building, Room 721. The Android version of the app was used for the study and the sessions were recorded using the Morae Usability Testing software. Each group member recruited and tested 2 participants, resulting in a total of 6 usability tests / observations.
Test Objectives
During our testing sessions, we asked our participants to complete a set of tasks by focusing on 4 key features of the Ventra app, and backing them up with scenarios so that we can evaluate the usability of the app. The goal is to have our participants navigate and complete each task so that we can observe their thoughts, feelings or reactions as they interact with the app’s interface.
The 4 features we focused on:
- Purchasing a Metra Ticket
- Adding Money to a Ventra Card
- Finding a Ventra Retail Location
- CTA Bus Tracking
Roles
A moderator’s job is to proceed with the test plan by introducing the study and presenting the consent form to the participant. They read the protocol / script and guide the participants through the set of tasks we have prepared.
An observer’s job is to record the session. They quietly listen in a separate room and take notes during the test through the Morae software.
Findings / Results
We reviewed our gathered data (observations and feedback from our participants) to make recommendations for the system’s improvement.
The following is an example of a task we’ve asked our participants to complete, along with data results from the usability test and our suggested solution to the problem.
Task: Finding a Ventra Retail Location
Objective: Can the user find a Ventra kiosk near their location?
Scenario: You prefer refilling funds on your Ventra card by using a kiosk. You are at a new place and you’re not sure where the nearest one is located. Can you show us how you would find the nearest Ventra kiosk?
Main issue:
- Most participants had difficulty finding the section for locating a kiosk
- The task took longer to complete
How can this be improved?
- Make the ‘Find A Ventra Retail Location’ more visible to the user
- Put this section as one of the options in the navigation menu
REFLECTION:
I gained skills in performing interviews and usability testing sessions in order to identify problems and suggest solutions so that users can have a better user experience. Using these methods, I was able to understand existing users and take their feedback to determine ways to make the app more easy to use, and the key features more faster to complete. It was surprising how recruiting people for the interviews turned out to be very challenging even though my team brainstormed ideas. I thought it would be easy at first, but it ended up being difficult finding the right people to talk to at the Union Station. For next time, what I would do differently is find new ways to recruit people and pick better timing. In the future, I would like to gather more data and hear from more people that use the app. I truly enjoyed learning about the users, and I had fun interviewing and testing them. I’m also glad that my team communicated and collaborated well with each other which made this project more fun.
Tools used:
GoogleDocs, GoogleSlides, Microsoft Word, Morae software, Slack, Stormboard, Ventra app
