Usability testing before launch

At Torre, we had the plan to enhance the talent seekers' pipeline of candidates to add valuable data points for identifying inefficiencies in the recruitment process, while also showcasing Torre's services to increase monetization.

An expandable 'Stats' section was designed to show additional information without overwhelming users and to introduce monetization options subtly.

The usability test highlighted benefits like time-saving but also identified issues such as small font sizes and difficulty finding purchase options, which were addressed in the final design.
Product Designer
Role
Torre
Company
February, 2024
Date completed
Context
Torre is an AI-powered platform that helps talent seekers find ideal candidates based on skills and preferences. One of the most recurrent views for them is the pipeline of candidates.

The pipeline is a board type view that shows detailed information about the recruitment process for a given job opportunity, helping talent seekers review and organize candidate cards from a potential match all the way to a potential hire.
In the pipeline the recruitment stages are shown as columns and each one is filled with candidate cards that can be moved. Similar to a Trello board.
What was the challenge?

The challenge was to increase Torre's perceived value in the pipeline. To do so, we wanted to provide data points that allowed them to identify issues in their recruitment process, take action and be more efficient. This also created an opportunity to showcase Torre's services as a hook to monetize more users.

‍

However, this meant a complex change in the pipeline's design because:

‍

  • The pipeline already had a lot of information. As you can see in the image above, it contains several columns with lots of candidate cards with their name, picture, ranking, and engagement options.
  • The user wasn't used to seeing monetization call-to-actions in the pipeline.
  • There were several scenarios to consider to show the correct data for each.
Designing and prototyping different scenarios depending if the user was a paying customer in Torre and/or if the job opening had any visitors at all.
How did we tackle it?
Step 1: Understand the opportunity

We had to decide what additional information could provide value to our users. Based on feedback and internal discussions, we decided to add:

‍

  • ‍The recruitment process as a funnel analysis: Providing talent seekers insights into which stage in the process wasn't efficient.
  • The stats of each attraction and referral channel: Allowing talent seekers to activate or pause specific candidate channels based on their performance.
  • The potential reach of the job opening via Torre's services: Showing talent seekers how many potential candidates could be attracted to the opportunity with the paid services.

‍

Step 2: Hypothesize

Our hypothesis was that adding this new stats section in the pipeline would improve the talent seeker's overall experience and increase Torre's revenue. To prove that, we had to track:

‍

  • Upgrades to paid services from the call-to-action buttons in the new stats section.
  • Number of times that a user engaged with the new stats section.
  • Increase in the talent seeker retention (Net Opening Retention Rate).

‍

Step 3: Design and prototype

Considering the challenges mentioned before, we thought that the best approach was adding an expandable 'Stats' column at the beginning of the pipeline. Why?

‍

  • It allowed the user to show or hide the information. Avoiding the feeling of being overwhelmed by unwanted information.
  • Monetization call-to-actions only appeared once the stats column was expanded. Making it less invasive for users.
  • We were consistent with the current column based design in the pipeline.

‍

Step 4: Run a usability test

This effort meant a big change in the design of the pipeline, therefore we decided to run a usability test. The parameters of the test were:

‍

  • There were three different flows considering two variables: 1) If the user was paying for a Torre service, 2) If the job opening had any visitors.
  • We had four testers for each flow, two users on a mobile device and two users on a desktop.
  • Each tester had twelve minutes to complete a series of tasks and answer a set of questions.
  • All the testers had to approach the test as if they were talent seekers recruiting for their company.
  • There were no demography filters for testers.

Six prototypes were built to cover the three flows and the two types of devices. Twelve random testers were recruited via UserBob. One report was delivered with the results of the usability test.

‍
The different tests that were completed using UserBob (left) and the report from the usability test (right). The codename for this effort was #moneyball.
What were the results?

There were mixed results in the report for the usability test:

‍

  • Even though more related to value perceived, more than half of the testers highlighted how important and time-saving this new stats section could be for them. Three of them had previous recruiting experience.
  • When asked to find specific data points in the 'Stats' column, 25 out of 36 times testers got it right. The 69% of success rate could increase with quick copy changes or small rearrangements in the design.
  • A serious issue found was that we were using fonts that were too small too fit all the information, and that had a negative effect on the user's experience.
  • The most critical issue was that several testers weren’t able to find the option to purchase our services (Reach or OS) in the 'Stats' column.

‍

We solved those issues with a final iteration in the design and sent it to development. Currently the effort is being developed and it will be live soon for all users.