Research process overhaul

Overview

When I joined Career Karma, the user research process was notably underdeveloped. The primary method of connecting with users was through surveys distributed by the sales team, which were sent out on a monthly basis, albeit with inconsistent frequency. At that time, the design team had been reduced to just two members: me and an off-shore counterpart. Recognizing the need for a more structured approach to understanding user needs and experiences, I took the initiative to build a research repository from scratch. This effort marked the beginning of a comprehensive overhaul of the user research process at Career Karma, aiming to embed user insights more deeply into the product development lifecycle.

Our research process underwent monthly iterations, incorporating feedback from team members dispersed across different locations and expertise levels.

Identifying research needs

With a larger team, leveraging three-dimensional framework would be a great place to start. Not so much for a series B startup. I started with a simplified framework as a decision tree. This decision tree allows all team members to initiate their own research with minimal design involvement.

Incorporate industry best practices

‍In the first three months, I observed the team's inclination towards user interviews and surveys during evaluative research. Consequently, I published a condensed version of industry best practices for conducting user interviews and creating surveys. This was largely based on two books: Surveys that work by Caroline Jarrett and Interviewing users by Steve Portigal. These served as initial resources for team members to understand the process. Here are a few gentle reminders I like to share with team members:

• Leave your opinions at the door
• Make it clear to the participant (and to yourself) that they are the expert and we are the novice.
• Build rapport not friendship. Focus on the participants and be very selective about talking about yourself.
• After asking a question, be silent.

Process development:

‍Drawing from the team's collective experience, we established clear requirements for conducting qualitative research. These include a clear research goal, participant cohort, recruitment methods, and budget. The research cannot proceed without meeting the aforementioned requirements.

Implementation and iteration:

‍As we started the research implementation, we faced challenges and identified opportunities for improvement. Here are our key takeaways:

• Regardless of whether it's a survey or an interview, it's best to start with a smaller sample size.
• A smaller sample size gives the researcher the flexibility to modify the interview guide, invitation, or even incentives as needed.

Results:‍

With the new process in place, we conducted user testing more frequently, with clear goals established. The results from this research were shared, categorized, and prioritized for future improvements.