Welcome to Amy and Susan’s Research Tips and Tricks. This post will be the first in a three-part series, where we’ll share our own best practices for planning a user research project. We’ll cover Participant Recruit, Research Methodology, and Technology & Team Composition.
But first, a bit about us…
Susan Stanger has been managing the obstacle course of complex digital projects for over 15 years. Among the many gauntlets she’s run over a dozen UX research projects at Spring Studio, from generative concept development to global customer insights to product usability.
Amy Geller has a 19-year history in management and content development. In 2011 she was proudly dubbed Research Manager at Spring Studio, and has since championed all recruiting and project oversight for generative, evaluative and usability research studies.
We’re excited to share our hard-won wisdom, and hope these tips and tricks will help you hone and strengthen your own user research management skills.
Managing a user research project can be a fulfilling experience that yields significant insights, if done well. In any given project you’ve got a timeline, a budget, participants to shepherd through multiple project stages, and ultimately, clients to satisfy. There are lots of moving parts and contingencies that you’ll need to oversee carefully to keep your project on track, and it can get stressful, but here’s some good news—with thorough planning, attention to minutiae, and advanced consideration of the potential pitfalls, you can stay on top of slippage and lead your team through a productive and rewarding process from recruiting to results.
Let’s start with the participant recruit.
Organization is essential. If possible, consider enlisting a Project Manager or Research Assistant to keep recruit details in order. For projects where that’s not an option, make sure you allow sufficient time to coordinate the myriad details of the recruit. It’s also important to remember that some logistics will be easier to control than others. Honing in on those elements you can actually control will help your team stay as organized as possible, and keep proper perspective on variables that typically defy the rules.
Let’s start with the areas you can control:
- Schedule Enough Time for the Recruit - Anyone who’s ever painted a room learns that the actual painting part takes less time than all the prep beforehand. The first step in being organized is recognizing all the things that need to be set up in preparation for the recruit. It’s important that your timeline allows for setting up the details of the recruit itself. These details include: finding the right recruiter, working with them to set up and ensure that they fully understand the criteria and goals of the project, and fielding any logistical details they might have as part of their process.
- Who should be included in your study?
This should be easy, right? You need to talk to a specific set of people, so you find them and talk to them. No big deal. The problem is, sometimes stakeholders (or researchers) bring too many distinct goals to the project and this makes for an unreasonably specific sample. One surefire way to identify and avoid this problem is to be wary of requests for participant segmentation that are extremely granular. For example:
We’ll include 24 people, 12 should be prospects and 12 existing customers. Of the 12, half should drink coffee on weekends and the other half all week. Of the weekday drinkers, half should drink decaf in the evenings ONLY. Must have some who have been drinking coffee for only a year and only at work. Include a mix of income brackets, ages, ethnicity, education across the 6.
See what I mean? The difficulty in finding these needles in a haystack in your recruiter’s database, in your timeframe and with the resources that you have will make you ask yourself, “do they even exist?” This level of parsing and specificity in the recruit criteria may bust your budget and push the timeline into the realm of infinity. If it seems too granular, step back, and ask yourself if the project goals are specific enough. Once you have a manageable project goal, along with attainable sample parameters, give yourself additional flexibility by separating that sample into “must have” and “nice to have.”
- Priorities Regarding People vs Time - For certain studies, timing is paramount, and for others it’s all about getting the right people. Is your study being done to inform stakeholders on how best to market a product that already has a hard launch date? If you’re dealing with hard deadlines, make sure your stakeholders and team are willing to prioritize around those dates. Or, are you managing a project whose research will be used to help a company figure out how to market to a new demographic? If targeting a specific group of people is paramount, make sure all involved prioritize the need to adequately represent that demographic in your sample. Understanding your priorities around people vs time is crucial to planning and executing a recruit that will serve your study.
- Avoid Hard Numbers – While hard numbers can feel reassuring on a spreadsheet, they may also be the source of unnecessary hardship when searching out participants. Why require ten participants of a very specific nature if fewer will give you enough data? Don’t create pain points by being unnecessarily rigid. Avoiding hard numbers where you can will serve as a strategy that increases flexibility and decreases stress. Rather than relying on a randomly assigned number, work within a reasonable range of numbers, and assess your sample for adequacy keeping in mind your specific goals. If you’re not certain your numbers are adequate, take a moment to reclarify your stated goals, then ask yourself if your current participant breakdown will give you adequate representation to achieve those goals.
- Special allowances concerning list pulls – If you’ve determined that a customer list pull is the best way to find your participants, keep in mind this can add the following complexities:
- Time – Requesting a list pull from the database manager is a process·that can take anywhere from 48 hours to 8 weeks. If the lead time on your request is significant, consider your criteria carefully because once the database query is set in motion, you won’t be able to amend it without resetting your entire timeline. Building in the opportunity to audit a sample list pull will help ensure you’ve got the recruit you need. At the very least, be sure to allow time to review the final list, once it’s delivered.
- Client / Vendor Permissions – It may be that only a C-level staff member will have the clearance to make or sponsor the list pull request. Also, in some organizations, only approved vendors may have access to the lists.
- Client’s Level of Experience – It’s important to understand how much experience with this process your client contact has. Less experienced clients will benefit from your expert assistance and coaching, as this will help internal deadlines stay on track.
- Filter Referencing – As the planning continues, the filters for the database query should be referenced, since they will impact who is ultimately included in the list pull, as well as determine further screening that may be needed.
- Legalities – Using a customer list often brings with it other requirements, such as legal or compliance approval.
- Security – Secure transfer of lists is often required.
It’s always a good idea to remember that the quality of different customer lists will vary, and your recruit is only as good as your list. For example, it’s not uncommon for emails, phone numbers, etc., to be outdated.
Next are some contingencies you won’t be able to control, but still need to factor into your plan:
- Participant Rescheduling / Cancellations / No-Shows / Unqualifieds – These scenarios are going to happen, so it’s important that you decide upfront how you’ll accommodate these unknowns. Will you over-recruit? Will you need to do “mop-up” sessions? Whatever you choose, make sure there’s room allowed to accommodate that strategy and its aftereffects. It’s important to set limits on rescheduling. Too many postponements can make your timeline unreasonable.
- Team Members - Don’t forget about your team members. Sure, they’re superheroes, but researchers and their families do get sick. Tight schedules can be a set-up for problems if there isn’t enough breathing room to recover from the unexpected office plague.
- Compensation for Participants - Compensation will cover your participant interview and follow-up touch points. The amount can be recommended by the recruiter and should take into account the duration of the interview as well as the scope of the additional touch points required. The recruit of doctors and other professionals can bump up compensation costs, as these folks may expect more money for their time, so keep that in mind while you crunch the numbers.
These best practices will help to keep your recruiting process under control and pave the way for a smooth transition into research. Next time, we’ll forge ahead with pro tips for putting together a top notch research plan. Stay tuned!