Compensating participants in service design research
A tool kit for government agencies to pay participants in service design research—for more effective and efficient services
Why pay research participants
Paying members of the public to participate in government service design research pays off in efficiency, cost savings, and public relations. While this practice has been common in the private sector for decades, government has generally lagged behind for a range of reasons. (We address the most common ones below.)
Where government does allow participants to be paid, it is generally done as part of a contract that includes other services, such as doing the research. This is less than ideal, as it uses the vendor as a kind of bank and sets up a situation where the only way to conduct design research is through vendors.
Government agencies should have the capacity to nimbly conduct research, themselves, without having to go through vendors. That includes paying participants.
Paying participants is critical to building government services that work well
There is simply no substitute for directly observing someone interact with a product or service to understand how to optimize the technology and design to achieve the desired outcomes. Direct observation helps teams know how someone is interacting with their product and why people behave the ways they do in those interactions.
It is common to use questionnaires that ask people to report on their experiences—or focus groups that encourage collective though. But these methods fall far short when it comes to understanding specific challenges and frustrations of users. These approaches leave teams to make inferences based on participants’ opinions and feelings.
Direct interaction with individuals as they walk through a specific experience delivers rich qualitative data. Directly observing people as they use products or services informs decisions about how to remediate frustrations and wrong turns before the products or services launch. It’s risk mitigation.
With this data, products and programs can confidently meet users’ needs, prevent customer service and support issues, and streamline processes for cost effectiveness and timely efficiency. People who allow teams to observe them interacting with government services should be compensated for their time and expertise.
The value to the government of observational data far outweighs the costs of paying participants
Paying small numbers of research participants saves the government money. When paying participants to provide their expertise and experience with government services the return on investment is extraordinarily high. It’s cheaper to spend nominal amounts on compensating participants now than to redesign later due to incomplete or biased feedback.
Let’s be clear about the amounts of money we are talking about. When teams engage individuals from the public to conduct design research for service design and delivery, this is a) largely qualitative research that is generative or evaluative in nature; and b) small sample sizes of between 5 and 50 people.
Here’s an example of cost-benefit: Every year, there are millions of claims for unemployment insurance payments. (Some people end up filing multiple claims.) By observing someone who is filing for unemployment or who is certifying that they are still unemployed, you can see and hear data that there is no other way to get. You can learn:
What questions people have about the benefit.
Where they encounter frustrations or make incorrect selections for their situation.
How they think through their answers to questions in the form.
How they interpret or misinterpret questions or form field prompts.
Where the form forces users to pick the closest, best answer rather than the real answer.
What people perceive to take a long time or a short time.
How they think the decisions about whether to give them benefits get made.
Any of these issues could cause users to call a call center with questions, contribute to users making mistakes on their claims that create delays, or even avoid applying for benefits that they need and deserve.
This type of observational research is one of the most effective tools there is for surfacing user needs and pain points.
If you do this with just 5 people, you start to see repeated patterns in where people get stuck or encounter challenges. Most teams find this to be enough data to make good design decisions, which they then test again.
There are also secondary effects that are beneficial to the government, such as reducing calls to call centers; avoiding fraud, overpayments and underpayments; less training for support personnel and quicker time-to-productivity; and so on. All of these have costs.
Using a call center as an example, let’s do some math:
5 participants for 1 hour each at $100 per participant = $500.
20-person call center = $500,000 annually as a baseline, plus costs for recruiting, hiring, and training people: $200,000. Total = $700,000.
If you could cut costs for training call center reps by even 1%, you’d save $7,000, which is the equivalent of 400 more hours of personnel time that reps can be spending on more critical issues than debugging software. Even a small state, like Massachusetts, has hundreds of call center reps for unemployment.
Paying participants increases equity in service design and delivery
Payments can be incentives for taking part in design research. Payments can also be compensation for time and expertise. The people most in need of government services should be prioritized for giving input and feedback on how well those services work for them, because they have direct experience interacting with services outside the research. Participants are giving their time and offering valuable information that can be deeply personal.
Paying participants is a public relations win
Public perceptions are often a concern in scenarios where there can appear to be public funds spent without a clear and obvious return of product or service. As argued above, the value of the data received from involving people in the design of products and services far outstrips the cash outlay. But there are other points that are true and effective:
Compensating participants is a way to show that their involvement—their time, expertise, and experience—are valuable to the government.
The government can show that it is actively involving the public in designing crucial services and programs. Design research and usability studies can be a form of community and public engagement that is interactive, collaborative, and helps people feel their voices are heard.
Obstacles to overcome
Barriers to compensating people for participating in design research are bureaucratic and cultural, not legal. Government has a responsibility to deliver cost effective, efficient, accessible services. But government agencies struggle to find appropriate mechanisms to pay people who participate in research that would contribute to those goals. There are a few reasons for this. Some are bureaucratic, some are cultural.
Bureaucratic barriers
Government simply is not set up to pay a few individuals small amounts of money for anything. Government is designed to buy large things with large sums of money. When teams consult with leadership and attorneys to create programs for paying participants in digital services research, they encounter structural barriers of assigning an appropriate part of the bureaucracy to place the expenditures in. There’s a general inclination to try to fit payments to participants in user research and usability studies into one of these buckets:
Appropriations: Can our budget be spent this way?
Procurement: Are we contracting temporary services in this transaction?
Workforce: Are we hiring an expert to give us technical assistance on something?
In addition, there are questions of internal controls to ensure the proper use and accounting of government funds. We want to make sure that there’s no conflict of interest and no personal gain on the part of government personnel in this transaction.
Unfortunately, the same bureaucracy applies whether you’re spending $1 billion on fixing bridges or paying someone $50 for allowing a researcher to observe them while they encounter inscrutable instructions or frustrating delays when they try to get government services like a permit to build a house or pay real estate taxes for that house.
Cultural barriers
Government personnel sensibly worry about the unknown: Whether they're allowed to pay participants and if it’s been done before. Logistically, it seems complicated and risky. Additionally, personnel have it drummed into them that they have a duty to protect taxpayer funds to avoid waste, fraud, and abuse. (This is good!)
There are understandable concerns about fairness and avoiding graft and corruption. (Also good!)
There are strict ethics rules around giving and receiving gifts. (Important and good!) Without proper internal controls (documentation of the expenditures), it might be too easy for someone to abscond with a $100 gift card or two.
Constituents should never perceive or feel that their taxpayer dollars are being misused. And, people deserve to be compensated for their time and expertise.
Legal precedents
Paying participants is legal when it is used to achieve a statutory purpose. There areprecedents for using legislatively appropriated funds as incentives to take part in programs that are important to carrying out statutory duties. Agencies that conduct clinical trials do this regularly. For example, when governments grant funding for clinical research, this type of spending is called “participant support.”
Compensating participants who are helping to develop or improve software, services, or programs, is no different.
In 2008, the US Government Accountability Office ruled that the National Telecommunications and Information Administration’s (NTIA) use of gift cards as incentives was appropriate to achieve the aims of a statute, especially considering that the amount was modest and the beneficiary of the expenditure is the government.
Many states have laws that include restrictions on gifts of public funds. California is one of them. The California constitution, Article XVI, Section 6 on Public Finance, contains a clause on gifts of public funds. However, the courts have clearly established that when deciding whether a government payment is an illegal gift of public money, the main question is whether the money is being used for a public purpose or a private one. If the spending is for a public purpose, it’s not considered a gift, even if it also helps private individuals. In those cases, the benefit to the public is treated like getting something in return (called "consideration" in legal terms), so it's not a free giveaway.
This principle has been supported by many earlier California court decisions involving cities and counties spending money for things like public buildings or social services. For example, if a government agency receives information, services, or goods that:
Serve a legitimate public purpose, and
Are worth as much or more than what was paid,
… then the payment is not a gift under Article XVI, Section 6 of the California constitution.
The California Supreme Court and other courts have repeatedly said that spending public money to help individuals can still be legal—if it clearly serves the broader public.
Cases in California supporting this legal assertion include:
County of Alameda v. Janssen (1940). Holding: "The primary question … is whether the funds are to be used for a public or private purpose. If they are to be used for a public purpose, they are not a gift … even though private persons are benefited therefrom."
City of Oakland v. Garrison (1924). Holding: Reinforces the “primary purpose” inquiry: “the purpose of the expenditure, not the recipient, is the most important factor.”
Orange County Foundation v. Irvine Co. (1983). Court of Appeal (citing Supreme Court precedents): “It is well settled that the primary question … is whether … for a public purpose … The benefit to the State … is the nature of consideration …” The court applied the test in the context of a state settlement payment to a private company.
Templates
These templates and examples are based on documents that have been used effectively in different parts of government (federal, state, and local). They are free for you to adopt and adapt for your own purposes. Read them carefully before you try to adopt them. Work with your attorneys on what will work in your organization.
To set a new policy
These sample memos help establish a new protocol or policy. They're from a senior executive to everyone in an agency or department.
Checklist to assess (state example from California to assess whether a payment would violate a prohibition on gifts of public funds)
Sources and contributors
This tool kit was compiled and written by Dana Chisnell, Bloom's Design Leader in Residence, with a lot of help from Megan Zehnder. Special thanks to Dave Zvenyach, who was a helpful advisor.
The templates come from various government agencies across the United States, including the Department of Health and Human Services, the Technology Transformation Service (TTS) of the federal General Services Administration, the state of California, and San Francisco Digital Services. Thank you to all the teams who have unlocked participant compensation for their governments and for sharing what you have learned with us so we could share it even further.
If you have questions about this kit or you use it to create your own approach to compensating members of the public who participate in user research or usability studies, we would love to hear from you. Email us at hello@bloomworks.digital.
