[Sora] Incentives for survey participation

Mon Nov 28 16:27:28 PST 2016

Here is an article discussing federal funding for incentives: http://www.marketingresearch.org/legal-article/respondent-incentives-federally-conducted-or-funded-survey-research
>From the article:

Government oversight/approval
An estimated 25 percent of federal government surveys use respondent incentives, although the Office of Management and Budget (OMB) usually turns down requests to use incentives, and incentives are forbidden for use in the government's customer satisfaction research. Incentives offered as part of a grant-funded research study are approved (or not) by various Institutional Review Boards (IRBs).

[DHS has a group that functions - somewhat - as an IRB-like group for organizations requesting to use DHS data - as I understand it. It's been a while since I sat in on a couple of meetings. But clearly it is NOT an IRB.]

Long-running federal surveys use respondent incentives

*       The National Health and Nutrition Examination Survey (NHANES)<http://www.cdc.gov/nchs/nhanes/about_nhanes.htm> is unique in that it combines interviews and physical examinations. NHANES has used incentives since it began in the 1960's.

*       The Bureau of Labor Statistics' National Longitudinal Surveys<http://www.bls.gov/nls/home.htm>, running since the 1960's.

*       The National Survey on Drug Use and Health (NSDUH)<https://nsduhweb.rti.org/> provides national and state-level data on tobacco, alcohol, and illegal drug use, prescription drug abuse, and mental health in the U.S.

Another report from 1993 notes some interesting issues in "The Problem" and "Defining Incentives"

http://www.copafs.org/reports/providing_incentives_to_survey_respondents.aspx

Here is a 2016 updated Guide on conducting surveys from OMB - https://www.whitehouse.gov/sites/default/files/omb/inforeg/pmc_survey_guidance_2006.pdf

Substantially space is dedicated to the discussion of response rates and incentives (p. 56 - 71).

*       Included is a section, "69. How can response rates be improved?"

*       Section "74. What are incentives?"

*       75. Why must agencies provide a justification to give incentives to respondents?

*       76. What factors should agencies address in their justification to give incentives to respondents?

In justifying their proposed use of incentives, agencies should consider the following principles, many of which overlap:

o   Data quality: One possible justification for requesting use of an incentive is improvement in data quality. For example, agencies may be able to provide evidence that, because of an increase in response rates, an incentive will significantly improve validity and reliability to an extent beyond that possible through other means.

o   Burden on the respondent: An agency can justify an incentive if it can demonstrate that there is a need to pay a respondent for exerting unusual effort or having an unusual reporting burden in responding to a collection of information. This type of effort can be seen in data collections that require respondents to keep daily logs for an extended period of time, participate in a medical examination, abstract information from a significant number of records, coordinate study team visits, and so forth.

o   Complex study design: Some studies require ongoing participation of various respondents, each of whom is important to the achievement of study goals. For example, there may be a panel study over multiple years that requires participation by the same schools, teachers, parents, and students.

o   Past experience: Agencies may be able to justify the use of incentives by relating past survey experience, results from pretests or pilot tests, or findings from similar studies. This is especially true where there is evidence of attrition and/or poor prior response rates.

o   Improved coverage of specialized respondents, rare groups, or minority populations: A survey may have as its target population a highly selective group. Offering incentives to this population can be justified by describing the importance and difficulty of obtaining their participation in the study. For example, a study on the health of the hearing-impaired may propose providing an incentive to obtain enough respondents with particular impairments to participate. Similarly, a justification to pay an incentive for a study that involves recruiting highly specialized physicians may be considered.

o   Reduced survey costs: If prior or similar surveys have devoted considerable resources to nonresponse follow-up, it may be possible to demonstrate that the cost of incentives will be less than the costs of extensive follow-up. While some personal visit surveys have found that the cost of incentives has been made up in reduced field interviewer time and travel costs, this is rarely true for other data collection modes. Thus, agencies should not assume that incentives will pay for themselves.

o   Equity: Agencies should treat all respondents equally with regard to incentives. OMB generally does not approve agency plans to give incentives solely to convert refusals, or treat specific subgroups differently, unless the plan is part of an experimental design for further investigation into the effects of incentives.

o   Research into the effects of incentives: Because OMB has expressed interest over the years in encouraging research into the effects of incentives, proposals that include experimental designs that provide insight into incentive effects are often approved. Agencies should plan to examine not only the impact on overall response rates by the use of an incentive, but also the effects on key estimates (with a similar purpose to that addressed in question #71).

From: Sora [mailto:sora-bounces at listsmart.osl.state.or.us] On Behalf Of HELVIG Judy K
Sent: Monday, November 28, 2016 8:30 AM
To: State of Oregon Research Academy <sora at listsmart.osl.state.or.us>
Subject: Re: [Sora] Incentives for survey participation

Also at DHS, Child Welfare research is often done by external research partners.  DHS does require our research partners to offer Gift Cards instead of a Cash incentive to our clients.....because of the rule in place, or interpretation of the rule....

Thanks, Judy

Judy Helvig
Research Analyst
Office of Business Intelligence
Data Collection & Reporting Unit
500 Summer St SE, E72
Salem OR 97301
PH:  503-945-6571
FAX: 503-581-6198

NOTE: This e-mail may contain information, data or reports that are privileged and confidential and for Internal DHS use only.  If you received information in error, please destroy immediately and notify the sender.

From: Sora [mailto:sora-bounces at listsmart.osl.state.or.us] On Behalf Of Jerry Curry
Sent: Wednesday, November 23, 2016 5:02 PM
To: sora at listsmart.osl.state.or.us<mailto:sora at listsmart.osl.state.or.us>
Subject: [Sora] FW: Incentives for survey participation

From: BETTS Chris [mailto:Chris.BETTS at dhsoha.state.or.us]
Sent: Tuesday, November 22, 2016 7:17 AM
To: Jerry Curry <JerryCurry at oslmail.osl.state.or.us<mailto:JerryCurry at oslmail.osl.state.or.us>>
Subject: RE: Incentives for survey participation

Jerry,
DHS has been performing surveys of program participants and providers for a long time.
We get good, periodic feedback data on program performance and customer experiences/satisfaction without offering incentives per se in most cases.
What we do is explain the purpose of the survey and what the benefit is before they take it. Then use that data to make improvements they can see.
Sometimes the improvements take a long time and other times they can be relatively speedy. It just depends on the topic and the program.

Good luck and Happy Thanksgiving!
Christopher Betts
Senior Project Manager
Oregon Department of Human services
Office of Business Intelligence
Survey Administration Services
503-945-6097 Desk
503-507-8140 Mobile
Chris.betts at dhsoha.state.or.us<mailto:Chris.betts at dhsoha.state.or.us>

[DHS Oregon Department of Human Services]<http://dhsdesign.hr.state.or.us/index.html>

Please let us know how we're doing! Feel free to provide feedback about our services using the following link:
http://www.surveygizmo.com/s3/2122236/DHS-Internal-Customer-Satisfaction-Survey (Select "Office of Business Intelligence" in the drop-down at the beginning of the survey)

From: Sora [mailto:sora-bounces at listsmart.osl.state.or.us] On Behalf Of Jerry Curry
Sent: Monday, November 21, 2016 3:58 PM
To: sora at listsmart.osl.state.or.us<mailto:sora at listsmart.osl.state.or.us>
Subject: [Sora] FW: Incentives for survey participation

From: Brandt Robin L [mailto:ROBIN.L.BRANDT at dhsoha.state.or.us]
Sent: Monday, November 21, 2016 3:35 PM
To: Jerry Curry <JerryCurry at oslmail.osl.state.or.us<mailto:JerryCurry at oslmail.osl.state.or.us>>; sora at listsmart.osl.state.or.us<mailto:sora at listsmart.osl.state.or.us>
Subject: RE: Incentives for survey participation

Hello SORA members,

The issue is, if you provide an incentive, it could be seen as a conflict of interest. That conflict could be perceived as paying a program participant to participate in a survey about services received and it could be seen as an incentive to say good things about the program. (Last time I looked at studies about survey respondents, the studies showed that most survey respondents are more likely to be happy about services anyway. It was more difficult to get respondents to report negative things - which unfortunately makes it difficult to understand how to better improve the program.) I don't have a citation for that information handy.

Previously I worked for a university in another state and consulted with the state Vocational Rehabilitation for many years to survey persons with disabilities closed from service to complete a satisfaction survey.

You note that state the OAM does not allow the purchase of "gifts" for participants. During my work elsewhere, Vocational Rehabilitation explored its ability to provide an incentive through the contract that my University program had. I recall that federal regulations forbid funds to purchase such items as well. I have attached a document about the federal regulations and you may want to refer to the e-CFRs that relate to the funding for your program: http://www.ecfr.gov/cgi-bin/text-idx?SID=5ecdeae3e118ec61efacf9cc17af5cc0&mc=true&tpl=/ecfrbrowse/Title02/2tab_02.tpl  These federal funds are considered "grants" and our programs must adhere to federal regulations about money.

Please note that when I was investigating whether Vocational Rehabilitation participants could pursue employment related to the production, manufacture, distribution or sale of cannabis, federal authorities explained that it was not only federal funds could not be used (because marijuana is illegal in federal law). Any state match to the grant funding the program was considered federal funds too. So, I think it follows that any state government match monies cannot be used to provide incentives anyway.

If you have questions, this is the hierarchy of precedence at the federal level is as follows.

[cid:image008.png at 01D24994.4ECC3940]
That's what I have to share. I hope it is useful and not too confusing.

Robin

From: Sora [mailto:sora-bounces at listsmart.osl.state.or.us] On Behalf Of Jerry Curry
Sent: Monday, November 21, 2016 2:52 PM
To: sora at listsmart.osl.state.or.us<mailto:sora at listsmart.osl.state.or.us>
Subject: [Sora] FW: Incentives for survey participation

Hello SORA folks,

Please respond  to Cheng-Fai via the SORA list as this may be a question many of us are interested in.

From: LAI Cheng-Fei -CEdO [mailto:cheng-fei.lai at state.or.us]
Sent: Monday, November 21, 2016 2:05 PM
To: MCCOY Jan - ODE <jan.mccoy at state.or.us<mailto:jan.mccoy at state.or.us>>; Jerry Curry <JerryCurry at oslmail.osl.state.or.us<mailto:JerryCurry at oslmail.osl.state.or.us>>; 'Chapman Natasha L' <natasha.l.chapman at state.or.us<mailto:natasha.l.chapman at state.or.us>>
Subject: Incentives for survey participation

Hello SORA colleagues
I'm curious to know if anyone has experience with incentivizing participants (e.g. school teachers/staff, parents, students, community members, etc.) to complete surveys in your studies?
I've been told that we can't as state agencies: "OAM 10.40.10 paragraph 115  notes items which may not be purchased with State funds and it specifically states gifts."

Any feedback/input/advise on this topic is greatly appreciated!

Cheng-Fei

Cheng-Fei Lai, Ph.D., Research Analyst
503-373-0067 | 775<mailto:email at state.or.us> Court Street NE, Salem, OR 97301 | cheng-fei.lai at state.or.us<mailto:cheng-fei.lai at state.or.us>
education.oregon.gov
[https://gallery.mailchimp.com/a47b05a8f1c8426cbfc2677ac/images/2a06af3d-c887-4be7-833d-aaec4c23d4c3.jpg?_ga=1.18949947.2041684798.1394553618]<http://education.oregon.gov/>[cid:image011.jpg at 01D24994.4ECC3940]<https://twitter.com/ORLearns>[cid:image013.jpg at 01D24994.4ECC3940]<http://education.oregon.gov/feed/>

-------------------------------------------------------
Jerry Curry
Reference Librarian
Oregon State Library
Salem, OR

jerry.w.curry at state.or.us<mailto:jerry.w.curry at state.or.us>
503-378-5008

Check out our databases & resources or
request a presentation to your work group.
http://library.state.or.us
-------------------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omls.oregon.gov/pipermail/sora/attachments/20161129/d6a72215/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.jpg
Type: image/jpeg
Size: 4391 bytes
Desc: image006.jpg
URL: <https://omls.oregon.gov/pipermail/sora/attachments/20161129/d6a72215/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image008.png
Type: image/png
Size: 83187 bytes
Desc: image008.png
URL: <https://omls.oregon.gov/pipermail/sora/attachments/20161129/d6a72215/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image009.png
Type: image/png
Size: 201 bytes
Desc: image009.png
URL: <https://omls.oregon.gov/pipermail/sora/attachments/20161129/d6a72215/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image011.jpg
Type: image/jpeg
Size: 813 bytes
Desc: image011.jpg
URL: <https://omls.oregon.gov/pipermail/sora/attachments/20161129/d6a72215/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image013.jpg
Type: image/jpeg
Size: 833 bytes
Desc: image013.jpg
URL: <https://omls.oregon.gov/pipermail/sora/attachments/20161129/d6a72215/attachment-0002.jpg>