Social service leaders who take a data-driven approach to policy and program development and resource allocation eclipse their peers in terms of driving meaningful social change. What sets these leaders apart is their tenacity when it comes to proactively investigating questions like: are there any significant gaps in services provided; where are there opportunities for more equitable distribution of programs and services; and how can we improve the degree to which an organization’s offerings are delivered using culturally relevant strategies. These questions, which are generally answered through the use of surveys, focus groups and one-on-one interviews, can then be further dissected by any number of demographics, including race, ethnicity, gender, and age.
But it is often not feasible to speak to every single individual in an organization’s service area; instead, researchers often use sampling strategies to better understand how various segments of the target population experience an organization’s programs and services. Not only does sampling help to significantly reduce the number of participants needed overall, it also reduces the cost to undertake the study. Using the right sampling strategy can help your organization glean valuable insights into targeted populations that can be more difficult to reach. Homeless populations or those without internet access are common populations where researchers use sampling to get the information they need to adequately represent the views and perceptions of these populations.
Probability (or Random) and Non-probability Sampling
There are two basic categories of sampling: probability (or random) sampling and non-probability sampling. Using a probability sampling approach, every member of the target audience (population) has an equal chance of being selected to participate in the study. This approach lends itself to drawing sweeping generalizations or broader trends, however it is extremely time-intensive, which translates into higher costs to perform the study. And, these approaches often don’t adequately provide insights into specific demographic groups. Probability sampling methods include clustered sampling, simple random sampling, stratified sampling, and systematic sampling.
For non-probability (non-random) sampling, some respondents have no chance of being selected for the sample, which can result in overrepresentation of a specific demographic. These strategies are most typically used in preliminary studies, where multiple hypotheses are initially explored. Non-probability sampling methods include convenience sampling, judgement (or purposive) sampling, quota sampling, and snowball sampling and, more generally, any study that involves self-selection of participants (e.g., an online survey where participants self-select to complete the survey).
Our Guiding Principles for Research: Consistency and Equity
When it comes to conducting research for our social sector clients, Measurement Resources abides by its internal guiding principles. First, we adhere to the methodology (which includes the sampling strategy) selected for the project; any deviation from this will skew the results. We also encourage our clients to use an equity lens to ensure the voices of specific demographic groups and hard-to-reach target populations are embedded in the methodology. Ensuring the voices of specific demographic groups are embedded in the methodology often results in a sample that is not fully representative of the population though provides critical information needed to understand the perceptions and experiences of subgroups within a population. Measurement Resources employs statistical methods (e.g., survey response weighting) when the goal of the study is to both understand the experiences of subgroups and generalize findings to the overall target audience.
For example, we recently helped a county-wide organization better understand the behavioral health needs of their community. As part of our overall population-level data collection strategy, we integrated tactics that would ensure an inclusive response. From providing translated surveys in multiple languages to offering the option of completing paper versions of the survey (as opposed to online only), we were able to secure a high volume of responses that reflected the various population segments throughout the county.
Additionally, our sampling strategy included first understanding what the population characteristics were of the target community to ensure our sample was both representative of the population—but also included enough responses to estimate differences in perceptions by varying demographic groups. Population characteristics were gathered from U.S. Census Bureau Data and were used to inform our sample targets by demographic groups (e.g., if the target community is comprised of 15% African American Males under the age of 40, we would expect our sample to also reflect this demographic group proportionally). We utilized the sample targets to conduct ongoing monitoring of survey completion rates by demographic groups to identify any gaps in response rates and then guide targeted survey outreach strategies.
Measurement Resources’ Expertise in Sampling Strategies
For each project Measurement Resources Company takes on, our team develops a customized methodology that will help answer our clients’ most urgent research questions, and bring to light the most significant opportunities for continuous improvement. Our expertise in this aspect of research design has helped more than 200 government and non-profit organizations confirm their suspicions, explain certain phenomena, and put them on a path to efficiently improve their outcomes.
