Weighting Survey Results: Calculating and Applying Weights
By: Scott Young, Ph.D., Principal, syoung@valtera.com
As we saw in the last blog on this topic, item weighting is an effective method of addressing disproportionate respondent rates. Today’s blog discusses how to calculate those weights.
Calculating Weights
The calculation of weights is simple and remains the same regardless of the number of variables on which the weights are based. The basic formula for computing a weight (where “cell” is the department of interest) is:
Cell Population / Total Population
Cell Returns / Total Returns
Consider this example. Below are the weight calculations for three departments (A, B, and C) surveyed within a company. Together, these three departments employ 5,000 people. Each department represents between 20% to 40% of the population. Only 500 employees responded to the survey, with participation rates varying by department. Though Department A represents 40% of the population, it only represents 20% of the respondents. Contrast this with Department C, which also represents 40% of the population, but 60% of the respondents.
If we were to simply combine all respondents (with no weighting) in the calculation of the overall results, Department C would have much more influence than Department A on the results, despite their equal representation in the population. If employees in Department C were much more satisfied than employees from Departments A or B, we would then end up with a higher proportion of satisfied employees in our sample than actually exists among the population of these three departments. This, therefore, is a good case to use weighting to help ensure that our overall picture is as accurate as possible.
|
Dept A
2,000/5,000 = 2.0
100/500
|
Dept B
1,000/5,000 = 1.0
100/500
|
Dept C
2,000/5,000 = 0.67
300/500
|
Notice that the responses from Department A are given a large weight (2.0), thus compensating for the lower proportion of returns from Department A compared to the proportion of Department A employees in the population. In a similar manner, responses from Department C are given less weight (0.67) while Department B responses are unit weighted (1.0). Weighting ensures that each group’s contribution corresponds to its representation in the population. 
Hint: Checking your work
To ensure that the weights were calculated correctly, multiply the returns for each group by their weights and sum the products. The result should be equal to the total number of returns as shown below:
(2.0 x 100) + (1.0 x 100) + (0.67 x 300) = 200 + 100 + 200 = 500
|
Applying the Weights
There are two ways to calculate the overall weighted average response for a survey item: 1) apply a weight to the response of each individual by multiplying each individual’s response by the weight corresponding to his or her group, sum these products across all individuals, and then divide by the total number of respondents, and 2) multiply the item mean or average for each group by its proportion in the population; sum the resulting products.
Equations for calculating weights based on incomplete information and multiple classification variables are available in the linked white paper.
