Need to consider lower CPS response rates for population estimates?

CPS
1

Hello,

I saw that the response rate for the CPS nationally had fallen recently from ~82% to ~67%, which may raise some issues about greater uncertainty when using the survey to estimate population characteristics. I was wondering if it was necessary to incorporate this change in the response rate when analyzing Basic Monthly Survey microdata from IPUMS–especially for smaller, subnational samples?

Specifically, I’m interested in calculating population estimates for a few mountain-plains states. I want to look at how employment and hours have changed over the past few months (and compared to that period in 2019 and 2018) along a few dimensions: race, education level, gender, wage, and industry (aggregating the IND variable to 2-digit NAICS level). Ideally, I would be able to have a cross-tab (e.g. compare how employment rates have changed between women of different races) of two or three of these variables. I was also thinking about comparing how ‘job losers’ who were included in the CPS rotation for multiple months of 2020 differed on one or more of these dimensions.

However, I’m worried about having a sufficiently large sample at the state level to estimate these values–especially because of the lower response rate. Is there anything I can do to ensure my estimates are okay, other than use the standard errors that I can calculate for population estimates (eg. using R)? Or would this lower response rate already be incorporated into standard error estimates, for example because it just translates to a smaller sample size?

Thank you!

2

I PUMS CPS has information regarding falling response rates and COVID-19 here.

I am not aware of any special instructions beyond using the relevant survey design variables to correctly estimate standard errors. However, I will caution against making population inferences from small sample sizes. At some point, your sample size may become so small that the standard errors around point estimates do not allow for meaningful interpretation of the data. There is no hard and fast rule defining how small is too small, and more cases are generally better. I am guessing that some of the unweighted cell counts of your proposed cross-tab are too fine-grained regardless of the decline in response rates, but each individual researcher needs to decide what is an appropriate threshold for their analyses.

1 Like