I am interested in determining whether a given set of months had significantly lower employment relative to the most recent month of data for one of the occupations within the occ2010 variable. I think I’ve figured out how to build my complex survey design (thanks to @Ivan_Strahof’s guidance in this topic), but now I need to determine what the best statistical test would be.
I’m more used to using logistic regressions to determine whether differences in proportions are statistically significant, and I did manage to create such a regression in order to test whether the proportion of adults ages 22-65 who are employed in this field changed significantly from a given month to my baseline. However, I’d still like to find a way to see whether the total number of individuals employed in this field, regardless of age, has increased or decreased. This will of course be affected by US population changes, but that’s OK.
I tried both a general (gaussian) regression and a poisson regression, but my results didn’t make sense–so I’m guessing that wasn’t the right approach. Any guidance on whether another regression model might work better would be very helpful!
(I do also have data on estimated monthly employment for this occupation, along with confidence intervals. If two 95% confidence intervals for two different months don’t overlap, it should be pretty safe to assume a significant difference in employment, correct? And for this sort of test, could I actually use 90% confidence intervals instead, since I’ll only be comparing the ‘right’ tail of one total to the ‘left’ tail of the other?
Thanks as always for your help!
We leave code review and analytical decisions up to individual researchers since these are beyond the purview of the user support team. However, I can share some general suggestions on approaches you may consider.
Stata’s svy suite (see svy estimation) includes a total command that allows researchers to use survey design parameters to estimate totals without requiring any additional aggregation or regression analysis. In your case, you can use it to estimate the total monthly employment for your occupation of interest and then follow it up with lincom to test the difference between the estimates while retaining the survey robust standard errors. For analyses that only include a subgroup from the entire sample, such as estimates of a single occupation’s total size, the svy subpop() option should also be used. I am not as familiar with R’s survey and srvyr packages, but I believe these offer analogous approaches.
Note that people can be recorded as having a particular occupation even if they are currently on layoff, are currently unemployed (available for work and actively searching) and their most recent job was in that occupation, or if they are not in the labor force but their most recent job within the past 12 months was in that occupation. If you want to get the total number of people employed in a particular occupation, you will want to only count those with EMPSTAT = 10 or 12. People without an occupation code are assigned OCC2010 = 9999.
We leave the decision of which level of confidence to report with estimates up to individual researchers. In some fields reporting a 95% interval is standard while in others a 90% may be sufficient. Comparing the confidence intervals of two separate estimates individually is generally a more conservative approach to testing differences. I recommend using lincom or some other test that directly compares your estimates with each other.
I also recommend taking a look at the annual occupation-level employment statistics provided by the BLS OEWS (Occupational Employment and Wage Statistics) program as a useful benchmark. While the data for the OEWS uses more detailed SOC occupation codes and won’t line up with monthly CPS data, it might still be a helpful external source of validity for your estimates!
This is very helpful, thank you! I appreciate the detailed response as always.
I’m using a relatively new Python package for complex survey analysis called svy (not to be confused with Stata’s svy), but I do compare its outputs to R pretty often for verification–so I’ll look into some R-based options for this analysis.