I learned from an earlier post that CPS is representative at the state level as long as appropriate weights are applied.
However, from this post, I understand that ASEC is no longer representative when zooming in on specific groups even with proper weighting.
Is there a general rule (or a best practice/experienced rated recommendation) when disaggregating is ‘too much’, i.e. state representativeness fails?
For example, suppose I am interested in computing mean incomes and federal taxes (FEDTAXAC) of households with a working age head. Can I expect that using ASECWTH as a weight will deliver results which are representative at the state level?
Thanks a lot for guidance on this.
(PS: Following @Matthew_Bombyk’s recommendation, I worked through section 10-3 of this report but I have not been able to find what I was looking for.)
In general, there is no bright-line rule regarding “too much disaggregation.” In practice, what will happen is the sampling error around estimated statistics will be relatively large and will, therefore, limit any informative interpretation from the data. It’s ultimately up to trial and error - calculate standard errors at a given level of disaggregation, and then decide if they’re too large to make the analysis useful.
Although the CPS samples are not stratified for the specific subsample that you’re interested in, it doesn’t mean that they aren’t representative of that subsample. The CPS is still a random sample, and the base sampling weights correct for differential probability of inclusion in the sample. This means unbiased estimates are possible at the state level, even for subsamples. Small subsamples will have high variance, and estimates of the total population of a subgroup will be just that, estimates. That’s as opposed to the total state population, where summing the weights will give the actual (projected) population of the state, by design.
In your specific case, since most households have a working age head, I doubt you’ll have any problems. Although I mentioned in the thread you linked that the samples are not constructed to sum to population controls for each single-year age group, the ratio-adjusted weights (aka WTFINL and ASECWT) do actually take age ranges into account. So for such a broad age group as “working age,” the weights should help to reduce the variance substantially. Also since you are using ASEC you should consider using replicate weights.
2 Likes
Many thanks Matthew for this detailed and informative answer! Very helpful.