In years since 1950, counties are not directly identifiable in public use microdata, which is why many observations will have a zero. The reply from @JeffBloem on this forum post does a good job of explaining why and provides some guidance for alternatives.
Whether or not you should drop the observations depends on your analysis, but it would certainly eliminate a large portion of your data.