Hi! I noticed in 2013 - 2017 5 yrs ACS, there are two PUMA in New Mexico that are showing missing for total population, here is the puma id:
35_01001
35_01002
Maybe there is something wrong at my end? Is there anyway I can troubleshoot this error?
I created an NHGIS extract for the data you’ve described, using a comma-delimited (CSV) format. When I open the file in Excel, the total populations aren’t missing…
But I see an issue that’s unique to these two PUMAs, which could cause additional problems… Both of them have a name that includes a special character:
- 01001: “Doña Ana County (Outer) PUMA, New Mexico”
- 01002: “Doña Ana County (Central)–Las Cruces, Mesilla Cities & University Park PUMA; New Mexico”
There are different standard “encoding” systems for text characters, each of which represents the “ñ” character differently. Software packages typically make an assumption about which encoding is in use when you open a file, and if the assumption is wrong, the software may read a special character as multiple characters. E.g., when I opened my NHGIS CSV data file including these PUMAs in Excel, Excel read the ñ characters as “±” (as in the screenshot).
If you requested the data in a fixed width format (rather than comma-delimited), then any misinterpreted characters like this would cause the whole data row to shift and misalign with the expected column positions. That could look like missing data.
I can think of a few ways you could try to address the problem:
- Get comma-delimited files instead of fixed width
- Hand-edit the fixed width files to remove the special characters and ensure that the data in these two rows are properly aligned before you load them into your data-analysis software
- See if your software supports an option to specify the encoding when loading the data file, and if so, try using UTF-8 encoding (instead of Latin-1).
Thank you so much for the answer! I will go ahead download the comma-delimited files for future reference!