Truncation of ddi metadata

CPS
1

Dear folks – (probably Greg Freedman Ellis Derek Burk)

I have a function, a large ungainly kludge that I will not bother you with unless necessary, that among other things reads the ddi’s for a set of files and, in the course of reading the files, saves all the file metadata from the ddi’s into a single list. Within this list there are many, many places that the file reports that it is truncated. A short sample is below. Note that all the places in bold below are surrounded by double underscores in the original.

ASECWT is a 10-digit numeric variable with four implied decimal places. That is, values of 0012345600 should be interpreted as 1,234.56. The IPUMS command files automatically divid"| truncated “\nNote: Values are as they appear in the original CPS data. Top-codes, bottom-codes, and NIU codes have not been harmonized by the IPUMS.\n\n1968-1975\n50000 = Top Code (50,000 and gr”| truncated “\n99999999 = N.I.U. (Not in Universe).\n99999998 = Missing.\nValues can be negative.\n\nSee User Note [URL omitted from DDI.] for an explanation of these codes, and the Topcodes Table”| truncated “\n9999999 = N.I.U. (Not in Universe).\n9999998 = Missing.\n\n\t\t\nIPUMS does not identify topcodes in the data. Refer to the Topcodes Tables link below for income topcodes documenta”| truncated "\n

Can you tell me if this truncation, designated by “| truncated ”, (<- note underscores) is done by ipumsr, the R/RStudio console, or one of the tidyverse programs? And if it is done by ipumsr, is there a way to turn it off?

2

The ipumsr package does not truncate metadata by design, so if it is doing so, that would be a bug. I suggest taking a single DDI file that you have observed getting truncated (e.g., the DDI file containing ASECWT from your sample) and read it into R with read_ipums_ddi(). If you observe truncation in the resulting object, that would be evidence of a bug, and we’d ask you to share a copy of your DDI file to help us reproduce the issue so we can find the source and fix it. If you don’t observe truncation, then the problem is not with ipumsr. Our team has not observed truncation exactly like this before, so we are not aware of the source. If the problem isn’t with ipumsr, I suggest posting your question on Stack Overflow, where it will be seen by a greater number of expert R users.