Hi, I’m having some trouble downloading extracts using the R API. I already submitted 16 extracts and am trying to download them using the following code:
samples ← get_sample_info(“usa”)
acs_5y_samples ← samples %>%
filter(str_detect(description, “ACS 5-year”))
for (i in 1:length(acs_5y_samples$name)) {
desc ← paste(“ACS 5-year extract”, i, “of 16. Used to calculate total employment in each occupation.”)
extract ← define_extract_micro(
collection = “usa”,
description = desc,
samples = acs_5y_samples$name[i],
variables = c(“occ1990”),
data_structure = “rectangular”
)
extract_submitted ← submit_extract(extract)
} # Submits extracts #8-23
for (i in 8:23) {
extract_num ← paste0(“usa:”,i)
print(extract_num)
extract_submitted ← get_extract_info(extract_num)
filepath ← download_extract(extract_submitted, overwrite = T)
ddi ← read_ipums_ddi(filepath)
df ← read_ipums_micro(ddi)
write_dta(df, paste0(“../data/ACS 5-year/acs_5y_”,i,“.dta”))
}
All of these files are relatively small; I have downloaded much larger files with no issue. What keeps happening is that one file will download with no problem, and then the next will get stuck on the download_extract() command, and then it’ll time out. If I wait 5-10 minutes and then try running it again, starting from the next extract, then it works but will time out again on the next extract in the loop.
Has anyone encountered this issue before? Thanks.