Proper format for simple API call in R

IPUMS API
1

Hi all. I’m new to using APIs in R, so please forgive me for this likely simple question; however, I’m pulling my hair out trying to get this to work. :slight_smile:

I’m not going to post my API key here, but suffice it to say that it is saved in R and it’s called in the code. I keep receiving the same error.

url ← “https://api.ipums.org/extracts/?collection=nhgis&version=v1”
mybody ← '{
“datasets”: {
“2016_2020_ACS5a”: {
“data_tables”: [“B15003”],
“geog_levels”: [“county”]
}
},
“geographic_instances”: [“010”],
“data_format”: “csv_no_header”,
“description”: “test_b15003”
}

’
mybody_json ← fromJSON(mybody, simplifyVector = FALSE)
result ← POST(url, add_headers(Authorization = my_key), body = mybody_json, encode = “json”, verbose())
res_df ← content(result, “parsed”, simplifyDataFrame = TRUE)
my_number ← res_df$number
my_number

The error is below.

→ POST /extracts/?collection=nhgis&version=v1 HTTP/1.1
→ Host: api.ipums.org
→ User-Agent: libcurl/7.64.1 r-curl/4.3.2 httr/1.4.4
→ Accept-Encoding: deflate, gzip
→ Accept: application/json, text/xml, application/xml, /
→ Content-Type: application/json
→ Authorization: 59cba10d8a5da536fc06b59d17125ffa0b114a2cb3f2f546cec95a68
→ Content-Length: 173
→

{“datasets”:{“2016_2020_ACS5a”:{“data_tables”:[“B15003”],“geog_levels”:[“county”]}},“geographic_instances”:[“010”],“data_format”:“csv_no_header”,“description”:“test_b15003”}

← HTTP/1.1 500 Internal Server Error
← Content-Length: 0
← Content-Type: text/html; charset=UTF-8
← Date: Thu, 06 Oct 2022 22:39:31 GMT
← Server: nginx/1.18.0 + Phusion Passenger(R) 6.0.15
← Status: 500 Internal Server Error
← Vary: Origin
← X-Powered-By: Phusion Passenger(R) 6.0.15
← X-Ratelimit-Limit: -1
← X-Ratelimit-Remaining: 0
← X-Ratelimit-Reset: 0
← X-Request-Id: a28802ed-ab31-4cec-aa07-70d77b4f284f
← X-Runtime: 0.059476
←

res_df ← content(result, “parsed”, simplifyDataFrame = TRUE)
my_number ← res_df$number
my_number
NULL

2

Hi John,

How are you forming your JSON payload? Are you writing it in something like Word and then copying it to your code? The reason I ask is that when I cut and paste your JSON, it includes “smart quote” characters instead of regular double quotes, which are not valid JSON. We often see that when text has passed through Word or a similar program.

You can validate that you are starting with valid JSON using a validator like jsonlint.com to rule this out as the cause.

3

Hi. I’m having a similar problem to John and cannot figure out what’s wrong with my code.

This code works:

submit data extract request

url ← “https://api.ipums.org/extracts/?collection=nhgis&version=v1”
mybody ← ’

{
“datasets”: {
“2000_SF1b”: {
“data_tables”: [
“NP001A”
],
“geog_levels”: [
“blck_grp”
]
}
},
“geographic_extents”: [“250”],
“data_format”: “csv_no_header”,
“description”: “sample6”
}

’

mybody_json ← fromJSON(mybody, simplifyVector = FALSE)
result ← POST(url, add_headers(Authorization = my_key), body = mybody_json, encode = “json”, verbose())

However, when I simply change the datatset, data table, and geographic level, the following code does not work:
mybody ← ’

{
“datasets”: {
“1980_STF1”: {
“data_tables”: [
“NT1A”
],
“geog_levels”: [
“tract”
]
}
},
“geographic_extents”: [“250”],
“data_format”: “csv_no_header”,
“description”: “sample6”
}

mybody_json ← fromJSON(mybody, simplifyVector = FALSE)
result ← POST(url, add_headers(Authorization = my_key), body = mybody_json, encode = “json”, verbose())

The error is below:
→ POST /extracts/?collection=nhgis&version=v1 HTTP/1.1
→ Host: api.ipums.org
→ User-Agent: libcurl/7.64.1 r-curl/4.3.2 httr/1.4.3
→ Accept-Encoding: deflate, gzip
→ Accept: application/json, text/xml, application/xml, /
→ Content-Type: application/json
→ Authorization: 59cba10d8a5da536fc06b59da78b632ef9224353a8a79182a1ccc1a0
→ Content-Length: 158
→

{“datasets”:{“1980_STF1”:{“data_tables”:[“NT1A”],“geog_levels”:[“tract”]}},“geographic_extents”:[“250”],“data_format”:“csv_no_header”,“description”:“sample6”}

← HTTP/1.1 400 Bad Request
← Cache-Control: no-cache
← Content-Type: application/json; charset=utf-8
← Date: Fri, 25 Nov 2022 19:25:45 GMT
← Referrer-Policy: strict-origin-when-cross-origin
← Server: nginx/1.18.0 + Phusion Passenger(R) 6.0.15
← Status: 400 Bad Request
← Vary: Origin
← X-Content-Type-Options: nosniff
← X-Download-Options: noopen
← X-Frame-Options: SAMEORIGIN
← X-Permitted-Cross-Domain-Policies: none
← X-Powered-By: Phusion Passenger(R) 6.0.15
← X-Ratelimit-Limit: -1
← X-Ratelimit-Remaining: 0
← X-Ratelimit-Reset: 0
← X-Request-Id: 82ea0305-b21c-4f15-bce2-bb556411cb47
← X-Runtime: 0.133891
← X-Xss-Protection: 1; mode=block
← Content-Length: 231

Again, all I changed was the datatset, data table, and geographic level. What am I doing wrong?

4

It looks like in the second code sample you are missing the closing ’ on the mybody variable assignment. When I add that back in, the code works for me.

5

Sorry, I copied my code incorrectly into the Reply. The closing ’ was actually there. Tried it again with the following and still no dice.

mybody ← ’

{
“datasets”: {
“1980_STF1”: {
“data_tables”: [“NT1A”],
“geog_levels”: [“tract”]
}
},
“geographic_extents”: [“250”],
“data_format”: “csv_no_header”,
“description”: “sample6”
}

’

6

Ah, I apologize, an issue with my rstudio environment had me thinking it was working when it was not.

I think I’ve discovered the true source of the issue: 1980_STF1’s tract geographic level does not support extent selection. I confirmed this using the NHGIS metadata API:

GET https://api.ipums.org/metadata/nhgis/datasets/1980_STF1?version=v1

...

        {
            "name": "tract",
            "description": "State--County--Census Tract",
            "has_geog_extent_selection": false,
            "sequence": 65
        },

...

If you remove the geographic_extents key-value pair from your JSON, it appears to work.

Hope this helps!

7

Additionally, the response body contains more details about validation errors. In this example, here is the response body:

> content(result)
$type
[1] "SemanticValidationError"

$status
$status$code
[1] 400

$status$name
[1] "Bad Request"


$detail
$detail[[1]]
[1] "Geographic extents Extent selection is not required for selected geog levels. Please remove the 'geographic_extents' section of you request."

>

Hopefully this is useful for your future debugging!