Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

maxresults #19

Open
maelle opened this issue Mar 27, 2017 · 3 comments
Open

maxresults #19

maelle opened this issue Mar 27, 2017 · 3 comments

Comments

@maelle
Copy link
Member

maelle commented Mar 27, 2017

In the doc of the get_inat_obs function it's not written what the max value of maxresults is. Could you add it? 😸

@emhart
Copy link
Contributor

emhart commented May 3, 2017

Well, the max is technically around 5 million I think (that's how many results there are total) and I think theoretically you could grab them all. maxresults is there mostly to make your life a little easier if you don't want to be overloaded with results.

@kueda
Copy link

kueda commented May 6, 2017

What endpoint are you guys hitting for get_inat_obs? If it's http://www.inaturalist.org/observations.json then your max is definitely 5+ million (though responses will get slower and slower the deeper in you get), but if it's http://api.inaturalist.org/v1/observations, it's actually limited to 100k results. This is a kind of annoying limitation of elasticsearch, which we use to back api.inaturalist.org. There are some ways we could change things to get around that, but the recommended ES technique is to use a feature they don't recommend for realtime responses like the ones you guys are doing. At some point we should probably do what GBIF does and allow API users to request a large download and get back to them when the full result is generated.

@kueda
Copy link

kueda commented Aug 17, 2018

Suggestion to the rinat devs: instead of using the page param when retrieving data from https://api.inaturalist.org/v1/observations, use the per_page and id_before or id_after params to iterate over observations. This limits the scope of the overall result set of any query and thus allows you to iterate over all records.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants