Update tools_validator.py - #1859
Conversation
Adding a delay to account for rate limit on FAIRsharing's APi
Increasing time considerably
|
With a 5-second delay, things work until we get a 500 Internal Server Error. Rerunning. |
| registry['fairsharing'] = check_fairsharing | ||
| if 'fairsharing' in registry.keys() and not registry['fairsharing']: | ||
| del registry['fairsharing'] | ||
| time.sleep(5) # to avoid issues with rate limit |
There was a problem hiding this comment.
ooof I am not a fan of this solution. This will increase the general time way to much no? I assume we can do something more clever on the requests side, or keeping track of the number of requests we've already done.
|
@bianchini88 You have better contacts with FAIRsharing, can this not be solved on their end? I understand rate limits for unauthenticated requests, but we are authenticated, and only doing around 1000 requests over 10min. This does not feel like a hard ask. I am not comfortable putting sleeps like this in the code |
|
Crashed twice at European Joint Programme on Rare Diseases Metadata Model, which should get https://fairsharing.org/FAIRsharing.3e0038. Not sure about the issue. |
|
Hi @bedroesb. This was meant as a temporary fix for the summer, but it does not look like it is working anyway (and I am off in 15 minutes to catch a flight). FAIRsharing recommends that we transition to the GraphQL API and the 'advancedSearchFast' query (https://api.fairsharing.org/schema/query.doc.html). |
2 participants
Adding a delay to account for rate limit on FAIRsharing's API