You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Repoint hardcoded bbop-sqlite S3 URLs to the SemanticSQL CDN
universalizer downloads pre-built OAK/SemanticSQL SQLite databases by hardcoding the raw S3 bucket URL. The maintainers of that bucket (Berkeley BOP / INCATools SemanticSQL) are retiring direct public access to the raw S3 bucket and moving all traffic behind a CDN. Because this URL is hardcoded (rather than resolved through OAK'''s sqlite:obo: selector or the semsql package), it will not auto-migrate and will break once raw-bucket access is removed.
You must also send a non-default User-Agent. The CDN sits behind a Browser Integrity Check that returns HTTP 403 to default client User-Agents (Python's Python-urllib/3.x, R's download.file/httr, and bare curl/wget defaults). A host-only swap will start returning 403 — set an explicit, non-default User-Agent header on the request.
If you fetch with urllib/requests, add an explicit header, e.g. urllib.request.Request(url, headers={"User-Agent": "universalizer/1.0"}).
The object names/paths are identical — only the host changes (s3.amazonaws.com/bbop-sqlite/ or bbop-sqlite.s3.amazonaws.com/ → semanticsql.berkeleybop.io/).
Filed as part of the coordinated downstream migration tracked in INCATools/semantic-sql#115. If you've already migrated or this is a non-issue, feel free to close — thanks!
Repoint hardcoded
bbop-sqliteS3 URLs to the SemanticSQL CDNuniversalizerdownloads pre-built OAK/SemanticSQL SQLite databases by hardcoding the raw S3 bucket URL. The maintainers of that bucket (Berkeley BOP / INCATools SemanticSQL) are retiring direct public access to the raw S3 bucket and moving all traffic behind a CDN. Because this URL is hardcoded (rather than resolved through OAK'''ssqlite:obo:selector or thesemsqlpackage), it will not auto-migrate and will break once raw-bucket access is removed.Where this repo is affected
universalizer/oak_utils.py—f"https://s3.amazonaws.com/bbop-sqlite/{db}.db"What to change
https://s3.amazonaws.com/bbop-sqlite/{db}.dbhttps://semanticsql.berkeleybop.io/{db}.dbImportant
You must also send a non-default
User-Agent. The CDN sits behind a Browser Integrity Check that returns HTTP 403 to default client User-Agents (Python'sPython-urllib/3.x, R'sdownload.file/httr, and barecurl/wgetdefaults). A host-only swap will start returning 403 — set an explicit, non-defaultUser-Agentheader on the request.If you fetch with
urllib/requests, add an explicit header, e.g.urllib.request.Request(url, headers={"User-Agent": "universalizer/1.0"}).Why now
s3.amazonaws.com/bbop-sqlite/orbbop-sqlite.s3.amazonaws.com/→semanticsql.berkeleybop.io/).Filed as part of the coordinated downstream migration tracked in INCATools/semantic-sql#115. If you've already migrated or this is a non-issue, feel free to close — thanks!