Download quota exceeded for this file

[anticdimi]

Hi @Juzezhang ,

Thank you for the amazing work!

Is there a workaround for the error Download quota exceeded for this file from Google Drive?
I keep getting this both for HOI-M3 and HODOME datasets. Are these, by any chance, hosted on some other platform that would make it easier to download, given the dataset size?

[Juzezhang]

Hi @anticdimi ,

Thank you for letting me know this problem. Do you have any recommendations for an alternative platform? I can have a try.

As for a potential Google Drive workaround, you can try:
Make a copy: Right-click the file and select “Make a copy.” Then, download the copy from your own Google Drive.

[anticdimi]

Hmmm, I'm not quite sure what would be a better option. Maybe some servers at your institution would be a good idea.
In my experience, nextcloud andamazon s3 might be good options, although I am unsure about the pricing. Nowadays, maybe huggingface is also an option (I don't know much about this, but I believe there needs to exist a cloud-stored dataset).

Also, many files are uploaded as .tar, which is not a compressed archive. It might be better to compress the data to .tar.xz ( I think this should be the most efficient compression, although I am far from an expert in compression 🤣 ) or .tar.gz. This might make the downloads and storage utilization more efficient for both you and the users.