Repository navigation
Why can't install funannotate2 Databases #78
Description
Activity
did you set the FUNANNOTATE2_DB to a path you want to store the databases in? eg
export FUNANNOTATE2_DB=/public/home/shagan_lab/shagan/funannotate2_db funannotate2 install -d allsome related instructions on setting the variable persistently in the conda env funannotate2 https://github.com/nextgenusfs/funannotate2#apple-silicon-m-series
Reacted by zut0may0yes, I did it，thanks！！But i have a new question
funannotate2 install -d all [Jul 15 04:00 PM] Python v3.10.20; funannotate2 v26.6.21; gfftk v26.5.22; buscolite v26.6.21; annorefine v2026.2.22 [Jul 15 04:00 PM] The backend database location is from the $FUNANNOTATE2_DB env variable: /data1/NFS/home/rdr422123/funannotate2_db [Jul 15 04:00 PM] Retrieving download links from GitHub Repo [Jul 15 04:00 PM] { "uniprot": "https://ftp.ebi.ac.uk/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_sprot.fasta.gz", "uniprot-release": "https://ftp.ebi.ac.uk/pub/databases/uniprot/current_release/knowledgebase/complete/reldate.txt", "merops": "https://ftp.ebi.ac.uk/pub/databases/merops/current_release/meropsscan.lib", "dbCAN": "https://dbcan.s3.us-west-2.amazonaws.com/db_v5-2-9_5-5-2026/dbCAN.hmm", "dbCAN-tsv": "https://raw.githubusercontent.com/nextgenusfs/funannotate2/main/funannotate2/resources/dbCAN-fam-HMMs.txt", "dbCAN-log": "https://raw.githubusercontent.com/nextgenusfs/funannotate2/main/funannotate2/resources/dbCAN.changelog.txt", "pfam": "https://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.gz", "pfam-tsv": "https://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.clans.tsv.gz", "pfam-log": "https://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam.version.gz", "repeats": "https://osf.io/vp87c/download?version=1", "go": "https://purl.obolibrary.org/obo/go.obo", "mibig": "https://dl.secondarymetabolites.org/mibig/mibig_prot_seqs_3.1.fasta", "interpro": "https://ftp.ebi.ac.uk/pub/databases/interpro/current_release/interpro.xml.gz", "interpro-tsv": "https://ftp.ebi.ac.uk/pub/databases/interpro/current_release/entry.list", "gene2product": "https://raw.githubusercontent.com/nextgenusfs/gene2product/master/ncbi_cleaned_gene_products.txt", "mito": "https://ftp.ncbi.nlm.nih.gov/refseq/release/mitochondrion/mitochondrion.1.1.genomic.fna.gz", "mito-release": "https://ftp.ncbi.nlm.nih.gov/refseq/release/RELEASE_NUMBER" } [Jul 15 04:00 PM] Downloading Merops database [Jul 15 04:01 PM] Building diamond database: diamond makedb --in merops.formatted.fa --db merops diamond v2.2.4.184 (C) Max Planck Society for the Advancement of Science, Benjamin J. Buchfink, University of Tuebingen Documentation, support and updates available at http://www.diamondsearch.org Please cite: http://dx.doi.org/10.1038/s41592-021-01101-x Nature Methods (2021) #CPU threads: 80 Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1) Database input file: merops.formatted.fa Opening the database file... [0.001s] Loading sequences... [0.014s] Masking sequences... [0.326s] Writing sequences... [0.002s] Hashing sequences... [0s] Loading sequences... [0.001s] Writing trailer... [0s] Closing the input file... [0s] Closing the database file... [0.003s] Database sequences 5098 Database letters 1303674 Database hash eec5a2ca499df0169c0de588711f6134 Total time 0.351000s [Jul 15 04:01 PM] Downloading UniProtKB/Swiss-Prot database HTTPS download attempt 1 failed: ('Connection broken: IncompleteRead(14855691 bytes read, 78850778 more expected)', IncompleteRead(14855691 bytes read, 78850778 more expected)). Retrying in 5 seconds... [Jul 15 05:31 PM] Building diamond database: diamond makedb --in uniprot_sprot.fasta --db uniprot diamond v2.2.4.184 (C) Max Planck Society for the Advancement of Science, Benjamin J. Buchfink, University of Tuebingen Documentation, support and updates available at http://www.diamondsearch.org Please cite: http://dx.doi.org/10.1038/s41592-021-01101-x Nature Methods (2021) #CPU threads: 80 Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1) Database input file: uniprot_sprot.fasta Opening the database file... [0.054s] Loading sequences... [1.375s] Masking sequences... [17.036s] Writing sequences... [0.388s] Hashing sequences... [0.118s] Loading sequences... [0.002s] Writing trailer... [0.094s] Closing the input file... [0s] Closing the database file... [3.823s] Database sequences 575503 Database letters 208906902 Database hash 13235b1a95ab6064d49e5d251937b926 Total time 22.895000s [Jul 15 05:31 PM] Downloading dbCAN database [Jul 15 06:45 PM] Copied dbCAN metadata files from package resources [Jul 15 06:45 PM] Creating dbCAN HMM database and pressing with pyhmmer [Jul 15 06:45 PM] Downloading Pfam database HTTPS download attempt 1 failed: ('Connection broken: IncompleteRead(3343689 bytes read, 414816825 more expected)', IncompleteRead(3343689 bytes read, 414816825 more expected)). Retrying in 5 seconds... HTTPS download attempt 2 failed: HTTPSConnectionPool(host='ftp.ebi.ac.uk', port=443): Read timed out.. Retrying in 5 seconds... HTTPS download failed after 3 attempts: ('Connection broken: IncompleteRead(783689 bytes read, 417376825 more expected)', IncompleteRead(783689 bytes read, 417376825 more expected)) Trying FTP fallback for https://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.gz FTP download failed: <urlopen error timed out> Traceback (most recent call last): File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/site-packages/urllib3/response.py", line 905, in _error_catcher yield File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/site-packages/urllib3/response.py", line 1043, in _raw_read raise IncompleteRead(self._fp_bytes_read, self.length_remaining) urllib3.exceptions.IncompleteRead: IncompleteRead(783689 bytes read, 417376825 more expected) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/site-packages/requests/models.py", line 937, in generate yield from self.raw.stream(chunk_size, decode_content=True) File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/site-packages/urllib3/response.py", line 1267, in stream data = self.read(amt=amt, decode_content=decode_content) File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/site-packages/urllib3/response.py", line 1155, in read data = self._raw_read(amt) File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/site-packages/urllib3/response.py", line 1021, in _raw_read with self._error_catcher(): File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/site-packages/urllib3/response.py", line 925, in _error_catcher raise ProtocolError(arg, e) from e urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(783689 bytes read, 417376825 more expected)', IncompleteRead(783689 bytes read, 417376825 more expected)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/site-packages/funannotate2/utilities.py", line 249, in download for chunk in r.iter_content(chunk_size=8192): File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/site-packages/requests/models.py", line 939, in generate raise ChunkedEncodingError(e) requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(783689 bytes read, 417376825 more expected)', IncompleteRead(783689 bytes read, 417376825 more expected)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/bin/funannotate2", line 10, in <module> sys.exit(main()) File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/site-packages/funannotate2/__main__.py", line 22, in main install(args) File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/site-packages/funannotate2/install.py", line 228, in install data = pfamDB(wget=args.wget) File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/site-packages/funannotate2/install.py", line 589, in pfamDB download(DBURL.get("pfam"), hmm + ".gz", wget=wget) File "/data1/NFS/home/rdr422123/anaconda3/envs/funannotate2_env/lib/python3.10/site-packages/funannotate2/utilities.py", line 273, in download raise Exception(f"Download failed: {str(e)}") Exception: Download failed: ('Connection broken: IncompleteRead(783689 bytes read, 417376825 more expected)', IncompleteRead(783689 bytes read, 417376825 more expected))
did i need Installing Databases one by one ?
@hyphaltip I need use VPN to install ?
Looks like your internet connection is quite slow and unstable. I'll try to push a fix that might improve this, no guarantees it will work though.
Try to install from latest, ie from your environment run this:
python -m pip install git+https://github.com/nextgenusfs/funannotate2.git --upgrade --force --no-deps
And then try the install command again and see if it's able to complete.
(funannotate2) [shagan@ln01 FUNANNOTATE2_DB]$ funannotate2 install -d all
[Jul 15 11:24 AM] Python v3.10.20; funannotate2 v26.6.21; gfftk v26.5.22; buscolite v26.6.21; annorefine v2026.2.22
Traceback (most recent call last):
File "/public/home/shagan_lab/shagan/miniconda3/envs/funannotate2/bin/funannotate2", line 10, in
sys.exit(main())
File "/public/home/shagan_lab/shagan/miniconda3/envs/funannotate2/lib/python3.10/site-packages/funannotate2/main.py", line 22, in main
install(args)
File "/public/home/shagan_lab/shagan/miniconda3/envs/funannotate2/lib/python3.10/site-packages/funannotate2/install.py", line 47, in install
os.makedirs(env["FUNANNOTATE2_DB"])
File "/public/home/shagan_lab/shagan/miniconda3/envs/funannotate2/lib/python3.10/os.py", line 210, in makedirs
head, tail = path.split(name)
File "/public/home/shagan_lab/shagan/miniconda3/envs/funannotate2/lib/python3.10/posixpath.py", line 103, in split
p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not bool