The FTP directory containing the source data for this ingest has a duplicate file for September 2015 which is causing the script to fail at the end since it creates an off-by-one problem that's not being handled.
The duplicate files are in the FTP directory for the nClimGrid dataset:
ftp://ftp.ncdc.noaa.gov/pub/data/climgrid/nClimGrid_v1.0-preliminary_monthly_201509_c20151005.tar.gz
ftp://ftp.ncdc.noaa.gov/pub/data/climgrid/nClimGrid_v1.0-preliminary_monthly_201509_c20151007.tar.gz
A short term solution is to just use the second of these since it appears to have been processed a couple of days after the first one, assuming that the last number in the file name represents the processing date.
This issue has been reported to NCEI and hopefully one of these files will be removed in short order, but until then we can add a hack to ignore the first of these two files.
The FTP directory containing the source data for this ingest has a duplicate file for September 2015 which is causing the script to fail at the end since it creates an off-by-one problem that's not being handled.
The duplicate files are in the FTP directory for the nClimGrid dataset:
ftp://ftp.ncdc.noaa.gov/pub/data/climgrid/nClimGrid_v1.0-preliminary_monthly_201509_c20151005.tar.gz
ftp://ftp.ncdc.noaa.gov/pub/data/climgrid/nClimGrid_v1.0-preliminary_monthly_201509_c20151007.tar.gz
A short term solution is to just use the second of these since it appears to have been processed a couple of days after the first one, assuming that the last number in the file name represents the processing date.
This issue has been reported to NCEI and hopefully one of these files will be removed in short order, but until then we can add a hack to ignore the first of these two files.