Error: [crop] cannot create dataset

[lbarqueira]

Thank you for the great tutorial.

I am trying to reproduce your code.

When I run the following code:

forest_height_rasters <- lapply( forest_height_list, function(x) { terra::crop( x, terra::vect( portugal_sf ), snap = "in", mask = T ) } )
The following error is shown in the console pane:

Error: [crop] cannot create dataset

Any idea on how to solve this issue?

Thank you very much,

Luis Barqueira

[milos-agathon]

Hi Luis,

I’m sorry to hear that you are having trouble creating a dataset. There could be several possible reasons for this error, depending on the context and the source of your data.

If you are trying to create a dataset from a raster file, you may encounter this error if you identify the raster file in a directory different from your working directory.

Can you check if this is the case?
Milos

[lbarqueira]

Thank you. That is not the case.
The problem has to do with calling mask = T, in the above code.
Is calling mask really important, or can I ignore it in the above code?

Thank you,

Luis Barqueira

[milos-agathon]

Thanks for your clarification!
mask = T declares all the raster values outside of the overlay object to be NA. In your case, terra::crop will only return the area of Portugal

[lbarqueira]

Thanks Milos.

[ENM256]

Dear Milos,
I saw your tutorial on yt and was eager to try it out for Germany. Thank you so much for sharing your approach in this level of detail.

In general it works but I have a similar issue as Luis:

When creating the forest_height_rasters list, it breaks somewhere in the lapply loop and throws this error:
"Fehler: [crop] file exists. You can use 'overwrite=TRUE' to overwrite it"

I can suppress this by setting the overwrite=T argument but this results in the data for the N54 tiles missing.
I posted the list of tiles below.

By the way, the server addresses have changed, therefore your script needs some adaptations to retrieve the destfile argument when downloading. I used: destfile = sub(".*=","", basename(url))

If you have any idea how this can be solved I would apprechiate it very much, this specific error seems rather uncommon.

urls = c(
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N54E006_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N54E009_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N54E012_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N51E006_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N51E009_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N51E012_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N48E006_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N48E009_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N48E012_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N45E006_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N45E009_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N45E012_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N51E003_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N48E003_Map.tif",
"https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N51E015_Map.tif"
)

As I can grab the tiles below N54, I suspected it might have something to do with the isolated polygons (islands) in Northern Germany? This is how I curated the geodata for cropping:

#get geodata for cropping (for Germany)
get_country_borders <- function(){
main_path <- getwd()
country_borders <- geodata::gadm(
country = "DEU",
level = 1,
path = main_path
) |>
sf::st_as_sf()
return(country_borders)
}

country_borders <- get_country_borders()

#define area to crop

cropped_area <- country_borders |>
sf::st_union()

[Juan-avzla]

Greetings Milos

I'm getting an error at the moment of downloading ETH data. I tried to use ENM256 approach to fix the URL problem, but it resulted in a timeout.

Error in download.file(url, destfile = sub(".*=", "", basename(url)),  : 
  download from 'https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N21W105_Map.tif' failed

Warning messages:
1: In download.file(url, destfile = sub(".*=", "", basename(url)),  :
  downloaded length 0 != reported length 0
2: In download.file(url, destfile = sub(".*=", "", basename(url)),  :
  URL 'https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N21W105_Map.tif': Timeout of 60 seconds was reached

[Juan-avzla]

Dear Milos, I saw your tutorial on yt and was eager to try it out for Germany. Thank you so much for sharing your approach in this level of detail.

In general it works but I have a similar issue as Luis:

When creating the forest_height_rasters list, it breaks somewhere in the lapply loop and throws this error: "Fehler: [crop] file exists. You can use 'overwrite=TRUE' to overwrite it"

I can suppress this by setting the overwrite=T argument but this results in the data for the N54 tiles missing. I posted the list of tiles below.

By the way, the server addresses have changed, therefore your script needs some adaptations to retrieve the destfile argument when downloading. I used: destfile = sub(".*=","", basename(url))

If you have any idea how this can be solved I would apprechiate it very much, this specific error seems rather uncommon.

urls = c( "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N54E006_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N54E009_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N54E012_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N51E006_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N51E009_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N51E012_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N48E006_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N48E009_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N48E012_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N45E006_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N45E009_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N45E012_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N51E003_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N48E003_Map.tif", "https://libdrive.ethz.ch/index.php/s/cO8or7iOe5dT2Rt/download?path=%2F3deg_cogs&files=ETH_GlobalCanopyHeight_10m_2020_N51E015_Map.tif" )

As I can grab the tiles below N54, I suspected it might have something to do with the isolated polygons (islands) in Northern Germany? This is how I curated the geodata for cropping:

#get geodata for cropping (for Germany) get_country_borders <- function(){ main_path <- getwd() country_borders <- geodata::gadm( country = "DEU", level = 1, path = main_path ) |> sf::st_as_sf() return(country_borders) }

country_borders <- get_country_borders()

#define area to crop

cropped_area <- country_borders |> sf::st_union()

Could you fix the issue with the server address to download ETH data?