Runtime difference between NORA3_wind_wave and individual wind/wave products

[hiro-takashina]

Noticed that using NORA3_wind_wave takes significantly more time to run compared to using NORA3_wind_sub and NORA3_wave_sub separately, even though I understand they are both using the same data.

The example below shows a short-term case, but the difference becomes even more pronounced for long-term periods.

example code

from metocean_api import ts
import time

# list of product
product_wave_only = 'NORA3_wave_sub'
product_wind_only = "NORA3_wind_sub"
product_wind_wave='NORA3_wind_wave'

# example
lon, lat = 9.1578, 63.7905  # Example coordinates
start_time, end_time = '2021-12-01', '2021-12-6'

# 'NORA3_wave_sub'
t = time.time()
df_ts_wave = ts.TimeSeries(lon=lon, lat=lat, start_time=start_time, end_time=end_time, product=product_wave_only)
df_ts_wave.import_data(save_csv=True, save_nc=False) 
print(f"wave: {time.time() - t:.2f} seconds")

# 'NORA3_wind_sub'
t = time.time()
df_ts_wind = ts.TimeSeries(lon=lon, lat=lat, start_time=start_time, end_time=end_time, product=product_wind_only)
df_ts_wind.import_data(save_csv=True, save_nc=False)
print(f"wind: {time.time() - t:.2f} seconds")

# 'NORA3_wind_wave'
t = time.time()
df_ts_combined = ts.TimeSeries(lon=lon, lat=lat, start_time=start_time, end_time=end_time, product=product_wind_wave)
df_ts_combined.import_data(save_csv=True, save_nc=False)
print(f"wind_wave: {time.time() - t:.2f} seconds")

output:

wave: 1.99 seconds
wind: 4.81 seconds
wind_wave: 25.74 seconds

[efvik]

Hi @hiro-takashina , thanks for opening the issue! This is a quite significant performance difference.

Yes, the data you retrieve should be the same, but the data is extracted from different sources. Looking at the source code of NORA3_wind_wave, the wind data is extracted from NORA3_wind_sub, but the wave data is extracted from NORA3_wave (not from NORA3_wave_sub). The difference is that NORA3_wave (on disk) consists of daily files instead of monthly, and also has a larger domain and therefore larger files. This makes it much slower to read from these files, especially when you need a long time-series.

To me it seems logical to change NORA3_wind_wave to use the same data source as NORA3_wave_sub. I think we will leave this issue open until it is resolved.

Meanwhile I recommend using the NORA3_wave_sub and NORA3_wind_sub separately.