- Security: This release includes changes that resolve CVE-2026-16637
- Cleanup pass: retired the cloudwatch-logback-appender jar, retired the Hyrax Gateway WebUI/API entirely (#308), and dropped leftover echo-token comments (#306, #304).
- "Dap4 package purge" (#287): removed leftover cruft from old WCS work, updated build.gradle, cleaned up commented-out production rules.
- Fixed
RequestCacheinitialization bugs in the Hyrax JSP pages, includingrobots.jsp(#302, #303). - Fixed a request-cache init bug: removed the retired
HAI_DEBUG_LOGGERappender and added error logging toRequestCache.getRequestId()(#301). - Replicated request-log info (collectionId, query_string) into the response log for NGAP troubleshooting (#298).
- Added a JSON-encoded OLFS log appender (#297); retired the w10n service entirely (#296); added an HTTP
Serverresponse header in Apache-style format (#295). - Fixed the Hyrax version shown on the DAP4 "contents" webui page, pulling it from the
HyraxVersionclass; retired old OSX build assets (#293, #291). - Refactored how Hyrax/OLFS version numbers are determined, stopped baking Hyrax version numbers into OLFS code/hyrax-docker recipes, and did a round of Sonar fixes (#286, #284, #289).
- Retired WCS (Web Coverage Service) code and resources.
- CI/build tweaks: enabled multi-OS Travis builds for hyrax-docker, turned on el9 builds, retargeted hyrax-docker for better build semantics, added
TARGET_OS/TOMCAT_MAJOR_VERSIONto the CI-to-docker handoff (#279, #280). - Added auto-merge support for tracking branches, fixed an issue updating a tracking branch, streamlined
update_tracking_branch.sh(#275, #276). - Refactored the "show" target used in the Ant production rules (#277).
- A back-and-forth on the Hyrax Admin Interface (HAI): retired it, then reverted the retirement, then re-applied the retirement ("Rerewind admin retirement"); also allowed JS files as a
returnToUrltarget, droppedadmin/haifromUrlRewrite, and added support for tracking test deployments (#263, #264, #267, #268, #269, #271, #273). - Check the query string for a client-submitted request id (#243).
- Fixed a logout-redirect-to-nowhere bug when running in the ROOT context, added a config parameter for the post-logout redirect target (#245); completed transmitting the EDL application client id to the BES (#257).
- Switched the interactive-tutorials link to the NASA tutorials (#255).
- Stopped logging "EDL token validation" when there was no token to validate, and switched to reading the user id from the EDL token instead of calling EDL directly (#249, #251).
- Added CloudWatch profiling logs/instrumentation across the service chain (token auth start/end, request id, cache id) (#239).
- Improved session management (#240): new session after auth, client-info/
X-Forwarded-Forchecks, configurable session timeout with a warning on invalid values, token revocation on logout, landing-page content control, bumpedweb.xmlto version 4.0. - Turned DAP4 checksums on all the time for the NGAP service, then fixed a bug where the checksum context could leak stale state across requests (#242, #244).
- Downgraded Travis's Gradle version back to 8.14.3, and set the Gradle Java source-compatibility level to 1.9 (#238).
- Fixed banner display issues in JSP pages, updated
help.jspfor DAP4 (#236). - Added optional DAP4 data responses with checksums: accepts a
dap4.checksumquery parameter and passes its state to the BES viasetContext(#235). - Added
dap4:Enumto the list of types incompatible with netCDF-3 (#234). - Multi-pass fix to SonarQube/SonarCloud CI integration: PR interop, branch-name handling, raw-scan step, master-branch scanning (#228, #229, #230, #232).
- Built out EDL token self-validation end-to-end (local JWT/JWKS public-key validation,
getEdlUserIdFromToken, unit tests forUrsIdP), then removed a stalegson-2.3.1dependency (#225, #226). - Added DMR++ retrieval support to the NGAP module: a
get dmr++document, a character-encoding class refactor, and config control for showing DMR++ response links (#221). - Retired the
build_dmrppservice from OLFS (#222). - Protected request/response/BES application logs from duplicate request ids via a new
RequestIDclass pairing each request id with a UUID; updated JUnit/Hamcrest (#214). - Added
query_stringto CloudWatch request logs, and normalizedrequestIdusage across the codebase, adding BES client id and client command count to the BES command root element (#212, #213). - Updated the landing-page NASA official's name (#211); @ndp-opendap briefly tried and then reverted an experimental change disentangling the user's request id from the BES client id.
- Documentation cleanup: moved
olfs.adocout to the separate documentation project, retired unused classes/XSLT, and worked through a broken Travis CI issue (mostly WIP commits, condensed here). - Possible fix for an NNVAP-related issue on the DAP4 request form (#210).
- Reordered the DAP POST handlers so
NgapDispatchHandleris evaluated beforeBesDapDispatchHandler(#209). - Logging cleanup: use
-1to indicate unknownoutput_size, drop units fromresponse_sizein logs (#207, #208). - Small changes to the request-log record format for NGAP metrics (#206).
- Began a Redisson upgrade to 3.22.0: added the new jars and updated logback/slf4j dependencies, but excluded the new Redisson jars from production builds for now; updated
build_dmrppproduction rules to match (#203, #204, #205). - Updated memcached to run under Tomcat 9.
- Updated the README/build offsets and bumped the OLFS release version to 1.81.15.
- Updated logo.
- Added logback appender for AWS CloudWatch.
- Updated landing page on NGAP.
- OLFS now uses Java17 (jdk17)
- Changed maxResponseSize type from int to long.
- Units scalar set to 2^10 (kilobytes) when the units value is junk.
- preformatted text for 400 errors
- Reduced beslistener pool from 200 to 25 for NGAP deployments
- Fixes EDL token handling so that redirected are not issued and session (aka cookie) management is not required of the client.
- Setting BES timeOut to 55 seconds (thanks CloudFront) for the NGAP deployment
- Added Attributes with DAP4 types to the "No NetCDF-3 Downloads For You" feature.
- Updated ncWMS to version 2.5.2
- Migrated to Java-11
- Retiring various XSLT stuff
- Fixed dap4Contents.xsl so that the viewers link works.
- Migrated the schema.org annotations from the node_contents.xsl to dap4Contents.xsl
- Rewrote schema.org links to https.
- Removed commented out code.
- Fixed page formatting with new css type
Nothing to see here
Updated ReqInfo.getRequestUrlPath() so that it utilizes the request headers:
-
- "CloudFront-Forwarded-Proto"
-
- "X-Forwarded-Proto"
-
- "X-Forwarded-Port" In reconstructing the "request url" dereferenced by the initiating client. This means the if some upstream entity rewrites the URL (Like CloudFront does when it drops the https protocol in favor of http for internal access) it can be detected and the links built by the server and returned to the client are now correct.
loop.
Request form context sensitive. If the dataset in question contains variables whose data types are found in DAP4 and not in DAP2/NetCDF-3 then the buttons are disabled. A more complete solution is envisioned where the projected variables are assessed and if only DAP2/NetCDF-3 types are selected then the buttons would would be enabled. This fix is only a step in that more dynamic direction.
In order to make the server's behavior more understandable to users, and its configuration more understandable to admins we have changed the way that the server responds to client requests for the unadorned Dataset URL and the way that the server generates Data Request Form links in its catalog pages. There are new configuration parameters to control these behaviors.
This configuration parameter has been DEPRECATED - Use the and elements to configure this behavior (see below).
The DatasetUrlResponse element is used to configure the type of response that the server will generate when a client attempts to access the Dataset URL. The type of response is controlled by the value of the type attribute.
Allowed Values:
- dsr - The dap4 DSR response will be returned for the dataset URL. Note: This setting is not compatible with DataRequestForm type of "dap2" as the DSR response URL collides with the DAP2 Data Request Form URL.
- download - If the configuration parameter AllowDirectDataSourceAccess is set (present) thenthe source data file will be returned for the dataset URL. If the configuration parameter AllowDirectDataSourceAccess is not present then a 403 forbidden will be returned for the dataset URL. (This is basically a file retrieval service, any constraint expression submitted with the unadorned dataset URL will be ignored.)
- requestForm - The Hyrax Data Request Form Page will be returned for the dataset URL.
Default: download
The DataRequestForm element defines the target DAP data model for the dataset links in the "blue-bar" catalog.html pages. These links point to the DAP Data Request Form for each dataset. This element also determines the type of Data request form page returned when the DatasetUrlResponse type="requestForm" and the request is for the Dataset URL. Allowed type values are: dap2 and dap4.
Default: dap4
When enabled users will be able to use Hyrax as a file server and download the underlying data files/granules/objects directly, without utilizing the DAP APIs.
Default: disabled.
The presence of this element will cause the Data Request Form interfaces to "force" the dataset URL to HTTPS. This is useful for situations where the sever is sitting behind a connection management tool (like AWS CloudFront) whose outward facing connections are HTTPS but Hyrax is not using HTTPS. Thus the internal URLs being received by Hyrax are on HTTP. When these URLs are exposed via the Data Request Forms they can cause some client's to have issues with session dropping because the protocols are not consistent.
Default: disabled
- Updated the dependency libraries as follows:
- Upgraded gson-2.3.1 to gson-2.8.9
- Upgraded slf4j-1.7.16 to slf4j-1.7.32
- Upgraded logback-core-1.1.11 to logback-core-1.2.9
- Upgraded logback-classic-1.2.0 to logback-classic-1.2.9
- Log entries for User-Agent, URL path, and query string are now scrubbed.
- Dropped link to in DAP2 Data Request Form to the no longer supported BES generated request form.
- Fixed gradle dependancies according snyk scan.
- Session serialization
- Added session manager jars to ngap resources.
- Updated production rules to include session manager code in the ngap application build.
- Made history_json output as JSONArray.
- New landing page for NGAP service.
- Added UNIX time to OLFS logs in NGAP.
- Fixed bug in computation of dimension sizes in the dap4 ifh xsl.
- Dropped DSR response for just the dataset_url.
- Fixed broken json error output.
- Changed the broken naming pattern for the PPT strings.
- Enabled ChunkedInputStream debugging and cleaned up the messages in support of stream pollution problem.
- Fixed broken css and deployment context links construction in DSR HTML page.
- Added file size and last modified headers to flat file transfer.
- Added auth.log independant of debug log.
- First pass at patching POST request handling.
- Changes to our packaging of the ngap-snapshot docker images.
- Added support for EDL token chaining (in concert with BES modifications)
- Improved BES connection lifecycle reliability improvements.
- Improved the servers response for HEAD requests, now the server correctly returns 405 Method Not Allowed
- Fixed bug in request routing for DAP4 requests.
- Fixed the JSON-LD content in the Data Request Form so that the required description property is generated from the data set using a heuristic, or if that fails a generic string is used. The length of this mandatory property is now correctly limited to [50 < length < 5000]
- Started developing Gradle production rules.
- Added an ngap service endpoint in support of the NASA NGAP project.
- Improved content encoding/scrubbing on outbound error messages.
- Fixed bugs in bootstrap init and init for all the various init() components.
- Stabilized the order of output in WCS-2.0
-
Added (alpha) support support for server authentication credentials
- Environment injection support for besstandalone and besd
- A credentials configuration file for multiple sets, in which credentials are associated with URL prefixes and multiple credentials can be defined in a bes.conf environment.
-
Combined Logs: The server can now be configured to add the OLFS request log content to the BES log.
-
Earthdata Login User Authentication Support
-
The Data Request Form requires data selection is now a configuration parameter, <RequireUserSelection /> in the olfs.xml file.
-
NoDynamicNavigation - Added a trial configuration parameter <NoDynamicNavigation/> to the olfs.xml file. If this element is uncommented in the olfs.xml file then all of the dynamically generated catalog/navigation pages will be disabled. When enabled the server admin must either supply and maintain THREDDS catalogs to be read by the server, or they have to provide their own system of navigation and discovery to generate links to the dataset endpoints (ex. Data Request Form)
- No DAS construction for data response.
- Memory leak fixed and unneeded copy removed in awsv4 code
- CovJson patch.
- ncml_handler: When an NCML aggregation named a dataset that the server could not read... crsh. Fixed..
- Server cache stabilization
- hdf4_handler: Fixed memory leaks
- hdf5_handler: Fixed memory leaks
OPeNDAP Data Request Forms and catalgo pages.
- Corrected JSON-LD encoding in the IFH generated pages.
- Corrected javascript code generation problems.
Path construction bug (causing double "/" characters) in THREDDS catalog presentation pages. (HK-338)
Security analysis and response.
- Sonar Scan
- Coverity Scan
-
Added in mempory LRU cache for BES showNode responses. This is a performance enhancement. (HK-366)
-
Added a site map cache file so that site maps for large instances of Hyrax(>500k datasets) can be navigated in a few seconds and not hours. (HK-375)
-
Updated Docker containers so that Tomcat is producing compressed responses when the requesting client indicates that this is acceptable. (HK-329)
-
Added response size logging to all metadata and data responses. Error responses do not have ssociated response size values in the log.
- Added missing code to support all DAP2/4 atomic numeric types in WCS-2.0 (HK-372)
-
Added a context/request sensitive message to all support email links.
-
Added support email links to directory/collection pages in addition to error pages.
-
Decoupled support email configuration from site/server administrator information.
-
Added sitemap generation and an (optional) ROOT.war service that adds a robots.txt service for said sitemap.
-
Enable THREDDS catalog brokering for remote catalogs.
-
Corrected the CovJson response Content-Type headers
-
Repaired incompatibility between IFH Request forms and CentOS-6
-
Fixed odd port problem with redirect code.
-
Repaired Viewers pages to correctly process encoded URL content.
-
Added new (beta) dynamic siteMap and robots.txt service.
-
Added confguration (to BES) for Site admin contact information that also gets used in the JSON-LD DataCatalog content.
- Minor changes mostly a release to support changes in the BES.
-
NEW! Improved! The OPeNDAP Data Request Form!
-
CovJson responses for compatible datasets.
-
Simplified the OLFS configuration file, olfs.xml (If you are upgrading you will need to replace your existing file with the new default one and edit to recover your changes.)
-
JSON-LD now in every contents.html page and in every Data Request Form.
-
Hyrax now supports the browsing of remote THREDDS catalogs. This feature is disabled by default, and when enabled will only be allowed to access remote systems based on the Gateway.Whitelist in the BES.
-
Added optional login controls and framework for adding URS (or other) authentication and identification providers.
-
Switched to the OWASP string encoder/decoder.
-
Added an additional default configuration location '/use/share/olfs'
-
Added URI encoding to requests generated by the Data Access Forms so that special characters such as '[' and ']' are now encoded correctly.
-
Applied a work-around patch to the THREDDS catalogs generated by Hyrax
to accomodate a java-netcdf problem in which no timezone abbreviations other than Z (or none at all) are allowed. The patch maps known GMT equivalents to Z and all other timzones are simpley dropped (which is also an allowed representation by java-netcdf) -
Repaired Gateway Service UI so that both DAP2 and DAP4 response are represented and functioning.
-
Updated URS client implementation to work with current code base.
-
Added WCS service.
- Works using templates associated by regex.
-
Made the OLFS authentication aware in its http activities by having it utilize the ~/.netrc credentials source or one of the same format specified in the configuration.
-
Fixed bug in the generation of the "viewers" pages.
-
Repaired broken RDF responses.
-
Updated dependecy Jars.
-
Enabled ThreddsCatalogUtil to authenticate against Earthdata Login
-
Merged eo-wcs features into master
-
Server timeout conditions now return HTTP status 504 (and not 418).
-
Bug fixes in the DAP4 Data Request Form pages.
-
Improved logging, including log rotation.
-
Fixed a null pointer exception in the AggregationServlet (the component that performs user-specified aggregations).
-
Repaired viewers.xml configuration for Godiva3
-
Issues with the netcdf file response History attribute were fixed, once again...
-
Prompted by user requests we have changed the HTTP status returned when the server encounters a BES timeout error from 418 (I'm a teapot) to 504 (Gateway Timeout) We really liked the teapot because we felt applying the 504 (Gateway Timeout) status to this error strains the formal definition of the 504 status code and could lead to confusion down the road. Time will tell...
-
Updated OLFS configuration defaults in viewers.xml file to reflect changes for ncWMS2 and Godiva3
-
Patched a problem with the way the XSL transforms are carried out that could have left the process running in an unexpected directory.
-
Added code to ammend the CF "history" attribute so that fileout_netcdf responses contain the ammended value that shows that Hyrax produced it along withg the constraint and id information of the dataset.
-
Added new rules for the Content-Disposition header download file names. Now the old behavior (add the netcdf file suffix to very file's basename) can be replaced so that if the suffix will end up as a duplicate (for example file.nc.nc or file.nc4.nc4), nothing is appended.
-
Added memory cache for BES Catalog responses. This allows the OLFS to stop asking the same things over and over again. The size and refresh interval are specified by the " element in the olfs.xml file. If the element is missing then the memory cache will be disabled.
-
Increased the number of simultaneous requests from 10 to 200 , which is the default limit for Tomcat.
-
Bug fixes
- Large file issues in w10n.
- Fixed build for WCS-2.0
-
The OLFS regression test harness has been migrated from DejaGnu to AutoTest. This makes writing and maintaining tests much easier. All of the DejaGnu tests have been migrated and many additional tests added. To configure the tests: autoconf -vif ./configure To run the tests: make check
-
Error handling has been refactored. Error messages from libdap and the BES are now correctly transmitted to the user. Additionally the HTTP status codes and associated pages are now working correctly and there should be a significant reduction in the number of "superman" (Internal Error) pages encountered in normal operation.
-
The default size of the BES client pool has been increased from 10 to 200 in response to numerous performance evaluators that failed to recognize that this could be done to increase performance in high traffic environments.
-
The OLFS now keeps the BES on a timeout leash so that BES requests that take longer than the time out interval stated in the olfs.xml will actually timeout and return an associated error to the OLFS.
-
!!!!! The location of the OLFS configuration directory has changed !!!! This change should resolve installation problems with Tomcat instances that are installed from RPM. By default the OLFS will utilize it's bundled configuration in the directory $CATALINA_HOME/webapps/opendap/WEB-INF/conf In order to localize the configuration so that your configuration changes are persistent you will need to do one of the following:
- For the user that will be running the OLFS (the Tomcat user), set the environment variable OLFS_CONFIG_DIR to an existing directory to which the Tomcat user has both read and write privileges. OR
- Create the directory /etc/olfs and set it's permissions/ownership so that the Tomcat user has both read and write permission. (If both of these steps are done then priority is given to the environment variable.) Restart Tomcat. When it starts the OLFS will check these locations and then install a copy of its default configuration into the new spot. Edit the configuration files as needed. If, for example, your beslistener is not running on localhost:10022 then you'll need to edit the olfs.xml file in the configuration directory and adjust the and values to reflect your situation.
Worked under the hood to ensure that Hyrax's character encoding is actually UTF-8.
Rollback from Java 8 requirement. Now only Java 7 is needed for the OLFS and Hyrax. Many bug fixes. Removal of unused parts of the API.
New patched bugs in datasetScan implementation.
New datasetScan implementation for THREDDS catalogs provides for all datasetScan activities as described here: http://www.unidata.ucar.edu/software/thredds/v4.6/tds/reference/DatasetScan.html As of the day of this writing.
New w10n navigation and data access service. Updated ncWMS integration. Bug fix in THREDDS catalog inherited metadata injection. New Swath data aggregation service.
Disabled bug in our viewers page that was preventing some valid WMS datasets from receiving WMS and Godiva links.
Refactored OLFS configuration file. Installing into an existing Hyrax instance will require installing the new opendap.war file; restarting tomcat; manually copying the file: $CATALINA_HOME/opendap/initialContent/olfs.xml to: $CATALINA_HOME/content/opendap/olfs.xml and restarting Tomcat again. Added security patches.
Repaired bug that was introducing whitespace characters into the MIME Type strings extracted from the clients Accept header, and thus causing ' the server to incorrectly return a HTTP status of 406
Refactored OLFS configuration. Added JSON and IJSN responses. Added WMS support via ncWMS integration.
Repaired bug that was breaking Hyrax's ability to serve plain files (not otherwise recognized as 'data' by the BES)".
Repaired broken nciso production rule.
Fixed a bug in the WebStart system.
Internal (but as yet not fully released support) for DAP4 responses, along with stored_result/asynchronous behavior Bug fixes. Major changes to internal API.
Support for geotiff and jpeg2000 responses. Bug fixes. Cruft code removal from code tree. Support for constraint expressions as POST bodies.
I created the NEWS file with version 1.10.0 so no prior versions are represented here.