- Bump gremlin-driver to 3.7.6
- Bump commons-beanutils to 1.11.0
- Bump plexus-utils to 4.0.3
- Bump jackson-databind to 2.18.2
- Fix S3 config file download failure when cleanOutputPath is true
- Bump Netty to 4.2.12.Final
- Use SPARQL over GSP for complete and named-graph exports in
export-rdffor Neptune versions >= 1.3.2.0 - Updated
NeptuneSparqlClient create()method signature to require NeptuneClusterMetadata - Add new parameter
expectedBucketOwnertoexport-serviceto validate target S3 buckets belong to the expected account
- Fix missing graph fields for complete and named-graph exports in
export-rdfwithnquadsformat - Fix --clone-cluster-enable-audit-logs overriding other Neptune cluster parameters
- Fix fallback for finding parameter group families via Neptune engine version
- Fix inability to upload large files (>5GB) to S3
This release fixes a bug where headers were previously missing from the default csv format for export-pg-from-queries. With this fix, headers will now be present by default in export-pg-from-queries. Use --format csvNoHeaders or "format": "csvNoHeaders" to revert to export as CSV without headers.
- Migrate to AWS SDK for Java v2
- Use the jobId from the Neptune Export Service (AWS Batch jobId) as the default exportId if available
- Enable csv file rewrites for
export-pg-from-queriesto ensure output conforms to inferred schema
- Add retries to Neptune clone creation
- Fix missing csv headers from
export-pg-from-queries - Disable DFE for all Gremlin queries
- Fix incorrect URL scheme for network load balancer endpoints
- Update TinkerPop to 3.7.3 and JanusGraph to 1.1.0
- Improve GremlinFilter compilation efficiency for highly concurrent exports
- Bump AWS SDK to 1.12.773
- Bump commons-io to 2.15.1
- Bump aws-lambda-java-core to 1.2.3
- Fix RDF GSP error extraction
- Disable client-side verification of URI syntax for export-rdf
- Throws exception in cases where S3 Handler cannot find local files
- Support Neptune 1.3 parameter groups when
describeDBClusterParameterGroupsunavailable - Surface server errors for RDF GSP exports
- Use AWS Batch Job ID as Neptune Export ID when available
- Fix bug in
export-rdfto Kinesis where Neptune Export may terminate before all data has been written to the stream.
- Add '--clone-cluster-enable-audit-logs' argument which enables audit logging on cloned clusters. Audit logging is disabled by default, which aligns with previous behavior.
- Update
--gremlin-filtersto block use of mutating stepsmergeV()andmergeE()
- Add
--split-queriesoption toexport-pg-from-queries. When invoked,range()steps will be injected in the beginning of queries, and they will be split according to the configured concurrency. - Add
--named-graphargument toexport-rdf. This argument allows for a single named graph URI to be specified to limit the scope of the export.
- Add support for
neptune_mlprofile toexport-pg-from-queries.
- Use Graph Store Protocol for complete RDF graph exports, improving performance for large exports.
- Resolves issue where certain special characters would cause RDF export jobs to fail.
- Resolves issue in which RDF outputs may contain unexpected and potentially faulty prefixes.
-
Introduce
--structured-outputCLI option toexport-pg-from-queries. This option, when used in conjunction with--format csv, will produce CSV output matching the structure of the Neptune bulk loader's gremlin data format. This is the same format as produced byexport-pg --format csv. The use of this option requires that queries produce elementMap()'s of nodes and edges. -
Add
--filter-edges-earlyoption to property graph exports. This option forcesgremlinFiltersto apply before the range() step which breaks up concurrent traversals. This may lead to improved performance in cases where the gremlinFilters are efficient and filter out the majority of edges.
- Cross Account Exports: New CLI options and parameters added to specify a role to assume when uploading to Amazon S3 buckets or Amazon Kinesis Data Streams
- New --credentials-profile CLI option to fetch AWS Credentials from non-default AWS CLI profiles
- Fixed bug which could lead to corrupted output in highly concurrent csv exports
- Resolves issue which caused the error
gremlin-groovy is not an available GremlinScriptEngineto appear when using the--gremlin-filteroption in the uber jar.
- Upgraded to Gremlin Dependency Version to 3.6.2
- Fixed NullPointerException bug during RDF Exports
- Added aws-java-sdk-sts as a dependency to enable WebIdentityTokenFileCredentialsProvider.
- Fixed bug which was preventing getting the output id in FileToStreamOutputWriter
- Updated NOTICE file in shaded jar
-
Added new
--disable-stream-aggregationoption for property graph exports to Kinesis streams. More details can be found here. -
Improved error messages from server side errors (such as timeout exceptions) for RDF exports.
Neptune Export is a tool to perform bulk data exports from AWS Neptune. Neptune Export is migrated from the AWS Labs Amazon Neptune Tools repository, and the old module is now deprecated. In this release, the release artifact neptune-export.jar has been renamed to neptune-export-1.0.0-all.jar. Going forward, Neptune Export will be following this new versioned naming scheme.
Instructions for running export jobs can be found in the docs/ directory.
A few changes are included since the migration.
- Corrected r6g instance type prefix (used to be listed as r6d).
- Added a new optional parameter to use customer managed KMS key for S3 server-side encryption.
- Added integration tests for developers requiring manual setup (see docs/dev/IntegrationTests.md).