Skip to content

Improve Handling of Cutoff Dates via CLI #49

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
johannesduesing merged 6 commits into from
Mar 6, 2026
Merged

Improve Handling of Cutoff Dates via CLI #49

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e8ca3aa
Accept YYYY-MM-DD format for cutoff dates via CLI. Change UNIX timest…
johannesduesing Jan 12, 2026
9316849
Update documentation for since-until in config builder, add convenien…
johannesduesing Jan 12, 2026
b28a4b2
Fix dependence on platform time zone
johannesduesing Jan 12, 2026
18ff1b0
Fix test dependence on index ordering
johannesduesing Mar 6, 2026
3eb79e2
Fix Javadoc warning
johannesduesing Mar 6, 2026
e78b386
Merge branch 'fix/test-data-dependencies' into feature/cutoff-dates
johannesduesing Mar 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,16 +29,16 @@ Additionally, the `MavenCentralArtifactAnalysis` provides the `resolveIndex` opt

Both analysis types provide a method called `void runAnalysis(String[] args)`. Invoking this method will start the analysis. By default, MARIN analyses will support the following CLI parameters:

| **Argument** | **Artifact Analysis** | **Library Analysis** | **Description** | **Example** |
|------------------------------------------------------------|-----------------------|----------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------|
| `-st <s>:<t>` <br/>`--skip-take <s>:<t>` | Yes | Yes | Skips the first `n` inputs and then only <br/> processes the next `t` ones. | `-st 200:10` |
| `-su <s>:<u>` <br/> `--since-until <s>:<u>` | Yes | No | Skips artifacts released before the <br/> timestamp `s` or after timestamp `t`. | |
| `-i <filepath>`<br/>`--inputs <filepath>` | Yes | Yes | Processes inputs from a file instead of the<br/> Maven Central index. Expects on input<br/>per line. Inputs are either G:A:V triple (artifacts)<br/>or G:A tuple (libraries). | `-i libraries.txt` |
| `-pof <filepath>`<br/>`--progress-output-file <filepath>` | Yes | Yes | File to periodically write number of processed <br/>artifacts to. Defaults to `./marin-progress`. | `-pof marin-progress` |
| `-prf <filename>`<br/>`--progress-restore-file <filepath>` | Yes | Yes | File to load a previous run's progress from. <br/>Inputs will be skipped until progress is restored.<br/> Not used by default. | `-prf marin-progress` |
| `-spi <num>` <br/> `--save-progress-interval <num>` | Yes | Yes | Number of inputs after which to store progress. <br/> Defaults to 100. | `-spi 10` |
| `-t <num>` <br/> `--threads <num>` | Yes | Yes | Number of threads to use. Defaults to 1. | `-t 8` |
| `-o <dir>` <br/> `--output <dir>` | Yes | No | Output directory to optionally write processed <br/> artifacts to. Depending on the artifact information<br/> required by the analysis, this can be `pom.xml`<br/>files, `*.jar` files or GAV triple. Not used by default. | `-o ./jars-processed/` |
| **Argument** | **Artifact Analysis** | **Library Analysis** | **Description** | **Example** |
|------------------------------------------------------------|-----------------------|----------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------|
| `-st <s>:<t>` <br/>`--skip-take <s>:<t>` | Yes | Yes | Skips the first `<s>` inputs and then only processes the next `<t>` ones. | `-st 200:10` |
| `-su <s>:<u>` <br/> `--since-until <s>:<u>` | Yes | No | Skips artifacts released before the timestamp `<s>` or after timestamp `<u>`. A timestamp is either a UNIX timestamp (in seconds) or a date of format YYYY-MM-DD. Note that `<u>` is automatically parsed as end-of-day if YYYY-MM-DD is used. | `-su 1768209126:2026-02-02` |
| `-i <filepath>`<br/>`--inputs <filepath>` | Yes | Yes | Processes inputs from a file instead of the Maven Central index. Expects on input per line. Inputs are either G:A:V triple (artifacts) or G:A tuple (libraries). | `-i libraries.txt` |
| `-pof <filepath>`<br/>`--progress-output-file <filepath>` | Yes | Yes | File to periodically write number of processed artifacts to. Defaults to `./marin-progress`. | `-pof marin-progress` |
| `-prf <filename>`<br/>`--progress-restore-file <filepath>` | Yes | Yes | File to load a previous run's progress from. Inputs will be skipped until progress is restored. Not used by default. | `-prf marin-progress` |
| `-spi <num>` <br/> `--save-progress-interval <num>` | Yes | Yes | Number of inputs after which to store progress. Defaults to 100. | `-spi 10` |
| `-t <num>` <br/> `--threads <num>` | Yes | Yes | Number of threads to use. Defaults to 1. | `-t 8` |
| `-o <dir>` <br/> `--output <dir>` | Yes | No | Output directory to optionally write processed artifacts to. Depending on the artifact information required by the analysis, this can be `pom.xml` files, `*.jar` files or GAV triple. Not used by default. | `-o ./jars-processed/` |


If you do not want to extend one of the aforementioned base classes, MARIN also provides corresponding implementations of the `java.util.Iterator` interface to enumerate and enrich artifacts or libraries. Note that those implementations are **not threadsafe** and cannot perform resolution in parallel. MARIN provides:
Expand Down
244 changes: 0 additions & 244 deletions src/main/java/org/tudo/sse/IndexWalker.java
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion src/main/java/org/tudo/sse/analyses/MavenCentralLibraryAnalysis.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -276,7 +276,7 @@ private List<ArtifactIdent> getReleaseIdentifiers(String groupId, String artifac
}
return identifiers;
} catch (Exception x) {
log.warn("Failed to obtain version list for library {}:{}", groupId, artifactId, x);
log.warn("Failed to obtain version list for library {}:{} ({})", groupId, artifactId, x.getCause().getMessage());

try { this.onVersionListError(groupId, artifactId, x); }
catch(Exception inner) {
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/org/tudo/sse/analyses/config/ArtifactAnalysisConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@ public class ArtifactAnalysisConfig extends LibraryAnalysisConfig {
static final Path DEFAULT_VALUE_OUTPUT = null;

/**
* Timestamp before which artifacts shall be excluded from analysis, or -1 if disabled.
* UNIX timestamp (in milliseconds) before which artifacts shall be excluded from analysis, or -1 if disabled.
*/
public long since;

/**
* Timestamp after which artifacts shall be excluded from analysis, or -1 if disabled.
* UNIX timestamp (in milliseconds) after which artifacts shall be excluded from analysis, or -1 if disabled.
*/
public long until;

Expand Down
18 changes: 16 additions & 2 deletions src/main/java/org/tudo/sse/analyses/config/ArtifactAnalysisConfigBuilder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import java.nio.file.Files;
import java.nio.file.Path;
import java.time.ZonedDateTime;

/**
* Configuration builder to obtain {@link ArtifactAnalysisConfig} instances programmatically.
Expand Down Expand Up @@ -104,8 +105,21 @@ public ArtifactAnalysisConfigBuilder withOutputDirectory(Path outDir) throws Inv
* Sets a time-based range to filter artifacts for. The resulting analysis will only process artifacts that have
* been released after the timestamp given in 'since', and before 'until'.
*
* @param since Timestamp marking the lower limit of the range
* @param until Timestamp marking the upper limit of the range
* @param since DateTime marking the lower limit of the range
* @param until DateTime marking the upper limit of the range
* @return The current configuration builder instance
* @throws InvalidConfigurationException If the given configuration values are not valid
*/
public ArtifactAnalysisConfigBuilder withSinceUntil(ZonedDateTime since, ZonedDateTime until) throws InvalidConfigurationException {
return this.withSinceUtil(since.toInstant().toEpochMilli(), until.toInstant().toEpochMilli());
}

/**
* Sets a time-based range to filter artifacts for. The resulting analysis will only process artifacts that have
* been released after the timestamp given in 'since', and before 'until'.
*
* @param since UNIX timestamp in milliseconds marking the lower limit of the range
* @param until UNIX timestamp in milliseconds marking the upper limit of the range
* @return The current configuration builder instance
* @throws InvalidConfigurationException If the given configuration values are not valid
*/
Expand Down
Loading
Loading