Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -180,5 +180,5 @@ prescient-sdk-ts/tsconfig.json
*.env

# Smoke test build artifacts
smoke-tests/dotnet/obj/
smoke-tests/dotnet/bin/
prescient-sdk-ts/smoke-tests/dotnet/obj/
prescient-sdk-ts/smoke-tests/dotnet/bin/
136 changes: 63 additions & 73 deletions prescient-sdk-ts/JSII_MIGRATION_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,21 +150,22 @@ prescient-sdk/
│ │ ├── settings.ts ← env var loading + validation
│ │ ├── client.ts ← PrescientClient class
│ │ └── upload.ts ← upload() function
│ │ └── __tests__/ ← Jest unit tests (64 tests)
│ │ └── __tests__/ ← Jest unit tests (80 tests)
│ ├── smoke-tests/ ← Docker-based multi-language smoke tests
│ │ ├── docker-compose.yml
│ │ ├── docker/ ← Dockerfiles (python, go, dotnet, java)
│ │ ├── config.env ← fake values for smoke tests
│ │ ├── js/smoke.js
│ │ ├── python/smoke.py
│ │ ├── go/main.go
│ │ ├── dotnet/smoke.csproj + src/
│ │ └── java/pom.xml + src/
│ ├── dist/ ← compiled JS (gitignored)
│ ├── targets/ ← jsii-pacmak output (gitignored)
│ ├── justfile ← docker smoke-test runner
│ ├── justfile ← smoke-test runner (just docker)
│ ├── package.json
│ ├── pnpm-workspace.yaml ← nodeLinker: hoisted (required for bundling)
│ └── .gitignore
├── smoke-tests/ ← Docker-based multi-language smoke tests
│ ├── docker-compose.yml
│ ├── docker/ ← Dockerfiles (python, go, dotnet, java)
│ ├── js/smoke.js
│ ├── python/smoke.py
│ ├── go/main.go
│ ├── dotnet/smoke.csproj + src/
│ └── java/pom.xml + src/
└── (original Python SDK files — kept, not deleted)
```

Expand Down Expand Up @@ -223,7 +224,7 @@ prescient-sdk/

### Phase 3 — Settings ✅

- [x] `src/settings.ts`: read `process.env` + optional `.env` via `dotenv`
- [x] `src/settings.ts`: read `process.env` + explicit `PrescientClientOptions` (file loading added in Phase 8)
- [x] `_googleClientSecret` loaded from `PRESCIENT_GOOGLE_CLIENT_SECRET` only (never in Options)
- [x] Validate: MICROSOFT requires `tenantId`; GOOGLE requires `PRESCIENT_GOOGLE_CLIENT_SECRET`
- [x] Unit tests: missing required fields throw, env vars override `.env`
Expand Down Expand Up @@ -296,68 +297,56 @@ Docker-based multi-language smoke tests. No local toolchain required beyond Dock
- [x] `src/__tests__/upload.test.ts` — aws-sdk-client-mock for S3, file scan logic
- [x] 64 tests, 4 suites, all passing

### Phase 7 — CI/CD Pipeline

Update `.github/workflows/ci.yaml`:

```yaml
jobs:
build-and-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: '22' }
- uses: pnpm/action-setup@v4
with: { version: '11' }
- run: pnpm install
- run: pnpm run build # jsii
- run: pnpm test # jest

package:
needs: build-and-test
runs-on: ubuntu-latest
steps:
- run: pnpm run package # jsii-pacmak → targets/

publish-npm:
if: startsWith(github.ref, 'refs/tags/v')
needs: package
steps:
- run: pnpm publish

publish-pypi:
if: startsWith(github.ref, 'refs/tags/v')
needs: package
steps:
- uses: actions/setup-python@v5
- run: pip install twine && twine upload targets/python/dist/*

publish-nuget:
if: startsWith(github.ref, 'refs/tags/v')
needs: package
steps:
- uses: actions/setup-dotnet@v4
- run: dotnet nuget push targets/dotnet/*.nupkg

publish-maven:
if: startsWith(github.ref, 'refs/tags/v')
needs: package
steps:
- uses: actions/setup-java@v4
- run: mvn deploy -f targets/java/pom.xml

publish-go:
if: startsWith(github.ref, 'refs/tags/v')
needs: package
steps:
- run: |
cd targets/go
git tag && git push # go modules published via git tags
# Requires sparkgeo/prescient-sdk-go repo to exist on GitHub
### Phase 7 — CI/CD Pipeline ✅

Two GitHub Actions workflows:

**`.github/workflows/ci.yaml`** — extended with `typescript-sdk` job (build + test + package all 5 language targets on every push/PR to `jsii-migration`).

**`.github/workflows/publish-sdk.yaml`** — NEW. Triggers on `v*.*.*` tags. Five publish jobs, all `needs: build`:

| Job | Tool | Key details |
|---|---|---|
| `publish-npm` | `npm publish targets/js/*.tgz` | Pre-built tgz from jsii-pacmak; no source checkout needed |
| `publish-pypi` | `twine upload targets/python/*` | Uploads wheel + sdist (`*` not `*.whl`) |
| `publish-nuget` | `dotnet nuget push targets/dotnet/*.nupkg` | API key via `env:` not interpolated into command |
| `publish-maven` | `publib-maven` | Handles jsii local-repo layout, injects `<distributionManagement>`, GPG-signs, promotes OSSRH staging |
| `publish-go` | `rsync` + `git tag prescientsdk/vX.Y.Z` | Syncs to `sparkgeo/prescient-sdk-go` repo; guards against duplicate remote tags for idempotent re-runs |

Security hardening applied: all actions SHA-pinned, `permissions: {}` at top level with per-job minimums, `NUGET_API_KEY` via env var block, per-registry GitHub Actions environments with protection rules.

> **Infrastructure prerequisites (not yet done):**
> - Create `sparkgeo/prescient-sdk-go` GitHub repo
> - Configure GitHub Actions environments: `npm-production`, `pypi-jsii-production`, `nuget-production`, `maven-production`, `go-production`
> - Add secrets: `NPM_TOKEN`, `PYPI_JSII_API_TOKEN`, `NUGET_API_KEY`, `MAVEN_USERNAME`, `MAVEN_PASSWORD`, `MAVEN_GPG_PRIVATE_KEY`, `MAVEN_GPG_PASSPHRASE`, `MAVEN_STAGING_PROFILE_ID`, `GO_DEPLOY_TOKEN`
> - Register package namespaces: npm (`prescient-sdk`), PyPI (`prescient-sdk-sparkgeo`), NuGet (`Sparkgeo.PrescientSdk`), OSSRH (`com.sparkgeo`)

### Phase 8 — Multi-language Configuration Experience

**Problem:** The original Python SDK automatically loaded `config.env` from the CWD via `pydantic_settings`. The jsii TypeScript `Settings` class reads only from `process.env` — users of the jsii packages (Python/Java/.NET/Go) have no file-based config loading unless we add it in the TypeScript layer (which all languages funnel through).

**Solution:** Add `envFile?: string` to `PrescientClientOptions`. When provided, Settings parses the file as `KEY=VALUE` pairs (pure TypeScript, no extra dependency) and applies values at lowest priority:

```
explicit options > process.env > envFile values > built-in defaults
```

> **Infrastructure prerequisite:** Create `sparkgeo/prescient-sdk-go` GitHub repo for Go module publishing before running publish-go.
`PRESCIENT_GOOGLE_CLIENT_SECRET` from the file flows into `_googleClientSecret` only — same as the env-var path. Never exposed in any public struct.

**Language usage after this change:**

| Language | Usage |
|---|---|
| TypeScript | `new PrescientClient({ envFile: 'config.env' })` |
| Python (module: `prescient_sdk`) | `PrescientClient(env_file="config.env")` |
| .NET | `new PrescientClient(new PrescientClientOptions { EnvFile = "config.env" })` |
| Java | `new PrescientClient(PrescientClientOptions.builder().envFile("config.env").build())` |
| Go | `prescientsdk.NewPrescientClient(&prescientsdk.PrescientClientOptions{EnvFile: jsii.String("config.env")})` |

**Files changed:**
- `src/types.ts` — add `readonly envFile?: string` to `PrescientClientOptions`
- `src/settings.ts` — `parseEnvFile()` private static + constructor overlay
- `src/__tests__/settings.test.ts` — new test cases (file loading, override order, secret from file, missing file error)

---

Expand All @@ -372,8 +361,9 @@ jobs:
| 5 — Upload | 1.0 | ✅ Done |
| 5.5 — Smoke Test Infrastructure | 1.0 | ✅ Done |
| 6 — Tests | 1.5 | ✅ Done |
| 7 — CI/CD | 1.0 | Not started |
| **Total** | **~10 days** | |
| 7 — CI/CD | 1.0 | ✅ Done (PR #50 open, infrastructure prerequisites pending) |
| 8 — Multi-language Config Experience | 0.5 | Not started |
| **Total** | **~10.5 days** | |

---

Expand Down
58 changes: 58 additions & 0 deletions prescient-sdk-ts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,64 @@ Polyglot SDK for integrating with Prescient services. Generated from a single Ty
| Java | Maven Central |
| Go | GitHub |

## Configuration

Copy `config.env` from the repo root and fill in your credentials. The SDK reads it at lowest priority — environment variables always win.

```env
# config.env — see https://sparkgeo.github.io/prescient-sdk/config.html
PRESCIENT_ENDPOINT_URL=https://sparkgeo.prescient.earth
PRESCIENT_AUTH_URL=https://sparkgeo.prescient.earth/oauth2/auth
PRESCIENT_CLIENT_ID=<your-client-id>
PRESCIENT_AUTH_PROVIDER=microsoft # or google
PRESCIENT_TENANT_ID=<your-tenant-id> # required for microsoft
# PRESCIENT_GOOGLE_CLIENT_SECRET=... # required for google (env var only)
```

Three ways to configure the client — all equivalent at runtime:

### File-based (recommended for local development)

| Language | Code |
| --- | --- |
| TypeScript | `new PrescientClient({ envFile: 'config.env' })` |
| Python (jsii) | `PrescientClient(env_file="config.env")` |
| C# / .NET | `new PrescientClient(new PrescientClientOptions { EnvFile = "config.env" })` |
| Java | `new PrescientClient(PrescientClientOptions.builder().envFile("config.env").build())` |
| Go | `prescientsdk.NewPrescientClient(&prescientsdk.PrescientClientOptions{EnvFile: jsii.String("config.env")})` |

All languages funnel through the Node.js jsii runtime, so the file is always read in the same place — no per-language dotenv library needed.

### Environment variables (recommended for CI / containers)

Set `PRESCIENT_*` variables in the process environment, then call the default constructor:

```typescript
// TypeScript — same default-constructor pattern in all languages
const client = new PrescientClient();
```

### Explicit options (useful for testing / multiple clients)

```typescript
const client = new PrescientClient({
endpointUrl: 'https://sparkgeo.prescient.earth',
clientId: 'my-client-id',
authUrl: 'https://sparkgeo.prescient.earth/oauth2/auth',
tenantId: 'my-tenant-id',
});
```

### Priority order

`explicit options` > `environment variables` > `envFile` > built-in defaults.

### Security note — Google client secret

`PRESCIENT_GOOGLE_CLIENT_SECRET` is intentionally absent from `PrescientClientOptions`. It must be supplied via environment variable or `envFile` — never as a constructor argument — to prevent it from appearing in jsii IPC logs when `JSII_DEBUG=1` is set.

---

## Development

See [JSII_MIGRATION_PLAN.md](JSII_MIGRATION_PLAN.md) for the full migration plan.
Expand Down
2 changes: 1 addition & 1 deletion prescient-sdk-ts/justfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
compose := "docker compose -f " + justfile_directory() + "/../smoke-tests/docker-compose.yml"
compose := "docker compose -f " + justfile_directory() + "/smoke-tests/docker-compose.yml"

default: build

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Mount layout mirrors the repo tree under /workspace/ so that relative paths
# already embedded in go.mod (../../prescient-sdk-ts/...), nuget.config, and
# already embedded in go.mod (../../...), nuget.config, and
# pom.xml all resolve without modification.
#
# All non-JS language runtimes are jsii-based and spawn a `node` subprocess,
Expand All @@ -16,8 +16,9 @@ services:
working_dir: /workspace/smoke-tests/js
volumes:
- ./js:/workspace/smoke-tests/js:ro
- ../prescient-sdk-ts/dist:/workspace/prescient-sdk-ts/dist:ro
- ../prescient-sdk-ts/node_modules:/workspace/prescient-sdk-ts/node_modules:ro
- ../../local.env:/workspace/smoke-tests/config.env:ro
- ../dist:/workspace/prescient-sdk-ts/dist:ro
- ../node_modules:/workspace/prescient-sdk-ts/node_modules:ro
command: node smoke.js

python:
Expand All @@ -28,7 +29,8 @@ services:
working_dir: /workspace/smoke-tests/python
volumes:
- ./python:/workspace/smoke-tests/python:ro
- ../prescient-sdk-ts/targets/python:/workspace/prescient-sdk-ts/targets/python:ro
- ../../local.env:/workspace/smoke-tests/config.env:ro
- ../targets/python:/workspace/prescient-sdk-ts/targets/python:ro
- pip-cache:/root/.cache/pip
command:
- sh
Expand All @@ -47,7 +49,8 @@ services:
working_dir: /workspace/smoke-tests/go
volumes:
- ./go:/workspace/smoke-tests/go:ro
- ../prescient-sdk-ts/targets/go:/workspace/prescient-sdk-ts/targets/go:ro
- ../../local.env:/workspace/smoke-tests/config.env:ro
- ../targets/go:/workspace/prescient-sdk-ts/targets/go:ro
- go-mod-cache:/root/go/pkg/mod
- go-build-cache:/root/.cache/go-build
environment:
Expand All @@ -62,7 +65,8 @@ services:
working_dir: /workspace/smoke-tests/dotnet
volumes:
- ./dotnet:/workspace/smoke-tests/dotnet:ro
- ../prescient-sdk-ts/targets/dotnet:/workspace/prescient-sdk-ts/targets/dotnet:ro
- ../../local.env:/workspace/smoke-tests/config.env:ro
- ../targets/dotnet:/workspace/prescient-sdk-ts/targets/dotnet:ro
- nuget-cache:/root/.nuget
- dotnet-obj:/workspace/smoke-tests/dotnet/obj
- dotnet-bin:/workspace/smoke-tests/dotnet/bin
Expand All @@ -76,7 +80,8 @@ services:
working_dir: /workspace/smoke-tests/java
volumes:
- ./java:/workspace/smoke-tests/java:ro
- ../prescient-sdk-ts/targets/java:/workspace/prescient-sdk-ts/targets/java:ro
- ../../local.env:/workspace/smoke-tests/config.env:ro
- ../targets/java:/workspace/prescient-sdk-ts/targets/java:ro
- maven-cache:/root/.m2
- maven-target:/workspace/smoke-tests/java/target
command: mvn --no-transfer-progress compile exec:java
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,7 @@

var client = new PrescientClient(new PrescientClientOptions
{
EndpointUrl = "https://api.example.com",
ClientId = "test-client-id",
AuthUrl = "https://login.microsoftonline.com",
TenantId = "test-tenant-id",
EnvFile = "/workspace/smoke-tests/config.env",
});

Console.WriteLine($"EndpointUrl : {client.Settings.EndpointUrl}");
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,7 @@ func main() {
defer jsii.Close()

client := prescientsdk.NewPrescientClient(&prescientsdk.PrescientClientOptions{
EndpointUrl: jsii.String("https://api.example.com"),
ClientId: jsii.String("test-client-id"),
AuthUrl: jsii.String("https://login.microsoftonline.com"),
TenantId: jsii.String("test-tenant-id"),
EnvFile: jsii.String("/workspace/smoke-tests/config.env"),
})

fmt.Println("endpointUrl :", *client.Settings().EndpointUrl())
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,7 @@ public class SmokeTest {
public static void main(String[] args) {
PrescientClient client = new PrescientClient(
PrescientClientOptions.builder()
.endpointUrl("https://api.example.com")
.clientId("test-client-id")
.authUrl("https://login.microsoftonline.com")
.tenantId("test-tenant-id")
.envFile("/workspace/smoke-tests/config.env")
.build()
);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,7 @@
// which resolves the full dep tree from the registry.
const { PrescientClient } = require('../../prescient-sdk-ts/dist/index');

const client = new PrescientClient({
endpointUrl: 'https://api.example.com',
clientId: 'test-client-id',
authUrl: 'https://login.microsoftonline.com',
tenantId: 'test-tenant-id',
});
const client = new PrescientClient({ envFile: '/workspace/smoke-tests/config.env' });

console.log('endpointUrl :', client.settings.endpointUrl);
console.log('stacCatalogUrl :', client.stacCatalogUrl);
Expand Down
7 changes: 7 additions & 0 deletions prescient-sdk-ts/smoke-tests/python/smoke.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
from prescient_sdk import PrescientClient

client = PrescientClient(env_file="/workspace/smoke-tests/config.env")

print("endpoint_url :", client.settings.endpoint_url)
print("stac_catalog_url:", client.stac_catalog_url)
print("✓ Python smoke test passed")
Loading
Loading