From 07c7efec5e14b7993d55bf1e75babb97fb3b7865 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 13 May 2026 14:24:18 -0600 Subject: [PATCH 01/76] added first draft for targetsource implementation docs --- docs/content/docs/user-guide/targetsource.md | 152 ++++++++----------- 1 file changed, 62 insertions(+), 90 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource.md b/docs/content/docs/user-guide/targetsource.md index d1fee0c4..a3a3eb15 100644 --- a/docs/content/docs/user-guide/targetsource.md +++ b/docs/content/docs/user-guide/targetsource.md @@ -10,15 +10,11 @@ The `TargetSource` resource enables dynamic discovery of network devices from ex ## Discovery Sources -TargetSource supports multiple discovery backends: +TargetSource supports the following discovery providers: | Source | Description | |--------|-------------| | `http` | Fetch targets from an HTTP endpoint | -| `consul` | Discover targets from Consul service registry | -| `configMap` | Read targets from a Kubernetes ConfigMap | -| `podSelector` | Create targets from Kubernetes Pods | -| `serviceSelector` | Create targets from Kubernetes Services | ## HTTP Discovery @@ -30,92 +26,55 @@ kind: TargetSource metadata: name: http-discovery spec: - http: - url: http://inventory-service:8080/targets - labels: + provider: + http: + url: http://inventory-service:8080/targets + targetProfile: default + targetLabels: source: inventory ``` -The HTTP endpoint should return a JSON array of target objects. - -## Consul Discovery - -Discover targets from Consul service registry: - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: consul-discovery -spec: - consul: - url: http://consul:8500 - labels: - source: consul - datacenter: dc1 +The HTTP endpoint should return a JSON array of target objects. The following is an example for a valid JSON array: + +```json +[ + { + "address": "spine1:57400", + "name": "spine1", + "labels": { + "role": "spine" + } + }, + { + "address": "leaf1:57400", + "name": "leaf1", + "labels": { + "role": "leaf" + } + }, + { + "address": "leaf2:57400", + "name": "leaf2", + "labels": { + "role": "leaf" + } + } +] ``` -## ConfigMap Discovery +## TargetProfile Inheritance -Read targets from a Kubernetes ConfigMap: +Within the `TargetSource`, the default `TargetProfile` for all targets can be defined using `targetProfile`. Each target discovered inherits the defined value. -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: configmap-targets -spec: - configMap: network-devices - labels: - source: configmap -``` - -The ConfigMap should contain target definitions in a structured format. - -## Kubernetes Pod Discovery - -Create targets from Kubernetes Pods matching a label selector: - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: pod-discovery -spec: - podSelector: - matchLabels: - app: network-simulator - gnmi: enabled - labels: - source: kubernetes - type: simulator -``` - -This is useful for: -- Containerized network simulators -- Virtual network functions (VNFs) -- Development/testing environments - -## Kubernetes Service Discovery +## Label Inheritance -Create targets from Kubernetes Services matching a label selector: +Each discovered target has a label defined to identify the owning `TargetSource`: +- `operator.gnmic.dev/targetsource: datacenter-a` -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: service-discovery -spec: - serviceSelector: - matchLabels: - protocol: gnmi - labels: - source: kubernetes -``` -## Label Inheritance +This label is needed to identify all targets owned by this resource and determine which devices get applied or removed. This label takes precedence over all other labels on the target. -Labels defined in the `TargetSource.spec.labels` field are applied to all discovered targets: +Labels defined in the `TargetSource.spec.targetLabels` field are applied to all discovered targets: ```yaml apiVersion: operator.gnmic.dev/v1alpha1 @@ -123,21 +82,33 @@ kind: TargetSource metadata: name: datacenter-a spec: - consul: - url: http://consul-dc-a:8500 - labels: + provider: + http: + url: http://datacenter-a:8080/targets + targetLabels: datacenter: dc-a environment: production - source: consul ``` All targets discovered from this source will have: - `datacenter: dc-a` - `environment: production` -- `source: consul` This enables using label selectors in Pipelines to select targets by their discovery source. +## Labels from Source of Truth + +Targets can also have labels defined by the external system. These get directly applied to the target with their original key/value pair. + +The gNMIc Operator has a reserved namespace for labels which alter the behavior of the target: +- `gnmic_operator_` + +Following are all supported operator-specific labels: + +| Label | Description | +|--------|-------------| +| `gnmic_operator_target_profile` | Overwrite the `TargetProfile` which is defined in the `TargetSource` | + ## Status The TargetSource status shows discovery state: @@ -155,7 +126,7 @@ status: | `targetsCount` | Number of targets discovered | | `lastSync` | Timestamp of last successful sync | -## Example: Multi-Source Discovery + ## Lifecycle @@ -219,12 +190,13 @@ spec: When a TargetSource discovers a new device: 1. A new `Target` resource is created -2. Labels from `spec.labels` are applied -3. Owner reference is set to the TargetSource +2. The `Profile` gets specified from `spec.targetProfile` +3. Labels from `spec.targetLabels` are applied +4. Owner reference is set to the TargetSource ### Target Updates -When a discovered device's properties change: +Discovered devices get reapplied each time the target gets discovered, overwriting any changes manually made: 1. The corresponding `Target` is updated 2. Clusters using that target are reconciled From 8bab96ce2719903fc42deed051220557ce764bd4 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 13 May 2026 15:46:47 -0600 Subject: [PATCH 02/76] aligning doc with existing pages --- docs/content/docs/user-guide/targetsource.md | 143 +++++++++++++------ 1 file changed, 96 insertions(+), 47 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource.md b/docs/content/docs/user-guide/targetsource.md index a3a3eb15..5895a796 100644 --- a/docs/content/docs/user-guide/targetsource.md +++ b/docs/content/docs/user-guide/targetsource.md @@ -8,53 +8,86 @@ description: > The `TargetSource` resource enables dynamic discovery of network devices from external sources. The operator automatically creates, updates, and deletes `Target` resources based on discovered devices. -## Discovery Sources - -TargetSource supports the following discovery providers: - -| Source | Description | -|--------|-------------| -| `http` | Fetch targets from an HTTP endpoint | - -## HTTP Discovery - -Discover targets from an HTTP endpoint that returns a JSON list of targets: +## Basic Configuration ```yaml apiVersion: operator.gnmic.dev/v1alpha1 kind: TargetSource metadata: - name: http-discovery + name: targetsource-1 spec: provider: - http: - url: http://inventory-service:8080/targets + # see Discovery Providers section targetProfile: default targetLabels: source: inventory ``` -The HTTP endpoint should return a JSON array of target objects. The following is an example for a valid JSON array: +## Spec Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `provider` | object | Yes | Provider-specific discovery configuration. Exactly one provider must be configured | +| `targetProfile` | string | Yes | Reference to `TargetProfile` applied to all targets | +| `targetLabels` | map[string]string | No | Labels added to all discovered targets | + + +## Discovery Providers + +`TargetSource` supports the following discovery providers: + +| Provider | Description | +|----------|-------------| +| `http` | Discover targets from an HTTP JSON endpoint | + +### HTTP Provider + +The HTTP provider discovers targets from an HTTP endpoint returning a JSON array of target definitions. + +```yaml +spec: + provider: + http: + url: http://inventory-service:8080/targets +``` + +#### HTTP Spec Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `url` | string | Yes | URL pointing to the inventory server | + +#### Response Format + +The endpoint must return a JSON array of objects with the following structure: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | string | Yes | Name of the generated `Target` resource | +| `address` | string | Yes | Device address in `host:port` format | +| `labels` | map[string]string | No | Labels added to the generated `Target` resource | + +Example response: ```json [ { - "address": "spine1:57400", "name": "spine1", + "address": "spine1:57400", "labels": { "role": "spine" } }, { - "address": "leaf1:57400", "name": "leaf1", + "address": "leaf1:57400", "labels": { "role": "leaf" } }, { - "address": "leaf2:57400", "name": "leaf2", + "address": "leaf2:57400", "labels": { "role": "leaf" } @@ -62,29 +95,32 @@ The HTTP endpoint should return a JSON array of target objects. The following is ] ``` -## TargetProfile Inheritance - -Within the `TargetSource`, the default `TargetProfile` for all targets can be defined using `targetProfile`. Each target discovered inherits the defined value. - ## Label Inheritance -Each discovered target has a label defined to identify the owning `TargetSource`: -- `operator.gnmic.dev/targetsource: datacenter-a` +Each generated `Target` receives an ownership label identifying the originating `TargetSource`: +```yaml +operator.gnmic.dev/targetsource: targetsource-1 +``` + +This label is automatically managed by the operator and is used to: +- Identify targets owned by a specific `TargetSource` +- Determine which targets should be updated or deleted during reconciliation +The `operator.gnmic.dev/targetsource` label is reserved and always takes precedence over any provider-supplied labels. -This label is needed to identify all targets owned by this resource and determine which devices get applied or removed. This label takes precedence over all other labels on the target. +### TargetSource Labels -Labels defined in the `TargetSource.spec.targetLabels` field are applied to all discovered targets: +Additional labels can be applied to all generated targets using `spec.targetLabels`: ```yaml apiVersion: operator.gnmic.dev/v1alpha1 kind: TargetSource metadata: - name: datacenter-a + name: targetsource-1 spec: provider: http: - url: http://datacenter-a:8080/targets + url: http://targetsource-1:8080/targets targetLabels: datacenter: dc-a environment: production @@ -94,20 +130,27 @@ All targets discovered from this source will have: - `datacenter: dc-a` - `environment: production` -This enables using label selectors in Pipelines to select targets by their discovery source. +This enables Pipelines to select targets using label selectors. -## Labels from Source of Truth +### Labels from Discovery Providers -Targets can also have labels defined by the external system. These get directly applied to the target with their original key/value pair. +Discovery providers may return additional labels for each target. These labels are applied directly to the generated `Target` resource. -The gNMIc Operator has a reserved namespace for labels which alter the behavior of the target: -- `gnmic_operator_` +The `gnmic_operator_` label prefix is reserved for operator-specific behavior. Labels using this prefix are interpreted by the operator and are not applied directly to the generated `Target` resource. -Following are all supported operator-specific labels: +Supported operator labels: | Label | Description | |--------|-------------| -| `gnmic_operator_target_profile` | Overwrite the `TargetProfile` which is defined in the `TargetSource` | +| `gnmic_operator_target_profile` | Overrides the `TargetProfile` configured in the `TargetSource` | + +### Label Precedence + +If the same label key is defined in multiple places, labels are applied in the following order (highest precedence first): + +1. `TargetSource` ownership label (`operator.gnmic.dev/targetsource`) +2. Labels from `TargetSource.spec.targetLabels` +3. Labels returned by the discovery provider ## Status @@ -188,27 +231,33 @@ spec: ### Target Creation -When a TargetSource discovers a new device: +When a `TargetSource` discovers a new device: + 1. A new `Target` resource is created -2. The `Profile` gets specified from `spec.targetProfile` +2. The `TargetProfile` referenced in `spec.targetProfile` is assigned 3. Labels from `spec.targetLabels` are applied -4. Owner reference is set to the TargetSource +4. The `TargetSource` is set as the owner reference ### Target Updates -Discovered devices get reapplied each time the target gets discovered, overwriting any changes manually made: -1. The corresponding `Target` is updated -2. Clusters using that target are reconciled +On each discovery cycle, existing `Target` resources are reconciled with the latest discovered state: + +1. The corresponding `Target` resource is updated and overwritten +2. Clusters consuming the target are reconciled automatically + +> Manual changes to `Target` resources managed by a `TargetSource` are overwritten on every reconciliation cycle. ### Target Deletion -When a device is no longer discovered: -1. The `Target` resource is deleted -2. Clusters stop collecting from that target +When a device is no longer returned by the discovery provider: + +1. The corresponding `Target` resource is deleted +2. Clusters automatically stop using the target ### TargetSource Deletion -When a TargetSource is deleted: -1. All Targets owned by it are deleted (via owner references) -2. Clusters are reconciled to remove those targets +When a `TargetSource` is deleted: + +1. All `Target` resources owned by it are deleted via owner references +2. Clusters are reconciled and remove the deleted targets From 2d1f5fb28a113511781e0c6623f676981b8a6db5 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 20 May 2026 19:31:17 -0600 Subject: [PATCH 03/76] updated targetSource guide based on new CRD --- docs/content/docs/user-guide/targetsource.md | 176 ++++++++++++++++++- 1 file changed, 170 insertions(+), 6 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource.md b/docs/content/docs/user-guide/targetsource.md index 5895a796..896b405b 100644 --- a/docs/content/docs/user-guide/targetsource.md +++ b/docs/content/docs/user-guide/targetsource.md @@ -18,6 +18,7 @@ metadata: spec: provider: # see Discovery Providers section + targetPort: 57400 targetProfile: default targetLabels: source: inventory @@ -28,6 +29,7 @@ spec: | Field | Type | Required | Description | |-------|------|----------|-------------| | `provider` | object | Yes | Provider-specific discovery configuration. Exactly one provider must be configured | +| `targetPort` | int32 | No | Default port used if the discovered target does not provide a port. | | `targetProfile` | string | Yes | Reference to `TargetProfile` applied to all targets | | `targetLabels` | map[string]string | No | Labels added to all discovered targets | @@ -49,6 +51,7 @@ spec: provider: http: url: http://inventory-service:8080/targets + ``` #### HTTP Spec Fields @@ -56,16 +59,173 @@ spec: | Field | Type | Required | Description | |-------|------|----------|-------------| | `url` | string | Yes | URL pointing to the inventory server | +| `acceptPush` | bool | No | Enable webhook-based target updates. Defaults to `false`. | +| `authorization` | object | No | Credentials used to access the HTTP endpoint. See _Authorization_ section. | +| `pollInterval` | metav1.Duration | No | Polling interval used to fetch targets from the endpoint. Defaults to `30s`. | +| `timeout` | metav1.Duration | No | Timeout for HTTP requests. Defaults to `10s`. | +| `tls` | object | No | Client TLS configuration for HTTPS endpoints. See _TLS_ section. | +| `pagination` | object | No | Pagination configuration for parsing responses from the HTTP endpoint. See _Pagination_ section. | +| `responseMapping` | object | No | JSON path mapping definitions. See _Response Mapping_ section. | + +##### Authorization + +The HTTP provider supports authenticated requests to the inventory endpoint. + +Exactly one authorization method can be configured. + +###### Basic Authentication + +Credentials can either be defined inline or referenced from a Secret. + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/targets + authorization: + basic: + username: admin + password: secret +``` + +Using a Secret reference: + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/targets + authorization: + basic: + credentialsSecretRef: + name: inventory-credentials + key: username +``` + +###### Token Authentication + +Static token authentication can be configured using either an inline token or a Secret reference. + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/targets + authorization: + token: + scheme: Bearer + token: eyJhbGciOi... +``` + +Using a Secret reference: + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/targets + authorization: + token: + scheme: Bearer + tokenSecretRef: + name: inventory-token + key: token +``` + +##### TLS + +TLS settings can be configured for HTTPS endpoints. + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/targets + tls: + insecureSkipVerify: false +``` + +###### TLS Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `insecureSkipVerify` | bool | No | Skip verification of the server certificate. Defaults to `false`. | +| `caBundle` | []byte | No | Base64-encoded PEM CA bundle used to validate the server certificate. | +| `caBundleSecretRef` | object | No | Reference to a Secret containing a PEM CA bundle. | + +`caBundle` and `caBundleSecretRef` are mutually exclusive. + +##### Pagination + +Pagination can be configured for APIs returning paginated responses. + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/devices + pagination: + itemsField: results + nextField: next +``` + +###### Pagination Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `itemsField` | string | No | Top-level JSON field containing the list of target objects. | +| `nextField` | string | No | Top-level JSON field containing the next page reference or pagination token. | + +The `nextField` value may either contain: +- A full URL for the next request +- A pagination token appended as a query parameter to the original URL + +##### Response Mapping + +By default, the HTTP response must follow the structure defined in the _Response Format_ section. + +`responseMapping` allows extracting target fields from arbitrary JSON structures using JSONPath expressions. + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/devices + responseMapping: + name: "$.hostname" + ip: "$.management.ip" + port: "$.gnmi.port" + targetProfile: "$.profile" + labels: + role: "$.metadata.role" + site: "$.metadata.site" +``` + +###### Response Mapping Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | string | Yes | JSONPath expression extracting the target name. | +| `ip` | string | Yes | JSONPath expression extracting the target IP address or hostname. | +| `port` | string | No | JSONPath expression extracting the gNMI port. | +| `targetProfile` | string | No | JSONPath expression extracting the `TargetProfile`. | +| `labels` | map[string]string | No | JSONPath expressions extracting target labels. | + +Labels extracted through `responseMapping.labels` are merged with labels from `spec.targetLabels`. + +If the same label key exists in both locations, labels extracted through `responseMapping.labels` take precedence. #### Response Format -The endpoint must return a JSON array of objects with the following structure: +If `responseMapping` is not configured, the endpoint must return a JSON array of objects with the following structure: | Field | Type | Required | Description | |-------|------|----------|-------------| | `name` | string | Yes | Name of the generated `Target` resource | -| `address` | string | Yes | Device address in `host:port` format | +| `address` | string | Yes | Device address (FQDN or IP address) | +| `port` | int32 | No | Port used for gNMI connections. If omitted, `spec.targetPort` is used. | | `labels` | map[string]string | No | Labels added to the generated `Target` resource | +| `targetProfile` | string | No | Reference to a `TargetProfile`. If omitted, `spec.targetProfile` is used. | Example response: @@ -73,21 +233,25 @@ Example response: [ { "name": "spine1", - "address": "spine1:57400", + "address": "spine1", + "port": 57400, "labels": { "role": "spine" - } + }, + "targetProfile": "spine-profile" }, { "name": "leaf1", - "address": "leaf1:57400", + "address": "leaf1", + "port": 57400, "labels": { "role": "leaf" } }, { "name": "leaf2", - "address": "leaf2:57400", + "address": "leaf2", + "port": 57400, "labels": { "role": "leaf" } From abbbee4d33aa4dc5d7c7a5d6098c7bdaafb178fc Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 20 May 2026 20:38:47 -0600 Subject: [PATCH 04/76] renamed response field ip to address --- docs/content/docs/user-guide/targetsource.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/docs/user-guide/targetsource.md b/docs/content/docs/user-guide/targetsource.md index 896b405b..9a699ffa 100644 --- a/docs/content/docs/user-guide/targetsource.md +++ b/docs/content/docs/user-guide/targetsource.md @@ -193,7 +193,7 @@ spec: url: https://inventory.example.com/devices responseMapping: name: "$.hostname" - ip: "$.management.ip" + address: "$.management.ip" port: "$.gnmi.port" targetProfile: "$.profile" labels: From 21496fc079150a2d74784f65c72533bf70b751b9 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 20 May 2026 20:48:43 -0600 Subject: [PATCH 05/76] moved http into a subfile of TargetSource --- .../docs/user-guide/targetsource/_index.md | 211 ++++++++++++++++ .../{targetsource.md => targetsource/http.md} | 233 ++---------------- 2 files changed, 225 insertions(+), 219 deletions(-) create mode 100644 docs/content/docs/user-guide/targetsource/_index.md rename docs/content/docs/user-guide/{targetsource.md => targetsource/http.md} (50%) diff --git a/docs/content/docs/user-guide/targetsource/_index.md b/docs/content/docs/user-guide/targetsource/_index.md new file mode 100644 index 00000000..16861e74 --- /dev/null +++ b/docs/content/docs/user-guide/targetsource/_index.md @@ -0,0 +1,211 @@ +--- +title: "TargetSource" +linkTitle: "TargetSource" +weight: 4 +description: > + Dynamic target discovery from external sources +--- + +The `TargetSource` resource enables dynamic discovery of network devices from external sources. The operator automatically creates, updates, and deletes `Target` resources based on discovered devices. + +## Basic Configuration + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: targetsource-1 +spec: + provider: + # see Discovery Providers section + targetPort: 57400 + targetProfile: default + targetLabels: + source: inventory +``` + +## Spec Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `provider` | object | Yes | Provider-specific discovery configuration. Exactly one provider must be configured | +| `targetPort` | int32 | No | Default port used if the discovered target does not provide a port. | +| `targetProfile` | string | Yes | Reference to `TargetProfile` applied to all targets | +| `targetLabels` | map[string]string | No | Labels added to all discovered targets | + + +## Discovery Providers + +`TargetSource` supports the following discovery providers: + +| Provider | Description | +|----------|-------------| +| `http` | Discover targets from an HTTP JSON endpoint. [Configuration]({{< relref "http.md" >}}) | + + +## Label Inheritance + +Each generated `Target` receives an ownership label identifying the originating `TargetSource`: +```yaml +operator.gnmic.dev/targetsource: targetsource-1 +``` + +This label is automatically managed by the operator and is used to: +- Identify targets owned by a specific `TargetSource` +- Determine which targets should be updated or deleted during reconciliation + +The `operator.gnmic.dev/targetsource` label is reserved and always takes precedence over any provider-supplied labels. + +### TargetSource Labels + +Additional labels can be applied to all generated targets using `spec.targetLabels`: + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: targetsource-1 +spec: + provider: + http: + url: http://targetsource-1:8080/targets + targetLabels: + datacenter: dc-a + environment: production +``` + +All targets discovered from this source will have: +- `datacenter: dc-a` +- `environment: production` + +This enables Pipelines to select targets using label selectors. + +### Labels from Discovery Providers + +Discovery providers may return additional labels for each target. These labels are applied directly to the generated `Target` resource. + +The `gnmic_operator_` label prefix is reserved for operator-specific behavior. Labels using this prefix are interpreted by the operator and are not applied directly to the generated `Target` resource. + +Supported operator labels: + +| Label | Description | +|--------|-------------| +| `gnmic_operator_target_profile` | Overrides the `TargetProfile` configured in the `TargetSource` | + +### Label Precedence + +If the same label key is defined in multiple places, labels are applied in the following order (highest precedence first): + +1. `TargetSource` ownership label (`operator.gnmic.dev/targetsource`) +2. Labels from `TargetSource.spec.targetLabels` +3. Labels returned by the discovery provider + +## Status + +The TargetSource status shows discovery state: + +```yaml +status: + status: Synced + targetsCount: 42 + lastSync: "2024-01-15T10:30:00Z" +``` + +| Field | Description | +|-------|-------------| +| `status` | Current sync status (Synced, Error, Pending) | +| `targetsCount` | Number of targets discovered | +| `lastSync` | Timestamp of last successful sync | + + + +## Lifecycle + +### Target Creation + +When a `TargetSource` discovers a new device: + +1. A new `Target` resource is created +2. The `TargetProfile` referenced in `spec.targetProfile` is assigned +3. Labels from `spec.targetLabels` are applied +4. The `TargetSource` is set as the owner reference + +### Target Updates + +On each discovery cycle, existing `Target` resources are reconciled with the latest discovered state: + +1. The corresponding `Target` resource is updated and overwritten +2. Clusters consuming the target are reconciled automatically + +> Manual changes to `Target` resources managed by a `TargetSource` are overwritten on every reconciliation cycle. + +### Target Deletion + +When a device is no longer returned by the discovery provider: + +1. The corresponding `Target` resource is deleted +2. Clusters automatically stop using the target + +### TargetSource Deletion + +When a `TargetSource` is deleted: + +1. All `Target` resources owned by it are deleted via owner references +2. Clusters are reconciled and remove the deleted targets + diff --git a/docs/content/docs/user-guide/targetsource.md b/docs/content/docs/user-guide/targetsource/http.md similarity index 50% rename from docs/content/docs/user-guide/targetsource.md rename to docs/content/docs/user-guide/targetsource/http.md index 9a699ffa..9fea5d33 100644 --- a/docs/content/docs/user-guide/targetsource.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -1,49 +1,11 @@ --- -title: "TargetSource" -linkTitle: "TargetSource" +title: "HTTP Provider" +linkTitle: "HTTP" weight: 4 description: > - Dynamic target discovery from external sources + HTTP TargetSource Discovery Provider --- -The `TargetSource` resource enables dynamic discovery of network devices from external sources. The operator automatically creates, updates, and deletes `Target` resources based on discovered devices. - -## Basic Configuration - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: targetsource-1 -spec: - provider: - # see Discovery Providers section - targetPort: 57400 - targetProfile: default - targetLabels: - source: inventory -``` - -## Spec Fields - -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| `provider` | object | Yes | Provider-specific discovery configuration. Exactly one provider must be configured | -| `targetPort` | int32 | No | Default port used if the discovered target does not provide a port. | -| `targetProfile` | string | Yes | Reference to `TargetProfile` applied to all targets | -| `targetLabels` | map[string]string | No | Labels added to all discovered targets | - - -## Discovery Providers - -`TargetSource` supports the following discovery providers: - -| Provider | Description | -|----------|-------------| -| `http` | Discover targets from an HTTP JSON endpoint | - -### HTTP Provider - The HTTP provider discovers targets from an HTTP endpoint returning a JSON array of target definitions. ```yaml @@ -54,7 +16,7 @@ spec: ``` -#### HTTP Spec Fields +## HTTP Spec Fields | Field | Type | Required | Description | |-------|------|----------|-------------| @@ -67,13 +29,13 @@ spec: | `pagination` | object | No | Pagination configuration for parsing responses from the HTTP endpoint. See _Pagination_ section. | | `responseMapping` | object | No | JSON path mapping definitions. See _Response Mapping_ section. | -##### Authorization +### Authorization The HTTP provider supports authenticated requests to the inventory endpoint. Exactly one authorization method can be configured. -###### Basic Authentication +#### Basic Authentication Credentials can either be defined inline or referenced from a Secret. @@ -102,7 +64,7 @@ spec: key: username ``` -###### Token Authentication +#### Token Authentication Static token authentication can be configured using either an inline token or a Secret reference. @@ -132,7 +94,7 @@ spec: key: token ``` -##### TLS +### TLS TLS settings can be configured for HTTPS endpoints. @@ -145,7 +107,7 @@ spec: insecureSkipVerify: false ``` -###### TLS Fields +#### TLS Fields | Field | Type | Required | Description | |-------|------|----------|-------------| @@ -155,7 +117,7 @@ spec: `caBundle` and `caBundleSecretRef` are mutually exclusive. -##### Pagination +### Pagination Pagination can be configured for APIs returning paginated responses. @@ -169,7 +131,7 @@ spec: nextField: next ``` -###### Pagination Fields +#### Pagination Fields | Field | Type | Required | Description | |-------|------|----------|-------------| @@ -180,7 +142,7 @@ The `nextField` value may either contain: - A full URL for the next request - A pagination token appended as a query parameter to the original URL -##### Response Mapping +### Response Mapping By default, the HTTP response must follow the structure defined in the _Response Format_ section. @@ -201,7 +163,7 @@ spec: site: "$.metadata.site" ``` -###### Response Mapping Fields +#### Response Mapping Fields | Field | Type | Required | Description | |-------|------|----------|-------------| @@ -215,7 +177,7 @@ Labels extracted through `responseMapping.labels` are merged with labels from `s If the same label key exists in both locations, labels extracted through `responseMapping.labels` take precedence. -#### Response Format +## Response Format If `responseMapping` is not configured, the endpoint must return a JSON array of objects with the following structure: @@ -258,170 +220,3 @@ Example response: } ] ``` - -## Label Inheritance - -Each generated `Target` receives an ownership label identifying the originating `TargetSource`: -```yaml -operator.gnmic.dev/targetsource: targetsource-1 -``` - -This label is automatically managed by the operator and is used to: -- Identify targets owned by a specific `TargetSource` -- Determine which targets should be updated or deleted during reconciliation - -The `operator.gnmic.dev/targetsource` label is reserved and always takes precedence over any provider-supplied labels. - -### TargetSource Labels - -Additional labels can be applied to all generated targets using `spec.targetLabels`: - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: targetsource-1 -spec: - provider: - http: - url: http://targetsource-1:8080/targets - targetLabels: - datacenter: dc-a - environment: production -``` - -All targets discovered from this source will have: -- `datacenter: dc-a` -- `environment: production` - -This enables Pipelines to select targets using label selectors. - -### Labels from Discovery Providers - -Discovery providers may return additional labels for each target. These labels are applied directly to the generated `Target` resource. - -The `gnmic_operator_` label prefix is reserved for operator-specific behavior. Labels using this prefix are interpreted by the operator and are not applied directly to the generated `Target` resource. - -Supported operator labels: - -| Label | Description | -|--------|-------------| -| `gnmic_operator_target_profile` | Overrides the `TargetProfile` configured in the `TargetSource` | - -### Label Precedence - -If the same label key is defined in multiple places, labels are applied in the following order (highest precedence first): - -1. `TargetSource` ownership label (`operator.gnmic.dev/targetsource`) -2. Labels from `TargetSource.spec.targetLabels` -3. Labels returned by the discovery provider - -## Status - -The TargetSource status shows discovery state: - -```yaml -status: - status: Synced - targetsCount: 42 - lastSync: "2024-01-15T10:30:00Z" -``` - -| Field | Description | -|-------|-------------| -| `status` | Current sync status (Synced, Error, Pending) | -| `targetsCount` | Number of targets discovered | -| `lastSync` | Timestamp of last successful sync | - - - -## Lifecycle - -### Target Creation - -When a `TargetSource` discovers a new device: - -1. A new `Target` resource is created -2. The `TargetProfile` referenced in `spec.targetProfile` is assigned -3. Labels from `spec.targetLabels` are applied -4. The `TargetSource` is set as the owner reference - -### Target Updates - -On each discovery cycle, existing `Target` resources are reconciled with the latest discovered state: - -1. The corresponding `Target` resource is updated and overwritten -2. Clusters consuming the target are reconciled automatically - -> Manual changes to `Target` resources managed by a `TargetSource` are overwritten on every reconciliation cycle. - -### Target Deletion - -When a device is no longer returned by the discovery provider: - -1. The corresponding `Target` resource is deleted -2. Clusters automatically stop using the target - -### TargetSource Deletion - -When a `TargetSource` is deleted: - -1. All `Target` resources owned by it are deleted via owner references -2. Clusters are reconciled and remove the deleted targets - From db090b3e6c8f5ff622e0677c5fe2fd0b6e211431 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 20 May 2026 21:00:38 -0600 Subject: [PATCH 06/76] restructured http chapter hierarchy --- .../docs/user-guide/targetsource/http.md | 88 ++++++++++--------- 1 file changed, 45 insertions(+), 43 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource/http.md b/docs/content/docs/user-guide/targetsource/http.md index 9fea5d33..b6ff8e4e 100644 --- a/docs/content/docs/user-guide/targetsource/http.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -13,7 +13,6 @@ spec: provider: http: url: http://inventory-service:8080/targets - ``` ## HTTP Spec Fields @@ -23,19 +22,19 @@ spec: | `url` | string | Yes | URL pointing to the inventory server | | `acceptPush` | bool | No | Enable webhook-based target updates. Defaults to `false`. | | `authorization` | object | No | Credentials used to access the HTTP endpoint. See _Authorization_ section. | -| `pollInterval` | metav1.Duration | No | Polling interval used to fetch targets from the endpoint. Defaults to `30s`. | -| `timeout` | metav1.Duration | No | Timeout for HTTP requests. Defaults to `10s`. | +| `pollInterval` | duration | No | Polling interval used to fetch targets from the endpoint. Defaults to `30s`. | +| `timeout` | duration | No | Timeout for HTTP requests. Defaults to `10s`. | | `tls` | object | No | Client TLS configuration for HTTPS endpoints. See _TLS_ section. | | `pagination` | object | No | Pagination configuration for parsing responses from the HTTP endpoint. See _Pagination_ section. | -| `responseMapping` | object | No | JSON path mapping definitions. See _Response Mapping_ section. | +| `responseMapping` | object | No | JSONPath mapping definitions. See _Response Processing_ section. | -### Authorization +## Authorization The HTTP provider supports authenticated requests to the inventory endpoint. Exactly one authorization method can be configured. -#### Basic Authentication +### Basic Authentication Credentials can either be defined inline or referenced from a Secret. @@ -64,7 +63,7 @@ spec: key: username ``` -#### Token Authentication +### Token Authentication Static token authentication can be configured using either an inline token or a Secret reference. @@ -94,7 +93,7 @@ spec: key: token ``` -### TLS +## TLS TLS settings can be configured for HTTPS endpoints. @@ -107,7 +106,7 @@ spec: insecureSkipVerify: false ``` -#### TLS Fields +### TLS Fields | Field | Type | Required | Description | |-------|------|----------|-------------| @@ -117,7 +116,7 @@ spec: `caBundle` and `caBundleSecretRef` are mutually exclusive. -### Pagination +## Pagination Pagination can be configured for APIs returning paginated responses. @@ -131,7 +130,7 @@ spec: nextField: next ``` -#### Pagination Fields +### Pagination Fields | Field | Type | Required | Description | |-------|------|----------|-------------| @@ -142,42 +141,16 @@ The `nextField` value may either contain: - A full URL for the next request - A pagination token appended as a query parameter to the original URL -### Response Mapping +## Response Processing -By default, the HTTP response must follow the structure defined in the _Response Format_ section. +The HTTP provider supports two methods for processing responses from the inventory endpoint: -`responseMapping` allows extracting target fields from arbitrary JSON structures using JSONPath expressions. +- **Default Response Format**: The endpoint returns a predefined JSON structure understood directly by the operator. +- **Response Mapping via JSONPath**: Arbitrary JSON structures can be mapped to target fields using JSONPath expressions. -```yaml -spec: - provider: - http: - url: https://inventory.example.com/devices - responseMapping: - name: "$.hostname" - address: "$.management.ip" - port: "$.gnmi.port" - targetProfile: "$.profile" - labels: - role: "$.metadata.role" - site: "$.metadata.site" -``` +If `responseMapping` is configured, the custom mappings are used. Otherwise, the default response format is expected. -#### Response Mapping Fields - -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| `name` | string | Yes | JSONPath expression extracting the target name. | -| `ip` | string | Yes | JSONPath expression extracting the target IP address or hostname. | -| `port` | string | No | JSONPath expression extracting the gNMI port. | -| `targetProfile` | string | No | JSONPath expression extracting the `TargetProfile`. | -| `labels` | map[string]string | No | JSONPath expressions extracting target labels. | - -Labels extracted through `responseMapping.labels` are merged with labels from `spec.targetLabels`. - -If the same label key exists in both locations, labels extracted through `responseMapping.labels` take precedence. - -## Response Format +### Default Response Format If `responseMapping` is not configured, the endpoint must return a JSON array of objects with the following structure: @@ -220,3 +193,32 @@ Example response: } ] ``` + +### Response Mapping via JSONPath + +`responseMapping` allows extracting target fields from arbitrary JSON structures using JSONPath expressions. + +```yaml +spec: + provider: + http: + url: https://inventory.example.com/devices + responseMapping: + name: "$.hostname" + address: "$.management.ip" + port: "$.gnmi.port" + targetProfile: "$.profile" + labels: + role: "$.metadata.role" + site: "$.metadata.site" +``` + +#### Response Mapping Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | string | Yes | JSONPath expression extracting the target name | +| `address` | string | Yes | JSONPath expression extracting the target IP address or hostname | +| `port` | string | No | JSONPath expression extracting the gNMI port | +| `targetProfile` | string | No | JSONPath expression extracting the `TargetProfile` | +| `labels` | map[string]string | No | JSONPath expressions extracting target labels | From 69d82bb884225763cb880ad2055a3f12f50326c2 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Wed, 20 May 2026 21:03:02 -0600 Subject: [PATCH 07/76] changed example router names --- docs/content/docs/user-guide/targetsource/http.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource/http.md b/docs/content/docs/user-guide/targetsource/http.md index b6ff8e4e..b5ff3d02 100644 --- a/docs/content/docs/user-guide/targetsource/http.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -168,7 +168,7 @@ Example response: [ { "name": "spine1", - "address": "spine1", + "address": "spine1.local", "port": 57400, "labels": { "role": "spine" @@ -177,7 +177,7 @@ Example response: }, { "name": "leaf1", - "address": "leaf1", + "address": "leaf1.local", "port": 57400, "labels": { "role": "leaf" @@ -185,7 +185,7 @@ Example response: }, { "name": "leaf2", - "address": "leaf2", + "address": "leaf2.local", "port": 57400, "labels": { "role": "leaf" From 7adee4bd2c11c3161b3fc143a6729eb24b00418b Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Thu, 21 May 2026 13:51:37 +0200 Subject: [PATCH 08/76] add acceptPush --- .../docs/user-guide/targetsource/http.md | 52 ++++++++++++++++++- 1 file changed, 51 insertions(+), 1 deletion(-) diff --git a/docs/content/docs/user-guide/targetsource/http.md b/docs/content/docs/user-guide/targetsource/http.md index b5ff3d02..041bd1bc 100644 --- a/docs/content/docs/user-guide/targetsource/http.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -20,7 +20,7 @@ spec: | Field | Type | Required | Description | |-------|------|----------|-------------| | `url` | string | Yes | URL pointing to the inventory server | -| `acceptPush` | bool | No | Enable webhook-based target updates. Defaults to `false`. | +| `acceptPush` | bool | No | Enable webhook-based target updates. Defaults to `false`. See _acceptPush_ section.| | `authorization` | object | No | Credentials used to access the HTTP endpoint. See _Authorization_ section. | | `pollInterval` | duration | No | Polling interval used to fetch targets from the endpoint. Defaults to `30s`. | | `timeout` | duration | No | Timeout for HTTP requests. Defaults to `10s`. | @@ -222,3 +222,53 @@ spec: | `port` | string | No | JSONPath expression extracting the gNMI port | | `targetProfile` | string | No | JSONPath expression extracting the `TargetProfile` | | `labels` | map[string]string | No | JSONPath expressions extracting target labels | + +## AcceptPush + +Setting acceptPush `True` to true, will enable an REST API endpoint. This allows real-time target updates from a Webhook (or any other application that can send HTTP requests) using `HTTP POST` requests. The API is defined as an openAPI contract. + +### URL + +The URL is `http://:8082/api/v1//target-source//createTargets`. + +- _clusterAddress_: Address of the kubernetes cluster. +- _namespace_: namespace the gNMIc Operator runs in +- _name_: name of the targetSource provider, often `http` . + +### Header + +These header options are mandatory: + +- `-H "Content-Type: application/json"` +- `H "Authorization: Bearer fEPGF5qwV...` + +If acceptPush is enabled, a Kubernetes secret `gnmic-api-auth` is created. It must be passed along in the authorization header, otherwise the POST request is rejected. + +Get `gnmic-api-auth `secret with `kubectl get secret -n gnmic-system gnmic-api-auth -o jsonpath="{.data.bearer-token}" | base64 --decode`. + +#### TLS + +In production, it is highly recommended to use `TLS` and `HTTPS`! For this it is recommended to use a ingress-router/load-balancer that terminates the TLS connection at the Kubernetes edge. + +### Body + +Can we use swagger docs here?! See example below + +#### Example POST request + +``` +curl -X POST "http://localhost:8082/api/v1/default/target-source/http-discovery/createTargets" \ + -H "Authorization: Bearer fEPGF5qwVfM7vvEw2vYuaPojcda/a78aOtqmW4oEFYZUJF67yXluSjDoTKmey5zU" \ + -H "Content-Type: application/json" \ + -d '[ + { + "address": "1.1.1.1:123", + "name": "Router1", + "operation": "created", + "profile": "defaultProfile", + "labels": [ + { "key": "tags", "value": "tag1, tag2" } + ] + } + ]' +``` \ No newline at end of file From 7c3417113509097c939f40a5ca3f1e014846ce9c Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Fri, 29 May 2026 07:48:44 +0000 Subject: [PATCH 09/76] update http provider documentation --- docs/content/docs/reference/api.md | 99 +++++++++++-- .../docs/user-guide/targetsource/_index.md | 126 ++++++++-------- .../docs/user-guide/targetsource/http.md | 135 +++++++++--------- 3 files changed, 220 insertions(+), 140 deletions(-) diff --git a/docs/content/docs/reference/api.md b/docs/content/docs/reference/api.md index 1cfe4fb0..8c6a41f7 100644 --- a/docs/content/docs/reference/api.md +++ b/docs/content/docs/reference/api.md @@ -153,30 +153,111 @@ description: > | Field | Type | Required | Default | Description | |-------|------|----------|---------|-------------| -| `http` | HTTPConfig | No | - | HTTP endpoint for target discovery | -| `consul` | ConsulConfig | No | - | Consul service discovery config | -| `configMap` | string | No | - | ConfigMap name containing targets | -| `podSelector` | LabelSelector | No | - | Select Kubernetes Pods as targets | -| `serviceSelector` | LabelSelector | No | - | Select Kubernetes Services as targets | -| `labels` | map[string]string | No | - | Labels to apply to discovered targets | +| `provider` | ProviderSpec | Yes | - | Provider-specific discovery configuration | +| `targetPort` | int32 | No | - | Default port used when the discovered target does not provide a port | +| `targetProfile` | string | Yes | - | Reference to `TargetProfile` applied to discovered targets | +| `targetLabels` | map[string]string | No | - | Labels added to all discovered targets | + +### ProviderSpec + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `http` | HTTPConfig | No | HTTP provider configuration | ### HTTPConfig +| Field | Type | Required | Default | Description | +|-------|------|----------|---------|-------------| +| `url` | string | No | - | HTTP endpoint used to pull targets. Required unless push is enabled | +| `method` | string | No | GET | HTTP request method | +| `headers` | map[string]string | No | - | HTTP headers to include in requests | +| `body` | string | No | - | Request body for POST requests | +| `authorization` | AuthorizationSpec | No | - | Authentication configuration for the HTTP endpoint | +| `interval` | duration | No | 6h | Polling interval used to refresh targets | +| `timeout` | duration | No | 10s | Timeout for HTTP requests | +| `tls` | ClientTLSConfig | No | - | Client TLS configuration for HTTPS endpoints | +| `pagination` | PaginationSpec | No | - | Pagination settings for parsing responses | +| `mapping` | ResponseMappingSpec | No | - | Response mapping configuration for JSON responses | +| `push` | PushSpec | No | - | Push-based update configuration | + +### ClientTLSConfig + +| Field | Type | Required | Default | Description | +|-------|------|----------|---------|-------------| +| `insecureSkipVerify` | bool | No | false | Skip verification of the server certificate | +| `caBundleRef` | ConfigMapKeySelector | No | - | Reference to a ConfigMap containing a PEM CA bundle | + +### AuthorizationSpec + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `basic` | BasicAuthSpec | No | Basic authentication configuration | +| `token` | TokenAuthSpec | No | Token authentication configuration | + +### BasicAuthSpec + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `credentialsSecretRef` | SecretKeySelector | Yes | Reference to a Secret containing username/password keys | + +### TokenAuthSpec + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `scheme` | string | Yes | Token scheme, e.g. Bearer | +| `tokenSecretRef` | SecretKeySelector | Yes | Reference to a Secret containing the token | + +### PaginationSpec + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `nextField` | string | No | JSON field containing the next page reference or pagination token | + +### ResponseMappingSpec + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `targetsField` | string | No | CEL expression selecting the list of targets from the response | +| `name` | string | No | CEL expression for the target name | +| `address` | string | No | CEL expression for the target address | +| `port` | string | No | CEL expression for the target port | +| `labels` | string | No | CEL expression returning a map of labels | +| `targetProfile` | string | No | CEL expression for the target profile | + +### PushSpec + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `enabled` | bool | No | Enable push updates | +| `auth` | PushAuthSpec | No | Push authentication configuration | + +### PushAuthSpec + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `bearer` | PushBearerAuthSpec | No | Bearer token authentication configuration | +| `signature` | PushSignatureAuthSpec | No | Signature authentication configuration | + +### PushBearerAuthSpec + | Field | Type | Required | Description | |-------|------|----------|-------------| -| `url` | string | Yes | URL of the HTTP endpoint | +| `tokenSecretRef` | SecretKeySelector | Yes | Reference to a Secret containing the bearer token | -### ConsulConfig +### PushSignatureAuthSpec | Field | Type | Required | Description | |-------|------|----------|-------------| -| `url` | string | Yes | Consul server URL | +| `secretRef` | SecretKeySelector | Yes | Reference to a Secret used to verify request signatures | +| `header` | string | Yes | Header containing the signature | +| `algorithm` | string | No | Signature algorithm | ### TargetSourceStatus | Field | Type | Description | |-------|------|-------------| | `status` | string | Sync status (Synced, Error, Pending) | +| `observedGeneration` | int64 | Observed generation of the spec | | `targetsCount` | int32 | Number of discovered targets | | `lastSync` | Time | Last successful sync timestamp | diff --git a/docs/content/docs/user-guide/targetsource/_index.md b/docs/content/docs/user-guide/targetsource/_index.md index 16861e74..7e957ecf 100644 --- a/docs/content/docs/user-guide/targetsource/_index.md +++ b/docs/content/docs/user-guide/targetsource/_index.md @@ -29,8 +29,8 @@ spec: | Field | Type | Required | Description | |-------|------|----------|-------------| | `provider` | object | Yes | Provider-specific discovery configuration. Exactly one provider must be configured | -| `targetPort` | int32 | No | Default port used if the discovered target does not provide a port. | -| `targetProfile` | string | Yes | Reference to `TargetProfile` applied to all targets | +| `targetPort` | int32 | No | Default port used when the discovered target does not provide a port | +| `targetProfile` | string | Yes | Reference to `TargetProfile` applied to all discovered targets | | `targetLabels` | map[string]string | No | Labels added to all discovered targets | @@ -40,9 +40,67 @@ spec: | Provider | Description | |----------|-------------| -| `http` | Discover targets from an HTTP JSON endpoint. [Configuration]({{< relref "http.md" >}}) | +| `http` | Discover targets from an HTTP JSON endpoint or receive webhook updates. [Configuration]({{< relref "http.md" >}}) | + + ## Label Inheritance Each generated `Target` receives an ownership label identifying the originating `TargetSource`: @@ -102,11 +160,12 @@ If the same label key is defined in multiple places, labels are applied in the f ## Status -The TargetSource status shows discovery state: +The `TargetSource` status shows discovery state: ```yaml status: status: Synced + observedGeneration: 1 targetsCount: 42 lastSync: "2024-01-15T10:30:00Z" ``` @@ -114,67 +173,10 @@ status: | Field | Description | |-------|-------------| | `status` | Current sync status (Synced, Error, Pending) | +| `observedGeneration` | Generation of the spec last processed by the controller | | `targetsCount` | Number of targets discovered | | `lastSync` | Timestamp of last successful sync | - - ## Lifecycle ### Target Creation diff --git a/docs/content/docs/user-guide/targetsource/http.md b/docs/content/docs/user-guide/targetsource/http.md index b5ff3d02..fd520942 100644 --- a/docs/content/docs/user-guide/targetsource/http.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -6,7 +6,7 @@ description: > HTTP TargetSource Discovery Provider --- -The HTTP provider discovers targets from an HTTP endpoint returning a JSON array of target definitions. +The HTTP provider discovers targets from an HTTP endpoint returning JSON, or receives webhook-based updates when push mode is enabled. ```yaml spec: @@ -17,39 +17,43 @@ spec: ## HTTP Spec Fields -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| `url` | string | Yes | URL pointing to the inventory server | -| `acceptPush` | bool | No | Enable webhook-based target updates. Defaults to `false`. | -| `authorization` | object | No | Credentials used to access the HTTP endpoint. See _Authorization_ section. | -| `pollInterval` | duration | No | Polling interval used to fetch targets from the endpoint. Defaults to `30s`. | -| `timeout` | duration | No | Timeout for HTTP requests. Defaults to `10s`. | -| `tls` | object | No | Client TLS configuration for HTTPS endpoints. See _TLS_ section. | -| `pagination` | object | No | Pagination configuration for parsing responses from the HTTP endpoint. See _Pagination_ section. | -| `responseMapping` | object | No | JSONPath mapping definitions. See _Response Processing_ section. | +| Field | Type | Required | Default | Description | +|-------|------|----------|---------|-------------| +| `url` | string | No | - | HTTP endpoint used to pull targets. Required unless `push.enabled` is enabled | +| `method` | string | No | GET | HTTP method used for requests | +| `headers` | map[string]string | No | - | HTTP headers to include in requests | +| `body` | string | No | - | Request body for POST requests | +| `authorization` | object | No | - | Authentication configuration for the HTTP endpoint | +| `interval` | duration | No | 6h | Polling interval used to refresh targets | +| `timeout` | duration | No | 10s | Timeout for HTTP requests | +| `tls` | object | No | - | Client TLS configuration for HTTPS endpoints | +| `pagination` | object | No | - | Pagination configuration for parsing HTTP responses | +| `mapping` | object | No | - | Response mapping configuration for JSON responses | +| `push` | object | No | - | Push-based update configuration | -## Authorization - -The HTTP provider supports authenticated requests to the inventory endpoint. - -Exactly one authorization method can be configured. - -### Basic Authentication +## Push Mode -Credentials can either be defined inline or referenced from a Secret. +The HTTP provider supports webhook-based target updates via `spec.provider.http.push`. ```yaml spec: provider: http: - url: https://inventory.example.com/targets - authorization: - basic: - username: admin - password: secret + push: + enabled: true ``` -Using a Secret reference: +When `push.enabled` is true, the operator accepts incoming webhook notifications and can update targets without polling a remote endpoint. The `url` field is optional when push mode is enabled, but can still be used for polling and fallback behavior. + +## Authorization + +The HTTP provider supports authenticated requests to the inventory endpoint. + +Exactly one authorization method can be configured. + +### Basic Authentication + +Credentials are referenced from a Secret. ```yaml spec: @@ -65,20 +69,7 @@ spec: ### Token Authentication -Static token authentication can be configured using either an inline token or a Secret reference. - -```yaml -spec: - provider: - http: - url: https://inventory.example.com/targets - authorization: - token: - scheme: Bearer - token: eyJhbGciOi... -``` - -Using a Secret reference: +Token authentication is configured using a Secret reference. ```yaml spec: @@ -104,17 +95,17 @@ spec: url: https://inventory.example.com/targets tls: insecureSkipVerify: false + caBundleRef: + name: inventory-ca + key: ca.crt ``` ### TLS Fields | Field | Type | Required | Description | |-------|------|----------|-------------| -| `insecureSkipVerify` | bool | No | Skip verification of the server certificate. Defaults to `false`. | -| `caBundle` | []byte | No | Base64-encoded PEM CA bundle used to validate the server certificate. | -| `caBundleSecretRef` | object | No | Reference to a Secret containing a PEM CA bundle. | - -`caBundle` and `caBundleSecretRef` are mutually exclusive. +| `insecureSkipVerify` | bool | No | Skip verification of the server certificate. Defaults to `false` | +| `caBundleRef` | object | No | Reference to a ConfigMap containing a PEM-encoded CA bundle | ## Pagination @@ -126,7 +117,6 @@ spec: http: url: https://inventory.example.com/devices pagination: - itemsField: results nextField: next ``` @@ -134,8 +124,7 @@ spec: | Field | Type | Required | Description | |-------|------|----------|-------------| -| `itemsField` | string | No | Top-level JSON field containing the list of target objects. | -| `nextField` | string | No | Top-level JSON field containing the next page reference or pagination token. | +| `nextField` | string | No | Top-level JSON field containing the next page reference or pagination token | The `nextField` value may either contain: - A full URL for the next request @@ -143,24 +132,24 @@ The `nextField` value may either contain: ## Response Processing -The HTTP provider supports two methods for processing responses from the inventory endpoint: +The HTTP provider supports two response processing modes: -- **Default Response Format**: The endpoint returns a predefined JSON structure understood directly by the operator. -- **Response Mapping via JSONPath**: Arbitrary JSON structures can be mapped to target fields using JSONPath expressions. +- **Default response format**: The endpoint returns a JSON array of target objects. +- **Response mapping**: Custom JSON structures are mapped to target fields using CEL expressions. -If `responseMapping` is configured, the custom mappings are used. Otherwise, the default response format is expected. +If `mapping` is configured, the custom mapping rules are used. Otherwise, the response itself must be a JSON array. ### Default Response Format -If `responseMapping` is not configured, the endpoint must return a JSON array of objects with the following structure: +If `mapping` is not configured, the endpoint must return a JSON array of objects with the following structure: | Field | Type | Required | Description | |-------|------|----------|-------------| | `name` | string | Yes | Name of the generated `Target` resource | | `address` | string | Yes | Device address (FQDN or IP address) | -| `port` | int32 | No | Port used for gNMI connections. If omitted, `spec.targetPort` is used. | +| `port` | int32 | No | Port used for gNMI connections. If omitted, `spec.targetPort` is used | | `labels` | map[string]string | No | Labels added to the generated `Target` resource | -| `targetProfile` | string | No | Reference to a `TargetProfile`. If omitted, `spec.targetProfile` is used. | +| `targetProfile` | string | No | Reference to a `TargetProfile`. If omitted, `spec.targetProfile` is used | Example response: @@ -194,31 +183,39 @@ Example response: ] ``` -### Response Mapping via JSONPath +### Response Mapping via CEL -`responseMapping` allows extracting target fields from arbitrary JSON structures using JSONPath expressions. +`mapping` allows extracting target fields from arbitrary JSON structures using CEL expressions. ```yaml spec: provider: http: url: https://inventory.example.com/devices - responseMapping: - name: "$.hostname" - address: "$.management.ip" - port: "$.gnmi.port" - targetProfile: "$.profile" + mapping: + targetsField: "self.results" + name: "item.hostname" + address: "item.management.ip" + port: "item.gnmi.port" + targetProfile: "item.profile" labels: - role: "$.metadata.role" - site: "$.metadata.site" + role: "item.metadata.role" + site: "item.metadata.site" ``` -#### Response Mapping Fields +#### Mapping Fields | Field | Type | Required | Description | |-------|------|----------|-------------| -| `name` | string | Yes | JSONPath expression extracting the target name | -| `address` | string | Yes | JSONPath expression extracting the target IP address or hostname | -| `port` | string | No | JSONPath expression extracting the gNMI port | -| `targetProfile` | string | No | JSONPath expression extracting the `TargetProfile` | -| `labels` | map[string]string | No | JSONPath expressions extracting target labels | +| `targetsField` | string | No | CEL expression selecting the target list from the response | +| `name` | string | No | CEL expression for the target name | +| `address` | string | No | CEL expression for the target address | +| `port` | string | No | CEL expression for the target port | +| `labels` | string | No | CEL expression returning a map of labels | +| `targetProfile` | string | No | CEL expression for the target profile | + +### CEL variables + +The mapping expressions support the following variables: +- `item`: the current target object +- `self`: the full JSON response From 28d8027f64c29a306f09d6a520780655a63c3157 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Fri, 29 May 2026 09:39:19 +0000 Subject: [PATCH 10/76] tune the http provider doc --- .../docs/user-guide/targetsource/http.md | 361 +++++++++++++++++- 1 file changed, 350 insertions(+), 11 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource/http.md b/docs/content/docs/user-guide/targetsource/http.md index fd520942..2e18d166 100644 --- a/docs/content/docs/user-guide/targetsource/http.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -24,7 +24,7 @@ spec: | `headers` | map[string]string | No | - | HTTP headers to include in requests | | `body` | string | No | - | Request body for POST requests | | `authorization` | object | No | - | Authentication configuration for the HTTP endpoint | -| `interval` | duration | No | 6h | Polling interval used to refresh targets | +| `interval` | duration | No | 30m | Polling interval used to refresh targets | | `timeout` | duration | No | 10s | Timeout for HTTP requests | | `tls` | object | No | - | Client TLS configuration for HTTPS endpoints | | `pagination` | object | No | - | Pagination configuration for parsing HTTP responses | @@ -109,7 +109,7 @@ spec: ## Pagination -Pagination can be configured for APIs returning paginated responses. +Pagination enables the operator to retrieve all targets from APIs that return results in multiple pages. This is common with inventory systems that limit response sizes for performance reasons. ```yaml spec: @@ -124,12 +124,50 @@ spec: | Field | Type | Required | Description | |-------|------|----------|-------------| -| `nextField` | string | No | Top-level JSON field containing the next page reference or pagination token | +| `nextField` | string | No | Top-level JSON field name containing the pagination reference or token | The `nextField` value may either contain: - A full URL for the next request - A pagination token appended as a query parameter to the original URL +### How Pagination Works + +The operator handles two common pagination patterns used by network inventory systems: + +#### 1. Cursor-Based Pagination (Tokens) +When your inventory API returns a pagination token (e.g., `"next": "abc123xyz"`), the operator automatically appends it as a query parameter to construct the next request: + +``` +First request: GET https://inventory.example.com/devices +Response contains: "next": "page2token" +Next request: GET https://inventory.example.com/devices?next=page2token +``` + +Example response: +```json +{ + "devices": [...], + "next": "page2token" +} +``` + +#### 2. URL-Based Pagination +When your API returns a complete URL for the next page (e.g., `"next": "https://inventory.example.com/devices?offset=50"`), the operator uses it directly without modification: + +``` +First request: GET https://inventory.example.com/devices +Response contains: "next": "https://inventory.example.com/devices?offset=50" +Next request: GET https://inventory.example.com/devices?offset=50 +``` + +Example response: +```json +{ + "devices": [...], + "next": "https://inventory.example.com/devices?offset=50" +} +``` + ## Response Processing The HTTP provider supports two response processing modes: @@ -185,7 +223,7 @@ Example response: ### Response Mapping via CEL -`mapping` allows extracting target fields from arbitrary JSON structures using CEL expressions. +When your inventory API's JSON structure differs from the default format, use CEL (Common Expression Language) mapping to extract target fields. ```yaml spec: @@ -198,24 +236,325 @@ spec: address: "item.management.ip" port: "item.gnmi.port" targetProfile: "item.profile" - labels: - role: "item.metadata.role" - site: "item.metadata.site" + labels: "{'role': item.metadata.role, 'site': item.metadata.site}" +``` + +#### Understanding `targetsField` + +The `targetsField` expression tells the operator where to find the list of target objects in your API response. It's particularly important when your API wraps the target list in a data structure. + +**When to use `targetsField`:** +- Your API returns `{"results": [...]}` -> use `"self.results"` +- Your API returns `{"data": {"devices": [...]}}` -> use `"self.data.devices"` +- Your API returns a plain array `[...]` -> omit `targetsField` (default behavior) + +**Example scenarios:** + +*Custom API response example 1:* +```json +{ + "count": 42, + "next": "https://...", + "results": [ + {"id": 1, "name": "device1", "primary_ip": "10.0.0.1"}, + {"id": 2, "name": "device2", "primary_ip": "10.0.0.2"} + ] +} +``` +Usage: `targetsField: "self.results"` + +*Custom API response example 2:* +```json +{ + "status": "success", + "data": { + "timestamp": "2024-01-01T00:00:00Z", + "devices": [ + {"name": "router1", "mgmt_ip": "192.168.1.1"}, + {"name": "router2", "mgmt_ip": "192.168.1.2"} + ] + } +} ``` +Usage: `targetsField: "self.data.devices"` #### Mapping Fields | Field | Type | Required | Description | |-------|------|----------|-------------| -| `targetsField` | string | No | CEL expression selecting the target list from the response | +| `targetsField` | string | No | CEL expression selecting the target list from the response. If omitted, assumes response is a direct JSON array | | `name` | string | No | CEL expression for the target name | | `address` | string | No | CEL expression for the target address | | `port` | string | No | CEL expression for the target port | | `labels` | string | No | CEL expression returning a map of labels | | `targetProfile` | string | No | CEL expression for the target profile | -### CEL variables +#### CEL Variables The mapping expressions support the following variables: -- `item`: the current target object -- `self`: the full JSON response +- `item`: the current target object being processed +- `self`: the complete unprocessed response from the HTTP endpoint + +### Performance: CEL vs Direct Mapping + +Understanding the performance implications helps optimize your configurations: + +**Direct Mapping (No CEL)** - *Fastest* +- Used when your API response matches the default structure exactly +- No expression compilation or evaluation overhead +- Suitable for high-frequency polling (e.g., every minute) +- Example: API returns `[{"name": "...", "address": "..."}]` + +**CEL Mapping** - *Slight overhead* +- CEL expressions are compiled once at startup (not per request) +- Evaluation is performed per target object during each poll cycle +- At high scale (10,000+ targets), consider the `interval` between polls + +**Best practices:** +- Use direct mapping if your API already returns the correct structure +- For large result sets, increase the interval +- Combine CEL and direct mapping for efficiency (see hybrid mapping below) +- Use CEL extensions (see reference table below) to reduce complexity and improve readability + +### CEL Extensions + +The operator includes a set of standard CEL extensions from the official [CEL Go library](https://github.com/google/cel-go) to enable more advanced expressions. + +These [extensions](https://pkg.go.dev/github.com/google/cel-go/ext) expand CEL with additional capabilities commonly needed when transforming API responses: + +| Extension | Purpose | +|----------|----------| +| **Strings** | String manipulation such as splitting values, case conversion, and extracting parts of text (e.g. parsing hostnames or IPs) | +| **Math** | Numeric operations and comparisons (e.g. calculations, min/max, type conversions) | +| **Lists** | Working with arrays (e.g. indexing, filtering, joining values) | +| **Sets** | Set-style operations such as membership checks and comparisons | +| **Regex** | Pattern matching and validation using regular expressions | +| **Bindings** | Defining intermediate variables to simplify complex expressions | + +**Examples:** + +```yaml +mapping: + # Extract site from hostname + labels: | + { + 'site': item.name.split('-')[0] + } + + # Conditional profile + targetProfile: "item.type == 'edge' ? 'edge' : 'core'" + + # Pattern-based classification + labels: | + { + 'role': item.name.matches('^spine') ? 'spine' : 'leaf' + } +``` + +### Combining CEL and Direct Mapping (Hybrid Approach) + +You don't need to map all fields with CEL. The operator supports mixing CEL expressions and direct field lookups for maximum efficiency: + +| Scenario | Behavior | Use Case | +|----------|----------|----------| +| `name`, `address` use CEL; others omitted | Extracts mapped fields via CEL; looks for `port`, `labels`, `targetProfile` directly in item JSON | Simple API where only some fields need transformation | +| Only `labels` uses CEL | Other fields use direct mapping; labels constructed from CEL expression | API returns correct `name`, `address`, `port` but custom labels need extraction | +| Only `address` uses CEL | Direct mapping for other fields; only address requires transformation | Most fields match API exactly except address requires CIDR parsing or format conversion | +| All fields use CEL | Complete transformation via expressions | API structure completely different from expected format | + +This hybrid approach optimizes performance by only compiling and evaluating CEL where needed. + +**Example - Partial CEL mapping (only transform what needs transforming):** +```yaml +mapping: + # name: "item.name" -> OMITTED: already matches default "name" field + address: "item.primary_ip4 != null ? item.primary_ip4.split('/')[0] : item.primary_ip6.split('/')[0]" # CEL: parse CIDR + # port: OMITTED: already exists as "port" field in item + labels: | # CEL: construct labels from custom fields + { + 'site': item.site.name, + 'role': item.device_role.name + } + # targetProfile: OMITTED: already exists as "targetProfile" field in item +``` + +In this example, only `address` and `labels` use CEL expressions; `name`, `port`, and `targetProfile` use direct field lookups for efficiency. + +### Using YAML `|` for Complex CEL Expressions + +When writing more complex CEL expressions, it is recommended to use YAML’s pipe (`|`) literal block instead of inline strings. + +This is especially useful for expressions that span multiple lines or contain nested logic. + +#### Recommended pattern (labels example) + +```yaml +mapping: + labels: | + { + "site": item.site.name, + "rack": item.rack != null ? item.rack.name : "", + "role": item.role != null ? item.role : "unknown", + "tags": item.tags.size() > 0 ? ','.join(item.tags) : "" + } +``` + +**Why use `|` instead of quoted strings:** +- **Readability**: Multi-line expressions are easier to understand +- **Maintainability**: Complex CEL expressions don't require escaping +- **YAML best practice**: Literal blocks handle special characters naturally + +## Recommended Production Settings + +When deploying HTTP TargetSource providers in production networks, follow these guidelines to ensure reliable and efficient target discovery: + +### Polling Configuration +| Scenario | Setting | Rationale | +|----------|---------|-----------| +| **Small environment** (< 100 targets) | `interval: 5m` | Frequent updates without excessive load | +| **Medium environment** (100-500 targets) | `interval: 10m` | Balance between freshness and API load | +| **Large environment** (500-2000 targets) | `interval: 15m` | Reduce API polling overhead | +| **Very large environment** (2000+ targets) | `interval: 30m` | Minimize impact on inventory system | +| **High-frequency changes** | Use `push` mode with `interval` | Enables sub-minute updates via push while periodic polling ensures completeness and consistency | + +**Timeout Configuration:** +```yaml +timeout: 30s # Allows for network latency +``` + +If timeouts consistently occur, increase `interval` instead of timeout (don't poll faster) + +### Authentication & Security + +**Always use TLS in production:** +```yaml +tls: + insecureSkipVerify: false # Never skip verification in production + caBundleRef: + name: inventory-ca-bundle + key: ca.crt +``` + +**For authenticated APIs:** +- Store credentials in Kubernetes Secrets +- Rotate credentials periodically +- Use token-based auth when possible (simpler secret rotation) + +Example: +```yaml +authorization: + token: + scheme: Bearer + tokenSecretRef: + name: inventory-api-token + key: token +``` + +### Pagination & Large Result Sets + +**Configuration for APIs returning large result sets:** +```yaml +pagination: + nextField: next # Always configure pagination if your API supports it + +interval: 30m # Increase interval for large datasets (reduces cumulative API load) +timeout: 60s # Increase only if individual requests are slow or responses are large +``` + +Pagination splits large datasets into multiple smaller HTTP requests. This improves reliability and reduces the likelihood of timeouts compared to fetching a single large response. + +**Optimization strategies:** +- Request API filtering (if supported) to reduce result set size (e.g. ?limit=1000 or ?status=active) +- If the API does not support pagination or filtering increase the timeout +- Consider webhook push mode for frequently-changing inventories (if API supports it) + +### Mapping Optimization + +**Use hybrid CEL and direct mapping for performance:** +```yaml +# EFFICIENT - Only CEL-transform what needs it +mapping: + # These fields exist directly in API response -> no CEL needed + name: "item.hostname" # Direct: omit CEL, fallback to field lookup + # port: (OMITTED) # Direct: exists as "port" in item + + # Only these need transformation -> use CEL + address: "item.primary_ip.split('/')[0]" # CEL: parse CIDR + labels: | # CEL: construct from nested fields + {'site': item.site.name} +``` + +**Avoid unnecessary CEL complexity:** +```yaml +# GOOD - Simple expressions +mapping: + address: "item.management_ip" + port: "int(item.gnmi_port)" + +# AVOID - Nested ternary logic (hard to debug) +mapping: + name: "item.has_override ? item.override_name : (item.hostname != '' ? item.hostname : 'default-' + string(item.id))" +``` + +**CEL expression best practices:** +- Compile expressions once at startup (not per request), so complexity is paid only once +- Use `ext.Bindings` for repeated expressions to avoid redundant evaluation +- Test CEL expressions thoroughly; they're compiled but errors only appear during evaluation +- Keep expressions under 200 characters for maintainability + +### Example Production Configuration + +```yaml +apiVersion: gnmic.openconfig.net/v1alpha1 +kind: TargetSource +metadata: + name: production-inventory +spec: + provider: + http: + # Security + url: https://inventory.prod.example.com/api/dcim/devices/?limit=100 + tls: + insecureSkipVerify: false + caBundleRef: + name: netbox-ca + key: ca.crt + + # Authentication + authorization: + token: + scheme: Bearer + tokenSecretRef: + name: api-token + key: token + + # Timing + interval: 15m # Balanced update frequency + timeout: 30s # Allow for network latency + + # Pagination + pagination: + nextField: next + + # Mapping for fields + mapping: + targetsField: "self.results" + #name: "item.name" -> already handled with fallback direct mapping + address: "item.primary_ip4 != null ? item.primary_ip4.split('/')[0] : item.primary_ip6.split('/')[0]" + port: "item.custom_fields.gnmi_port" + labels: "{\n 'site': item.site.name,\n 'role': item.device_role.name,\n 'status': item.status.value\n }" + targetProfile: "item.custom_fields.gnmi_profile" + + # Global settings + targetPort: 9339 + targetProfile: default-profile +``` + +This configuration ensures: + +- Secure HTTPS communication with certificate validation +- API authentication with token-based credentials +- Balanced polling interval for stable environments +- Proper pagination handling for large device inventories +- Rich label extraction from custom fields +- Fallback to defaults when fields are missing From 929a4c2e1812c54724273b9c5f33278df9eab989 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Fri, 29 May 2026 09:42:54 +0000 Subject: [PATCH 11/76] update default value of timeout --- docs/content/docs/user-guide/targetsource/http.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/docs/user-guide/targetsource/http.md b/docs/content/docs/user-guide/targetsource/http.md index 2e18d166..9e569be1 100644 --- a/docs/content/docs/user-guide/targetsource/http.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -25,7 +25,7 @@ spec: | `body` | string | No | - | Request body for POST requests | | `authorization` | object | No | - | Authentication configuration for the HTTP endpoint | | `interval` | duration | No | 30m | Polling interval used to refresh targets | -| `timeout` | duration | No | 10s | Timeout for HTTP requests | +| `timeout` | duration | No | 30s | Timeout for HTTP requests | | `tls` | object | No | - | Client TLS configuration for HTTPS endpoints | | `pagination` | object | No | - | Pagination configuration for parsing HTTP responses | | `mapping` | object | No | - | Response mapping configuration for JSON responses | From c3fe3ca0abb415646893ddc14a1c0f9c65e16470 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Fri, 29 May 2026 09:49:39 +0000 Subject: [PATCH 12/76] fix CEL expression --- docs/content/docs/user-guide/targetsource/http.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/docs/user-guide/targetsource/http.md b/docs/content/docs/user-guide/targetsource/http.md index 9e569be1..cafa24bd 100644 --- a/docs/content/docs/user-guide/targetsource/http.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -395,7 +395,7 @@ mapping: "site": item.site.name, "rack": item.rack != null ? item.rack.name : "", "role": item.role != null ? item.role : "unknown", - "tags": item.tags.size() > 0 ? ','.join(item.tags) : "" + "tags": item.tags.size() > 0 ? item.tags.join(',') : "" } ``` From 0389b9ff024bc940577d41c56b3959b602eb4038 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Fri, 29 May 2026 12:57:40 +0000 Subject: [PATCH 13/76] docs: update pull mode --- .../docs/user-guide/targetsource/http.md | 26 ++++++++++++++----- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource/http.md b/docs/content/docs/user-guide/targetsource/http.md index cafa24bd..9789961c 100644 --- a/docs/content/docs/user-guide/targetsource/http.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -8,13 +8,6 @@ description: > The HTTP provider discovers targets from an HTTP endpoint returning JSON, or receives webhook-based updates when push mode is enabled. -```yaml -spec: - provider: - http: - url: http://inventory-service:8080/targets -``` - ## HTTP Spec Fields | Field | Type | Required | Default | Description | @@ -31,6 +24,25 @@ spec: | `mapping` | object | No | - | Response mapping configuration for JSON responses | | `push` | object | No | - | Push-based update configuration | +## Pull Mode + +The HTTP provider supports pull-based target discovery by periodically querying a remote HTTP endpoint that returns target data in JSON format. + +```yaml +spec: + provider: + http: + url: http://inventory-service:8080/targets +``` + +In pull mode, the operator sends HTTP requests to the configured url at a fixed interval and updates targets based on the response. The `push.enabled` field is optional when pull mode is enabled, but can still be used for accepting incoming webhook notifications. + +*How Pull Mode Works* +1. The operator sends an HTTP request to the configured url +2. The response is parsed (either directly or via mapping) +3. Targets are created, updated, or removed based on the returned data +4. This process repeats according to the configured interval + ## Push Mode The HTTP provider supports webhook-based target updates via `spec.provider.http.push`. From 31fd036e2afca76555ba603e0f05832aae811426 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Mon, 1 Jun 2026 09:48:17 +0000 Subject: [PATCH 14/76] update docs pagination and hybrid CEL mapping --- .../docs/user-guide/targetsource/http.md | 96 ++++++++++++------- 1 file changed, 63 insertions(+), 33 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource/http.md b/docs/content/docs/user-guide/targetsource/http.md index 9789961c..df1cfd6f 100644 --- a/docs/content/docs/user-guide/targetsource/http.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -121,7 +121,7 @@ spec: ## Pagination -Pagination enables the operator to retrieve all targets from APIs that return results in multiple pages. This is common with inventory systems that limit response sizes for performance reasons. +Pagination enables the operator to retrieve complete result sets from APIs that return data in multiple pages. The operator automatically follows pagination until no further pages are available. ```yaml spec: @@ -129,14 +129,15 @@ spec: http: url: https://inventory.example.com/devices pagination: - nextField: next + nextField: "self.next" ``` ### Pagination Fields | Field | Type | Required | Description | |-------|------|----------|-------------| -| `nextField` | string | No | Top-level JSON field name containing the pagination reference or token | +| `nextField` | string | No | CEL expression used to extract the next page reference from the response | +| `requestParam` | string | No | Query parameter used when the extracted value is a token | The `nextField` value may either contain: - A full URL for the next request @@ -144,42 +145,64 @@ The `nextField` value may either contain: ### How Pagination Works -The operator handles two common pagination patterns used by network inventory systems: +The operator handles the following pagination patterns: + +#### 1. Link Header Pagination +If the API provides a Link response header with `rel="next"`, the operator will automatically follow it. -#### 1. Cursor-Based Pagination (Tokens) -When your inventory API returns a pagination token (e.g., `"next": "abc123xyz"`), the operator automatically appends it as a query parameter to construct the next request: +Example response header: +``` +Link: ; rel="next" +``` +Behavior: ``` -First request: GET https://inventory.example.com/devices -Response contains: "next": "page2token" -Next request: GET https://inventory.example.com/devices?next=page2token +Request 1: GET /devices?page=1 +Request 2: GET /devices?page=2 +Request 3: GET /devices?page=3 +... ``` +#### 2. URL-Based Pagination +If the response contains a full URL in the body (e.g. `"next": "https://..."`), it will be used directly. + Example response: ```json { "devices": [...], - "next": "page2token" + "next": "https://inventory.example.com/devices?offset=50" } ``` -#### 2. URL-Based Pagination -When your API returns a complete URL for the next page (e.g., `"next": "https://inventory.example.com/devices?offset=50"`), the operator uses it directly without modification: +#### 3. Token-Based Pagination +If the response contains a pagination token, the operator appends it as a query parameter. +Example: +```yaml +pagination: + nextField: "self.next_token" + requestParam: "page_token" ``` -First request: GET https://inventory.example.com/devices -Response contains: "next": "https://inventory.example.com/devices?offset=50" -Next request: GET https://inventory.example.com/devices?offset=50 + +Example: +``` +GET /devices +-> "next_token": "abc123" +GET /devices?page_token=abc123 ``` -Example response: -```json -{ - "devices": [...], - "next": "https://inventory.example.com/devices?offset=50" -} +#### CEL-Based Extraction +The nextField is evaluated as a CEL expression using: +- `self` -> entire JSON response + +Example: +```yaml +pagination: + nextField: "self['@odata.nextLink']" ``` +This allows extracting values from nested or special keys. + ## Response Processing The HTTP provider supports two response processing modes: @@ -379,15 +402,22 @@ This hybrid approach optimizes performance by only compiling and evaluating CEL **Example - Partial CEL mapping (only transform what needs transforming):** ```yaml mapping: - # name: "item.name" -> OMITTED: already matches default "name" field + # Use CEL only when you need to transform a field + name: "item.hostname" address: "item.primary_ip4 != null ? item.primary_ip4.split('/')[0] : item.primary_ip6.split('/')[0]" # CEL: parse CIDR - # port: OMITTED: already exists as "port" field in item - labels: | # CEL: construct labels from custom fields + + # Fields that already exist should be omitted + # Port already exists as "port" field in item + # port: item.port <- omit this + + # Use CEL for structured or derived values + labels: | { - 'site': item.site.name, - 'role': item.device_role.name + "site": item.site.name, + "role": item.device_role.name } - # targetProfile: OMITTED: already exists as "targetProfile" field in item + + # targetProfile can also be omitted if already present or not needed ``` In this example, only `address` and `labels` use CEL expressions; `name`, `port`, and `targetProfile` use direct field lookups for efficiency. @@ -407,7 +437,7 @@ mapping: "site": item.site.name, "rack": item.rack != null ? item.rack.name : "", "role": item.role != null ? item.role : "unknown", - "tags": item.tags.size() > 0 ? item.tags.join(',') : "" + "tags": item.tags.join(',') } ``` @@ -427,7 +457,7 @@ When deploying HTTP TargetSource providers in production networks, follow these | **Medium environment** (100-500 targets) | `interval: 10m` | Balance between freshness and API load | | **Large environment** (500-2000 targets) | `interval: 15m` | Reduce API polling overhead | | **Very large environment** (2000+ targets) | `interval: 30m` | Minimize impact on inventory system | -| **High-frequency changes** | Use `push` mode with `interval` | Enables sub-minute updates via push while periodic polling ensures completeness and consistency | +| **High-frequency changes** | Use `push` mode with `interval` | Enables updates via push while periodic polling ensures completeness and consistency | **Timeout Configuration:** ```yaml @@ -486,13 +516,13 @@ Pagination splits large datasets into multiple smaller HTTP requests. This impro ```yaml # EFFICIENT - Only CEL-transform what needs it mapping: - # These fields exist directly in API response -> no CEL needed - name: "item.hostname" # Direct: omit CEL, fallback to field lookup - # port: (OMITTED) # Direct: exists as "port" in item + # + name: "item.hostname" # CEL expression + # port: (OMITTED) # Direct: exists as "port" in item # Only these need transformation -> use CEL address: "item.primary_ip.split('/')[0]" # CEL: parse CIDR - labels: | # CEL: construct from nested fields + labels: | # CEL: construct from nested fields {'site': item.site.name} ``` From c12683d2811fdee774da299d4ff288cef4e863f3 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Mon, 1 Jun 2026 14:39:47 +0000 Subject: [PATCH 15/76] add Netbox example --- .../examples/NetBox/Export Template/_index.md | 338 ++++++++++++++++++ docs/content/docs/examples/NetBox/_index.md | 8 + 2 files changed, 346 insertions(+) create mode 100644 docs/content/docs/examples/NetBox/Export Template/_index.md create mode 100644 docs/content/docs/examples/NetBox/_index.md diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md new file mode 100644 index 00000000..2691a13a --- /dev/null +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -0,0 +1,338 @@ +--- +title: "NetBox (Export Template)" +linkTitle: "NetBox Export" +weight: 3 +description: > + Discover targets from NetBox using HTTP provider with export templates +--- + +This guide shows how to use **NetBox Export Templates** with the HTTP provider to discover and sync targets. + +Export Templates offer powerful filtering, transformation, and formatting directly in NetBox, reducing load on the operator and enabling complex discovery logic. + +## Overview + +An **Export Template** is a Jinja2 template defined in NetBox that: + +1. **Queries** NetBox's internal database (devices, interfaces, etc.) +2. **Filters** results based on custom criteria +3. **Transforms** data into your desired output format (JSON, YAML, CSV, etc.) +4. **Returns** the formatted output via a custom REST API endpoint + +When used with gNMIc's HTTP provider, the operator simply fetches the rendered template and parses the result — no additional transformation needed. + +--- + +## Prerequisites + +- A running Kubernetes cluster with gNMIc Operator installed +- A reachable NetBox instance with **permissions to create Export Templates** +- A NetBox API token +- `kubectl` access to your cluster +- Familiarity with Jinja2 templates + +--- + +## Step 1: Create a Secret for the NetBox API Token + +Create a Kubernetes Secret containing your NetBox API token. This keeps credentials secure and out of your TargetSource manifests. + +```bash +# Substitute YOUR_NETBOX_TOKEN with your actual token +kubectl create secret generic netbox-api-token \ + --from-literal=token=YOUR_NETBOX_TOKEN \ + -n your-namespace +``` + +Verify the Secret was created: + +```bash +kubectl get secret netbox-api-token -n your-namespace -o yaml +``` + +--- + +## Step 2: Create an Export Template in NetBox + +Log in to your NetBox instance and navigate to **Customization > Export Templates**. + +### Step 2a: Create a New Template + +Click **Add Export Template** and fill in the details: + +| Field | Value | Notes | +|-------|-------|-------| +| **Name** | `gNMIc Device Export` | Descriptive name for your template | +| **Content Type** | `dcim > device` | Export template applies to Device objects | +| **Template Code** | (see below) | Jinja2 template | +| **File Extension** | `json` | Output format | +| **Mime Type** | `application/json` | Correct MIME type for JSON | + +### Step 2b: Template Code Example + +#### Basic Template (All Devices) + +```jinja2 +{ + "targets": [ + {% for device in queryset %} + { + "name": "{{ device.name }}", + "address": "{{ device.primary_ip4.address.split('/')[0] }}:57400", + "labels": { + "site": "{{ device.site.name }}", + "role": "{{ device.device_role.name }}", + "region": "{{ device.site.region.name }}", + "type": "{{ device.device_type.model }}" + } + }{{ "," if not loop.last }} + {% endfor %} + ] +} +``` + +#### Advanced Template (Filtered by Status and Role) + +```jinja2 +{ + "targets": [ + {% for device in queryset.filter(status='active', device_role__name__in=['leaf', 'spine']) %} + { + "name": "{{ device.name }}", + "address": "{{ device.primary_ip4.address.split('/')[0] }}:57400", + "labels": { + "site": "{{ device.site.name }}", + "role": "{{ device.device_role.name }}", + "region": "{{ device.site.region.name }}", + "model": "{{ device.device_type.model }}", + "serial": "{{ device.serial }}", + "asset_tag": "{{ device.asset_tag }}" + } + }{{ "," if not loop.last }} + {% endfor %} + ] +} +``` + +**Key template elements:** + +- `queryset`: The filtered set of devices (all unless you add `.filter()`) +- `device.name`: Device hostname +- `device.primary_ip4.address.split('/')[0]`: Extract IP from CIDR (e.g., `192.0.2.1/24` to `192.0.2.1`) +- `device.site.name`, `device.device_role.name`: NetBox relationships (site, role, etc.) +- `loop.last`: Jinja2 loop variable to avoid trailing comma on last item + +### Step 2c: Save and Access the Template + +Once saved, NetBox exposes the template via: + +``` +http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc+Device+Export +``` + +Or fetch it directly: + +```bash +# Replace with your NetBox URL and template name +curl -H "Authorization: Token YOUR_NETBOX_TOKEN" \ + "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" +``` + +The response is a JSON array of targets ready for gNMIc. + +--- + +## Step 3: Create a TargetProfile + +Define how discovered targets should be configured: + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetProfile +metadata: + name: netbox-devices + namespace: your-namespace +spec: + credentialsRef: device-credentials + timeout: 10s +``` + +--- + +## Step 4: Create a TargetSource Using Export Template + +Create a `TargetSource` that references your NetBox export template endpoint: + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: netbox-export-source + namespace: your-namespace +spec: + # Specify the HTTP provider + provider: + http: + # NetBox API endpoint with export template query + # Replace: netbox.example.com, template name (gNMIc+Device+Export), token + url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" + + # Do not embed plaintext tokens in the TargetSource YAML. Instead, use a secret reference: + token: + scheme: Bearer + tokenSecretRef: + name: netbox-api-token + key: token + + + # Reference the TargetProfile + targetProfile: netbox-device + + # Optional: Apply labels to all discovered targets + targetLabels: + inventory: netbox + sync-source: export-template +``` + +--- + +## Step 5: Verify Target Discovery + +Once the `TargetSource` is deployed, verify that targets are being discovered: + +```bash +# List discovered targets +kubectl get targets -n your-namespace + +# Check TargetSource status and sync details +kubectl describe ts netbox-export-source -n your-namespace +``` + +Successful sync shows: + +- `status.status`: "success" +- `status.targetsCount`: number of devices +- `status.lastSync`: recent timestamp + +--- + +## Example: Complete Setup + +Here's a full example combining all components: + +```yaml +--- +# Secret for NetBox API token +apiVersion: v1 +kind: Secret +metadata: + name: netbox-api-token + namespace: gnmic +type: Opaque +data: + # base64-encoded token (echo -n "YOUR_TOKEN" | base64) + token: YOUR_BASE64_ENCODED_TOKEN + +--- +# TargetProfile for NetBox devices +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetProfile +metadata: + name: netbox-device + namespace: gnmic +spec: + credentialsRef: device-credentials + timeout: 10s + +--- +# TargetSource using Export Template +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: netbox-export-source + namespace: gnmic +spec: + provider: + http: + url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" + targetProfile: netbox-device + targetLabels: + inventory: netbox + sync-source: export-template + +--- +# Device Credentials (referenced by TargetProfile) +apiVersion: v1 +kind: Secret +metadata: + name: device-credentials + namespace: gnmic +type: Opaque +data: + username: Z25taWM= # base64: gnmic + password: Z25taaWNQYXNz= # base64: gnmicPass +``` + +--- + +## Advantages of Export Templates + +- **Powerful Filtering**: Filter devices by site, status, role, tags, etc. directly in NetBox +- **Reduced Operator Load**: NetBox handles data transformation; operator just fetches JSON +- **Reusability**: One template can serve multiple consumers +- **Maintainability**: Update discovery logic in NetBox without changing Kubernetes manifests +- **Performance**: Avoids REST API pagination for large inventories + +--- + +## Limitations & Considerations + +### 1. Reverse Proxy and URL Path Rewriting + +If NetBox is behind a reverse proxy with URL path rewriting: + +- **Issue**: The export template endpoint uses query parameters that may not survive proxy transformation. +- **Solution**: + - Ensure the proxy preserves query strings exactly. + - Test the export URL directly: + ```bash + curl -H "Authorization: Token YOUR_TOKEN" \ + "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" + ``` + - If the proxy blocks or modifies parameters, consider using a direct NetBox endpoint without proxying. + +### 2. Large Inventory Rendering + +- Very large device counts can cause NetBox to take time rendering the template. +- **Solution**: + - Use `.filter()` in your template to limit results. + - Create separate export templates for different device groups (e.g., by site or role). + +### 3. Complex Jinja2 Logic + +- NetBox's Jinja2 sandbox restricts some Python functions for security. +- **Solution**: Keep templates simple and use NetBox's built-in filters and objects. Test in the NetBox UI before deploying. + +--- + +## Template Troubleshooting + +### Missing Data in Output + +- **Check**: Are all required fields populated in NetBox? (e.g., `primary_ip4` may be `None` if not set) +- **Solution**: Add conditional checks: + ```jinja2 + {% if device.primary_ip4 %} + "address": "{{ device.primary_ip4.address.split('/')[0] }}" + {% endif %} + ``` + +### Authorization Fails + +If you get a 403 error: + +- Verify the token is valid and not expired. +- Check token permissions in NetBox admin (User > API Tokens). +- Ensure the API token is enabled. + +--- diff --git a/docs/content/docs/examples/NetBox/_index.md b/docs/content/docs/examples/NetBox/_index.md new file mode 100644 index 00000000..4885d60e --- /dev/null +++ b/docs/content/docs/examples/NetBox/_index.md @@ -0,0 +1,8 @@ +--- +title: "NetBox" +linkTitle: "NetBox" +weight: 6 +# draft: true +description: > + Discover targets from NetBox using the HTTP provider +--- From b9ddb8b7b2723f4965f764667ab8f259daef89c0 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Mon, 1 Jun 2026 14:50:35 +0000 Subject: [PATCH 16/76] imporve export template documentation --- .../examples/NetBox/Export Template/_index.md | 39 ++++++++++++++++--- 1 file changed, 34 insertions(+), 5 deletions(-) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index 2691a13a..23bf20ef 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -33,12 +33,25 @@ When used with gNMIc's HTTP provider, the operator simply fetches the rendered t --- -## Step 1: Create a Secret for the NetBox API Token +## Step 1: Create a NetBox API Token -Create a Kubernetes Secret containing your NetBox API token. This keeps credentials secure and out of your TargetSource manifests. +### Step 1a: Create the API Token in NetBox + +Create a dedicated API token in NetBox for gNMIc Operator access. + +1. Log in to NetBox. +2. Go to **Admin > API Tokens**. +3. Click **Add** or **Add token**. +4. Enter a descriptive name such as `gNMIc Operator`. +6. Do not grant "Write enabled" +7. Copy the token value and store it safely; NetBox will not show it again. + +### Step 1b: Store the Token in a Kubernetes Secret + +Create a Kubernetes Secret containing the token so it is not embedded in manifests. ```bash -# Substitute YOUR_NETBOX_TOKEN with your actual token +# API Token Format: nbt_. kubectl create secret generic netbox-api-token \ --from-literal=token=YOUR_NETBOX_TOKEN \ -n your-namespace @@ -144,19 +157,35 @@ The response is a JSON array of targets ready for gNMIc. ## Step 3: Create a TargetProfile -Define how discovered targets should be configured: +Define how discovered targets should be configured. The `TargetProfile` points to a Secret containing device credentials, such as username/password or client certificates. + +Create a credentials Secret first, then reference it from the profile. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: device-credentials + namespace: your-namespace +type: Opaque +stringData: + username: gnmic + password: gnmicPass +``` ```yaml apiVersion: operator.gnmic.dev/v1alpha1 kind: TargetProfile metadata: - name: netbox-devices + name: netbox-device namespace: your-namespace spec: credentialsRef: device-credentials timeout: 10s ``` +For more TargetProfile options and credential handling, see the operator documentation for `TargetProfile`. + --- ## Step 4: Create a TargetSource Using Export Template From 9b7dc0b6b5155788e6bc4adcc44800964ebacdb1 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Mon, 1 Jun 2026 09:38:00 -0600 Subject: [PATCH 17/76] docs: made TargetProfile optional --- docs/content/docs/user-guide/targetsource/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/docs/user-guide/targetsource/_index.md b/docs/content/docs/user-guide/targetsource/_index.md index 7e957ecf..10157b7a 100644 --- a/docs/content/docs/user-guide/targetsource/_index.md +++ b/docs/content/docs/user-guide/targetsource/_index.md @@ -30,7 +30,7 @@ spec: |-------|------|----------|-------------| | `provider` | object | Yes | Provider-specific discovery configuration. Exactly one provider must be configured | | `targetPort` | int32 | No | Default port used when the discovered target does not provide a port | -| `targetProfile` | string | Yes | Reference to `TargetProfile` applied to all discovered targets | +| `targetProfile` | string | No | Reference to default `TargetProfile` applied to all discovered targets if no profile was discovered | | `targetLabels` | map[string]string | No | Labels added to all discovered targets | From b47901792e21496107c2ca7c13179f7c95dfb199 Mon Sep 17 00:00:00 2001 From: Valentino Diller Date: Mon, 1 Jun 2026 10:26:08 -0600 Subject: [PATCH 18/76] changed authorization to authentication --- docs/content/docs/reference/api.md | 4 ++-- docs/content/docs/user-guide/targetsource/http.md | 14 +++++++------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/content/docs/reference/api.md b/docs/content/docs/reference/api.md index 8c6a41f7..d20cb3e5 100644 --- a/docs/content/docs/reference/api.md +++ b/docs/content/docs/reference/api.md @@ -172,7 +172,7 @@ description: > | `method` | string | No | GET | HTTP request method | | `headers` | map[string]string | No | - | HTTP headers to include in requests | | `body` | string | No | - | Request body for POST requests | -| `authorization` | AuthorizationSpec | No | - | Authentication configuration for the HTTP endpoint | +| `authentication` | AuthenticationSpec | No | - | Authentication configuration for the HTTP endpoint | | `interval` | duration | No | 6h | Polling interval used to refresh targets | | `timeout` | duration | No | 10s | Timeout for HTTP requests | | `tls` | ClientTLSConfig | No | - | Client TLS configuration for HTTPS endpoints | @@ -187,7 +187,7 @@ description: > | `insecureSkipVerify` | bool | No | false | Skip verification of the server certificate | | `caBundleRef` | ConfigMapKeySelector | No | - | Reference to a ConfigMap containing a PEM CA bundle | -### AuthorizationSpec +### AuthenticationSpec | Field | Type | Required | Description | |-------|------|----------|-------------| diff --git a/docs/content/docs/user-guide/targetsource/http.md b/docs/content/docs/user-guide/targetsource/http.md index df1cfd6f..2c93fce9 100644 --- a/docs/content/docs/user-guide/targetsource/http.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -16,7 +16,7 @@ The HTTP provider discovers targets from an HTTP endpoint returning JSON, or rec | `method` | string | No | GET | HTTP method used for requests | | `headers` | map[string]string | No | - | HTTP headers to include in requests | | `body` | string | No | - | Request body for POST requests | -| `authorization` | object | No | - | Authentication configuration for the HTTP endpoint | +| `authentication` | object | No | - | Authentication configuration for the HTTP endpoint | | `interval` | duration | No | 30m | Polling interval used to refresh targets | | `timeout` | duration | No | 30s | Timeout for HTTP requests | | `tls` | object | No | - | Client TLS configuration for HTTPS endpoints | @@ -57,11 +57,11 @@ spec: When `push.enabled` is true, the operator accepts incoming webhook notifications and can update targets without polling a remote endpoint. The `url` field is optional when push mode is enabled, but can still be used for polling and fallback behavior. -## Authorization +## Authentication The HTTP provider supports authenticated requests to the inventory endpoint. -Exactly one authorization method can be configured. +Exactly one authentication method can be configured. ### Basic Authentication @@ -72,7 +72,7 @@ spec: provider: http: url: https://inventory.example.com/targets - authorization: + authentication: basic: credentialsSecretRef: name: inventory-credentials @@ -88,7 +88,7 @@ spec: provider: http: url: https://inventory.example.com/targets - authorization: + authentication: token: scheme: Bearer tokenSecretRef: @@ -484,7 +484,7 @@ tls: Example: ```yaml -authorization: +authentication: token: scheme: Bearer tokenSecretRef: @@ -563,7 +563,7 @@ spec: key: ca.crt # Authentication - authorization: + authentication: token: scheme: Bearer tokenSecretRef: From 50866482b21335aa679c3471b466ef5d87d1e3c1 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Tue, 2 Jun 2026 08:51:57 +0000 Subject: [PATCH 19/76] add netbox rest api example --- .../docs/examples/NetBox/REST API/_index.md | 318 ++++++++++++++++++ 1 file changed, 318 insertions(+) create mode 100644 docs/content/docs/examples/NetBox/REST API/_index.md diff --git a/docs/content/docs/examples/NetBox/REST API/_index.md b/docs/content/docs/examples/NetBox/REST API/_index.md new file mode 100644 index 00000000..7998069e --- /dev/null +++ b/docs/content/docs/examples/NetBox/REST API/_index.md @@ -0,0 +1,318 @@ +--- +title: "NetBox (REST API)" +linkTitle: "NetBox REST" +weight: 2 +description: > + Discover targets from NetBox using the HTTP provider and NetBox REST API +--- + +This guide shows how to configure the HTTP provider to discover targets from NetBox using its REST API. + +The REST API approach is direct and straightforward — query NetBox's standard API endpoints to retrieve devices that match your criteria. + +## Prerequisites + +- A running Kubernetes cluster with gNMIc Operator installed +- `kubectl` access to your cluster +- A reachable NetBox instance (inside or outside the cluster) +- A NetBox API token + +## Overview + +The HTTP `TargetSource` loader performs these steps: + +1. **Fetch** JSON device data from a NetBox REST API endpoint (`/api/dcim/devices/`) +2. **Transform** each device record into a gNMIc target using CEL expressions +3. **Create** or **update** `Target` resources in Kubernetes with the extracted data + +--- + +## Step 1: Create a NetBox API Token and Store It Securely + +### Step 1a: Create the API Token in NetBox + +Create a dedicated API token in NetBox for gNMIc Operator access. + +1. Log in to NetBox. +2. Open your user profile or go to **User > API Tokens**. +3. Click **Add** or **Add token**. +4. Enter a descriptive name such as `gNMIc Operator`. +5. Grant the minimum permissions required for read-only device discovery. +6. Copy the token value and store it safely; NetBox will not show it again. + +### Step 1b: Store the Token in a Kubernetes Secret + +Create a Kubernetes Secret containing the token so it is not embedded in manifests. + +```bash +# Substitute YOUR_NETBOX_API_TOKEN with your actual token +# Bearer Token Format (v2): nbt_. +kubectl create secret generic netbox-api-token \ + --from-literal=token=YOUR_NETBOX_API_TOKEN \ + -n your-namespace +``` + +Verify the Secret was created: + +```bash +kubectl get secret netbox-api-token -n your-namespace -o yaml +``` + +--- + +## Step 2: Create a TargetProfile + +Define how discovered targets should be configured. The `TargetProfile` points to a Secret containing device credentials, such as username/password or client certificates. + +Create a credentials Secret first, then reference it from the profile. + +```yaml +# Replace YOUR_DEVICE_USERNAME and YOUR_DEVICE_PASSWORD with your corresponding default device username and password +apiVersion: v1 +kind: Secret +metadata: + name: device-credentials + namespace: your-namespace +type: Opaque +stringData: + username: YOUR_DEVICE_USERNAME + password: YOUR_DEVICE_PASSWORD +``` + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetProfile +metadata: + name: netbox-device + namespace: your-namespace +spec: + credentialsRef: device-credentials + timeout: 10s +``` + +For more TargetProfile options and credential handling, see the operator documentation for `TargetProfile`. + +--- + +## Step 3: Create a TargetSource Using REST API + +The following `TargetSource` queries NetBox's REST API to discover devices: + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: netbox-rest-source + namespace: your-namespace +spec: + targetPort: 57400 + targetProfile: netbox-device + targetLabels: + inventory: netbox + sync-source: rest-api + provider: + http: + url: "http://netbox.example.com:8000/api/dcim/devices/?limit=1000" + method: GET + interval: 5m + timeout: 30s + authorization: + token: + scheme: Bearer + tokenSecretRef: + name: netbox-api-token + key: token + mapping: + targetsField: "self.results" + address: "item.primary_ip4 != null ? item.primary_ip4.address.split('/')[0] : ''" + labels: | + { + "site": item.site.name, + "role": item.device_role.name, + "model": item.device_type.model, + "status": item.status.value + } +``` + + +The HTTP loader supports `targetsField` and individual CEL expressions for: + +- `name` +- `address` +- `port` +- `labels` +- `targetProfile` + +Use `self` for the full response and `item` for each candidate object. + + +--- + +## Step 4: Verify Target Discovery + +Once the `TargetSource` is deployed, check that targets are being discovered and synced: + +```bash +# List discovered targets +kubectl get targets -n your-namespace + +# Check TargetSource status +kubectl describe targetsource netbox-rest-source -n your-namespace +``` + +Look for: +- `status.status`: "success" (or similar) +- `status.targetsCount`: number of discovered devices +- `status.lastSync`: recent timestamp + +--- + +## Example: Complete Setup + +Here's a complete example combining all resources: + +```yaml +--- +# Secret for NetBox API token +apiVersion: v1 +kind: Secret +metadata: + name: netbox-api-token + namespace: gnmic +type: Opaque +data: + # base64-encoded token (echo -n "YOUR_TOKEN" | base64) + token: YOUR_BASE64_ENCODED_TOKEN + +--- +# Secret for Target Credential +apiVersion: v1 +kind: Secret +metadata: + name: device-credentials + namespace: your-namespace +type: Opaque +stringData: + username: YOUR_DEVICE_USERNAME + password: YOUR_DEVICE_PASSWORD + +--- +# TargetProfile +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetProfile +metadata: + name: netbox-device + namespace: your-namespace +spec: + credentialsRef: device-credentials + timeout: 10s + +--- +# TargetSource with REST API +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: netbox-rest-source + namespace: gnmic +spec: + targetPort: 57400 + targetProfile: netbox-device + targetLabels: + inventory: netbox + sync-source: rest-api + provider: + http: + url: "http://netbox.example.com:8000/api/dcim/devices/?limit=1000" + method: GET + interval: 5m + timeout: 30s + authorization: + token: + scheme: Bearer + tokenSecretRef: + name: netbox-api-token + key: token + mapping: + targetsField: "self.results" + address: "item.primary_ip4 != null ? item.primary_ip4.address.split('/')[0] : ''" + labels: | + { + "site": item.site.name, + "role": item.device_role.name, + "model": item.device_type.model, + "status": item.status.value + } +``` + +--- + +## Performance Considerations & Limitations + +### REST API Query Limits + +- **Query Size**: The example uses `limit=1000`. Adjust based on your NetBox instance's pagination settings and response size limits. +- **Response Timeout**: Large device lists can take time. Set appropriate timeouts in your `TargetSource`. + +### Reverse Proxy Considerations + +If NetBox is behind a reverse proxy: + +- **Base URL**: Ensure the reverse proxy correctly handles the `/api/dcim/devices/` path. +- **Authentication**: Some proxies may require additional headers; verify with your proxy and NetBox admin. +- **HTTPS**: If using HTTPS, ensure certificates are trusted by the operator or else use the `tls` setting. + +### Large Inventories + +For inventories with thousands of devices: + +- Consider using **Export Templates** (see [NetBox Export Templates]({{< relref "../Export Template" >}})) for better filtering and performance. +- Implement pagination or filtering in the REST API URL (e.g., `?site=us-west&status=active`). + +--- + +## Security Considerations + +### Token Storage + +- **Never** embed plaintext tokens in manifests or YAML files. +- Always store tokens in Kubernetes Secrets. +- Restrict RBAC permissions on the Secret to only necessary service accounts. + +### HTTPS and Certificates + +If connecting to NetBox via HTTPS: + +- Ensure cluster DNS resolves the hostname correctly. +- Mount CA certificates if using self-signed certificates. +- Verify the operator's HTTP client configuration for certificate validation. + +--- + +## Troubleshooting + +### Show TargetSource Errors + +```bash +kubectl describe targetsource netbox-rest-source -n your-namespace +``` + +### Targets Not Appearing + +- Check that the `TargetProfile` exists and is correctly referenced. +- Verify labels and addresses are being extracted correctly from the NetBox response. +- Review operator logs for parsing errors: + ```bash + kubectl logs -l app=gnmic-operator -n gnmic-operator-system + ``` + +### Rate Limiting or Timeouts + +Increase the sync interval in your `TargetSource` or adjust timeouts: + +```yaml +spec: + provider: + http: + interval: 1h + timeout: 1m +``` From fc6af9cf725484250c3e8aee3534952b5bf5ccad Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Tue, 2 Jun 2026 09:11:06 +0000 Subject: [PATCH 20/76] fix incorrect namespace --- docs/content/docs/examples/NetBox/REST API/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/content/docs/examples/NetBox/REST API/_index.md b/docs/content/docs/examples/NetBox/REST API/_index.md index 7998069e..33d486fe 100644 --- a/docs/content/docs/examples/NetBox/REST API/_index.md +++ b/docs/content/docs/examples/NetBox/REST API/_index.md @@ -178,7 +178,7 @@ apiVersion: v1 kind: Secret metadata: name: netbox-api-token - namespace: gnmic + namespace: your-namespace type: Opaque data: # base64-encoded token (echo -n "YOUR_TOKEN" | base64) @@ -213,7 +213,7 @@ apiVersion: operator.gnmic.dev/v1alpha1 kind: TargetSource metadata: name: netbox-rest-source - namespace: gnmic + namespace: your-namespace spec: targetPort: 57400 targetProfile: netbox-device From f0d63ffed106ce5c0124a24f3e880af959ce33fc Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Tue, 2 Jun 2026 09:11:17 +0000 Subject: [PATCH 21/76] update netbox export template guide --- .../examples/NetBox/Export Template/_index.md | 185 +++++++++--------- 1 file changed, 96 insertions(+), 89 deletions(-) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index 23bf20ef..ad6c35f5 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -1,14 +1,14 @@ --- title: "NetBox (Export Template)" -linkTitle: "NetBox Export" +linkTitle: "NetBox Export Template" weight: 3 description: > - Discover targets from NetBox using HTTP provider with export templates + Discover targets from NetBox using HTTP provider with NetBox Export Template --- This guide shows how to use **NetBox Export Templates** with the HTTP provider to discover and sync targets. -Export Templates offer powerful filtering, transformation, and formatting directly in NetBox, reducing load on the operator and enabling complex discovery logic. +Export Templates offer powerful filtering, transformation, and formatting directly in NetBox, reducing the load on the operator. ## Overview @@ -19,41 +19,42 @@ An **Export Template** is a Jinja2 template defined in NetBox that: 3. **Transforms** data into your desired output format (JSON, YAML, CSV, etc.) 4. **Returns** the formatted output via a custom REST API endpoint -When used with gNMIc's HTTP provider, the operator simply fetches the rendered template and parses the result — no additional transformation needed. +When used with gNMIc's HTTP provider, the operator simply fetches the rendered template and parses the result — no additional gNMIc Operator transformation needed if done correctly. --- ## Prerequisites - A running Kubernetes cluster with gNMIc Operator installed -- A reachable NetBox instance with **permissions to create Export Templates** -- A NetBox API token - `kubectl` access to your cluster +- A reachable NetBox instance with permissions to create Export Templates +- A NetBox API token - Familiarity with Jinja2 templates --- -## Step 1: Create a NetBox API Token +## Step 1: Create a NetBox API Token and Store It Securely ### Step 1a: Create the API Token in NetBox Create a dedicated API token in NetBox for gNMIc Operator access. 1. Log in to NetBox. -2. Go to **Admin > API Tokens**. +2. Open your user profile or go to **User > API Tokens**. 3. Click **Add** or **Add token**. 4. Enter a descriptive name such as `gNMIc Operator`. -6. Do not grant "Write enabled" -7. Copy the token value and store it safely; NetBox will not show it again. +5. Grant the minimum permissions required for read-only device discovery. +6. Copy the token value and store it safely; NetBox will not show it again. ### Step 1b: Store the Token in a Kubernetes Secret Create a Kubernetes Secret containing the token so it is not embedded in manifests. ```bash -# API Token Format: nbt_. +# Substitute YOUR_NETBOX_API_TOKEN with your actual token +# Bearer Token Format (v2): nbt_. kubectl create secret generic netbox-api-token \ - --from-literal=token=YOUR_NETBOX_TOKEN \ + --from-literal=token=YOUR_NETBOX_API_TOKEN \ -n your-namespace ``` @@ -86,45 +87,41 @@ Click **Add Export Template** and fill in the details: #### Basic Template (All Devices) ```jinja2 -{ - "targets": [ - {% for device in queryset %} - { - "name": "{{ device.name }}", - "address": "{{ device.primary_ip4.address.split('/')[0] }}:57400", - "labels": { - "site": "{{ device.site.name }}", - "role": "{{ device.device_role.name }}", - "region": "{{ device.site.region.name }}", - "type": "{{ device.device_type.model }}" - } - }{{ "," if not loop.last }} - {% endfor %} - ] -} +[ + {% for device in queryset %} + { + "name": "{{ device.name }}", + "address": "{{ device.primary_ip4.address.split('/')[0] }}", + "labels": { + "site": "{{ device.site.name }}", + "role": "{{ device.device_role.name }}", + "region": "{{ device.site.region.name }}", + "type": "{{ device.device_type.model }}" + } + }{{ "," if not loop.last }} + {% endfor %} +] ``` #### Advanced Template (Filtered by Status and Role) ```jinja2 -{ - "targets": [ - {% for device in queryset.filter(status='active', device_role__name__in=['leaf', 'spine']) %} - { - "name": "{{ device.name }}", - "address": "{{ device.primary_ip4.address.split('/')[0] }}:57400", - "labels": { - "site": "{{ device.site.name }}", - "role": "{{ device.device_role.name }}", - "region": "{{ device.site.region.name }}", - "model": "{{ device.device_type.model }}", - "serial": "{{ device.serial }}", - "asset_tag": "{{ device.asset_tag }}" - } - }{{ "," if not loop.last }} - {% endfor %} - ] -} +[ + {% for device in queryset.filter(status='active', device_role__name__in=['leaf', 'spine']) %} + { + "name": "{{ device.name }}", + "address": "{{ device.primary_ip4.address.split('/')[0] }}", + "labels": { + "site": "{{ device.site.name }}", + "role": "{{ device.device_role.name }}", + "region": "{{ device.site.region.name }}", + "model": "{{ device.device_type.model }}", + "serial": "{{ device.serial }}", + "asset_tag": "{{ device.asset_tag }}" + } + }{{ "," if not loop.last }} + {% endfor %} +] ``` **Key template elements:** @@ -147,12 +144,16 @@ Or fetch it directly: ```bash # Replace with your NetBox URL and template name -curl -H "Authorization: Token YOUR_NETBOX_TOKEN" \ +# Substitute YOUR_NETBOX_API_TOKEN with your actual token +# Bearer Token Format (v2): nbt_. +curl -H "Authorization: Bearer YOUR_NETBOX_API_TOKEN" \ "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" ``` The response is a JSON array of targets ready for gNMIc. +> If you instead return a JSON object with a nested array, add a mapping section such as `targetsField: "self.targets"` to the TargetSource CR. + --- ## Step 3: Create a TargetProfile @@ -162,6 +163,7 @@ Define how discovered targets should be configured. The `TargetProfile` points t Create a credentials Secret first, then reference it from the profile. ```yaml +# Replace YOUR_DEVICE_USERNAME and YOUR_DEVICE_PASSWORD with your corresponding default device username and password apiVersion: v1 kind: Secret metadata: @@ -169,8 +171,8 @@ metadata: namespace: your-namespace type: Opaque stringData: - username: gnmic - password: gnmicPass + username: YOUR_DEVICE_USERNAME + password: YOUR_DEVICE_PASSWORD ``` ```yaml @@ -199,28 +201,23 @@ metadata: name: netbox-export-source namespace: your-namespace spec: - # Specify the HTTP provider + targetPort: 57400 + targetProfile: netbox-device + targetLabels: + inventory: netbox + sync-source: export-template provider: http: - # NetBox API endpoint with export template query - # Replace: netbox.example.com, template name (gNMIc+Device+Export), token url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" - - # Do not embed plaintext tokens in the TargetSource YAML. Instead, use a secret reference: - token: - scheme: Bearer + method: GET + interval: 30m + timeout: 30s + authorization: + token: + scheme: Token tokenSecretRef: name: netbox-api-token key: token - - - # Reference the TargetProfile - targetProfile: netbox-device - - # Optional: Apply labels to all discovered targets - targetLabels: - inventory: netbox - sync-source: export-template ``` --- @@ -234,12 +231,12 @@ Once the `TargetSource` is deployed, verify that targets are being discovered: kubectl get targets -n your-namespace # Check TargetSource status and sync details -kubectl describe ts netbox-export-source -n your-namespace +kubectl describe targetsource netbox-export-source -n your-namespace ``` Successful sync shows: -- `status.status`: "success" +- `status.status`: "success" (or similar) - `status.targetsCount`: number of devices - `status.lastSync`: recent timestamp @@ -256,50 +253,61 @@ apiVersion: v1 kind: Secret metadata: name: netbox-api-token - namespace: gnmic + namespace: your-namespace type: Opaque data: # base64-encoded token (echo -n "YOUR_TOKEN" | base64) token: YOUR_BASE64_ENCODED_TOKEN --- -# TargetProfile for NetBox devices +# Secret for Target Credential +apiVersion: v1 +kind: Secret +metadata: + name: device-credentials + namespace: your-namespace +type: Opaque +stringData: + username: YOUR_DEVICE_USERNAME + password: YOUR_DEVICE_PASSWORD + +--- +# TargetProfile apiVersion: operator.gnmic.dev/v1alpha1 kind: TargetProfile metadata: name: netbox-device - namespace: gnmic + namespace: your-namespace spec: credentialsRef: device-credentials timeout: 10s + --- # TargetSource using Export Template apiVersion: operator.gnmic.dev/v1alpha1 kind: TargetSource metadata: name: netbox-export-source - namespace: gnmic + namespace: your-namespace spec: - provider: - http: - url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" + targetPort: 57400 targetProfile: netbox-device targetLabels: inventory: netbox sync-source: export-template - ---- -# Device Credentials (referenced by TargetProfile) -apiVersion: v1 -kind: Secret -metadata: - name: device-credentials - namespace: gnmic -type: Opaque -data: - username: Z25taWM= # base64: gnmic - password: Z25taaWNQYXNz= # base64: gnmicPass + provider: + http: + url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" + method: GET + interval: 30m + timeout: 30s + authorization: + token: + scheme: Token + tokenSecretRef: + name: netbox-api-token + key: token ``` --- @@ -340,13 +348,13 @@ If NetBox is behind a reverse proxy with URL path rewriting: ### 3. Complex Jinja2 Logic - NetBox's Jinja2 sandbox restricts some Python functions for security. -- **Solution**: Keep templates simple and use NetBox's built-in filters and objects. Test in the NetBox UI before deploying. +- **Solution**: Keep templates simple and use NetBox's built-in filters and objects. Test URL with curl or similar before deploying. --- ## Template Troubleshooting -### Missing Data in Output +### Missing Targets in Kubernetes - **Check**: Are all required fields populated in NetBox? (e.g., `primary_ip4` may be `None` if not set) - **Solution**: Add conditional checks: @@ -361,7 +369,6 @@ If NetBox is behind a reverse proxy with URL path rewriting: If you get a 403 error: - Verify the token is valid and not expired. -- Check token permissions in NetBox admin (User > API Tokens). - Ensure the API token is enabled. --- From 8aa4ac842a424d5eeacf59cb5846e29631099948 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Tue, 2 Jun 2026 09:13:21 +0000 Subject: [PATCH 22/76] update authorization to authentication --- docs/content/docs/examples/NetBox/Export Template/_index.md | 4 ++-- docs/content/docs/examples/NetBox/REST API/_index.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index ad6c35f5..cb360b0c 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -212,7 +212,7 @@ spec: method: GET interval: 30m timeout: 30s - authorization: + authentication: token: scheme: Token tokenSecretRef: @@ -302,7 +302,7 @@ spec: method: GET interval: 30m timeout: 30s - authorization: + authentication: token: scheme: Token tokenSecretRef: diff --git a/docs/content/docs/examples/NetBox/REST API/_index.md b/docs/content/docs/examples/NetBox/REST API/_index.md index 33d486fe..1ae58b98 100644 --- a/docs/content/docs/examples/NetBox/REST API/_index.md +++ b/docs/content/docs/examples/NetBox/REST API/_index.md @@ -116,7 +116,7 @@ spec: method: GET interval: 5m timeout: 30s - authorization: + authentication: token: scheme: Bearer tokenSecretRef: @@ -226,7 +226,7 @@ spec: method: GET interval: 5m timeout: 30s - authorization: + authentication: token: scheme: Bearer tokenSecretRef: From c3c58af562f74adf7ac2255661eaa08d18f041a0 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Tue, 2 Jun 2026 13:04:59 +0000 Subject: [PATCH 23/76] re-word credentialsSecretRef --- docs/content/docs/user-guide/targetsource/http.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/docs/user-guide/targetsource/http.md b/docs/content/docs/user-guide/targetsource/http.md index 2c93fce9..96ef58c5 100644 --- a/docs/content/docs/user-guide/targetsource/http.md +++ b/docs/content/docs/user-guide/targetsource/http.md @@ -74,7 +74,7 @@ spec: url: https://inventory.example.com/targets authentication: basic: - credentialsSecretRef: + credentialSecretRef: name: inventory-credentials key: username ``` From 7ea9ade81e898fbd58701d0824c6a75b637d114c Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 09:14:51 +0000 Subject: [PATCH 24/76] update targetsource structure --- docs/content/docs/user-guide/targetsource/_index.md | 10 +++++----- .../user-guide/targetsource/{ => providers}/http.md | 0 2 files changed, 5 insertions(+), 5 deletions(-) rename docs/content/docs/user-guide/targetsource/{ => providers}/http.md (100%) diff --git a/docs/content/docs/user-guide/targetsource/_index.md b/docs/content/docs/user-guide/targetsource/_index.md index 10157b7a..00d1e692 100644 --- a/docs/content/docs/user-guide/targetsource/_index.md +++ b/docs/content/docs/user-guide/targetsource/_index.md @@ -17,7 +17,8 @@ metadata: name: targetsource-1 spec: provider: - # see Discovery Providers section + # Configure one of the supported providers: + # https://{{< relref "providers" >}} targetPort: 57400 targetProfile: default targetLabels: @@ -34,13 +35,12 @@ spec: | `targetLabels` | map[string]string | No | Labels added to all discovered targets | -## Discovery Providers - -`TargetSource` supports the following discovery providers: + #### Basic Template (All Devices) From 80021b78e2ac1d2643748a1710ccd04c6d40c6bb Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 13:45:58 +0000 Subject: [PATCH 34/76] comment out relref --- docs/content/docs/examples/NetBox/REST API/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/docs/examples/NetBox/REST API/_index.md b/docs/content/docs/examples/NetBox/REST API/_index.md index 5ffbdfd4..797d15ef 100644 --- a/docs/content/docs/examples/NetBox/REST API/_index.md +++ b/docs/content/docs/examples/NetBox/REST API/_index.md @@ -266,7 +266,7 @@ If NetBox is behind a reverse proxy: For inventories with thousands of devices: -- Consider using **Export Templates** (see [NetBox Export Templates]({{< relref "../Export Template" >}})) for better filtering and performance. + - Implement filtering in the REST API URL (e.g., `?site=us-west&status=active`). --- From 9112eafa1e63b3eda9a42d571b1c7f63022147fb Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 13:48:04 +0000 Subject: [PATCH 35/76] comment out relref --- docs/content/docs/examples/NetBox/REST API/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/docs/examples/NetBox/REST API/_index.md b/docs/content/docs/examples/NetBox/REST API/_index.md index 797d15ef..4b3da8f2 100644 --- a/docs/content/docs/examples/NetBox/REST API/_index.md +++ b/docs/content/docs/examples/NetBox/REST API/_index.md @@ -138,7 +138,7 @@ spec: > This mapping only works for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox API details, see the [NetBox REST API](https://netboxlabs.com/docs/netbox/integrations/rest-api/) documentation. -The HTTP loader supports `targetsField` and individual CEL expressions for `name`, `address`, `port`, `labels`, and `targetProfile`. See the HTTP provider docs "Response Mapping via CEL" section for more details: {{< relref "user-guide/targetsource/providers/http.md" >}}. + Use `self` for the full response and `item` for each candidate object. From 46ca88d091d3b4b4caec56d21f9798ab613ea6f6 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 13:51:15 +0000 Subject: [PATCH 36/76] comment out links --- docs/content/docs/examples/NetBox/Export Template/_index.md | 6 +++--- docs/content/docs/examples/NetBox/REST API/_index.md | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index 36f057b6..99d4d23a 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -48,7 +48,7 @@ Create a dedicated API token in NetBox for gNMIc Operator access. ### Step 1b: Store the Token in a Kubernetes Secret -Create a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing the token. + ```bash # Substitute YOUR_NETBOX_API_TOKEN with your actual token @@ -84,7 +84,7 @@ Click **Add Export Template** and fill in the details: ### Step 2b: Template Code Example -The following Export Templates only work for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox data model details, see the [NetBox Devices Data Model](https://netboxlabs.com/docs/netbox/models/dcim/device/) documentation. + @@ -183,7 +183,7 @@ Sample JSON output produced by the basic export template: ## Step 3: Create a TargetProfile -Define how discovered targets should be configured. The `TargetProfile` points to a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing device credentials, such as username/password or client certificates. + Create a credentials Secret first, then reference it from the TargetProfile. diff --git a/docs/content/docs/examples/NetBox/REST API/_index.md b/docs/content/docs/examples/NetBox/REST API/_index.md index 4b3da8f2..9502f38e 100644 --- a/docs/content/docs/examples/NetBox/REST API/_index.md +++ b/docs/content/docs/examples/NetBox/REST API/_index.md @@ -42,7 +42,7 @@ Create a dedicated API token in NetBox for gNMIc Operator access. ### Step 1b: Store the Token in a Kubernetes Secret -Create a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing the token so it is not embedded in manifests. + ```bash # Substitute YOUR_NETBOX_API_TOKEN with your actual token @@ -62,7 +62,7 @@ kubectl get secret netbox-api-token -n gnmic-system -o yaml ## Step 2: Create a TargetProfile -Define how discovered targets should be configured. The `TargetProfile` points to a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing device credentials, such as username/password or client certificates. + Create a credentials Secret first, then reference it from the profile. @@ -136,7 +136,7 @@ spec: } ``` -> This mapping only works for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox API details, see the [NetBox REST API](https://netboxlabs.com/docs/netbox/integrations/rest-api/) documentation. + From 4ac7261edfec57977d03731be86c7539f9ead01c Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 13:54:53 +0000 Subject: [PATCH 37/76] comment out netbox examples --- docs/content/docs/examples/NetBox/Export Template/_index.md | 4 ++-- docs/content/docs/examples/NetBox/REST API/_index.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index 99d4d23a..9f9b4b2f 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -9,7 +9,7 @@ description: > This guide shows how to use **NetBox Export Templates** with the HTTP provider to discover and sync targets. Export Templates offer powerful filtering, transformation, and formatting directly in NetBox, reducing the load on the operator. - + diff --git a/docs/content/docs/examples/NetBox/REST API/_index.md b/docs/content/docs/examples/NetBox/REST API/_index.md index 9502f38e..e3b6247a 100644 --- a/docs/content/docs/examples/NetBox/REST API/_index.md +++ b/docs/content/docs/examples/NetBox/REST API/_index.md @@ -9,7 +9,7 @@ description: > This guide shows how to configure the HTTP provider to discover targets from NetBox using its REST API. The REST API approach is direct and straightforward — query NetBox's standard API endpoints to retrieve devices that match your criteria. - + From e43fcdacda007db67ce9432199dbeb84589785d7 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:00:01 +0000 Subject: [PATCH 38/76] try finding docu compiling error --- docs/content/docs/examples/NetBox/Export Template/_index.md | 4 ++-- docs/content/docs/examples/NetBox/REST API/_index.md | 4 ++-- docs/content/docs/user-guide/targetsource/providers/http.md | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index 9f9b4b2f..99d4d23a 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -9,7 +9,7 @@ description: > This guide shows how to use **NetBox Export Templates** with the HTTP provider to discover and sync targets. Export Templates offer powerful filtering, transformation, and formatting directly in NetBox, reducing the load on the operator. - +--- diff --git a/docs/content/docs/examples/NetBox/REST API/_index.md b/docs/content/docs/examples/NetBox/REST API/_index.md index e3b6247a..9502f38e 100644 --- a/docs/content/docs/examples/NetBox/REST API/_index.md +++ b/docs/content/docs/examples/NetBox/REST API/_index.md @@ -9,7 +9,7 @@ description: > This guide shows how to configure the HTTP provider to discover targets from NetBox using its REST API. The REST API approach is direct and straightforward — query NetBox's standard API endpoints to retrieve devices that match your criteria. - +``` diff --git a/docs/content/docs/user-guide/targetsource/providers/http.md b/docs/content/docs/user-guide/targetsource/providers/http.md index de0f3b6b..44f31412 100644 --- a/docs/content/docs/user-guide/targetsource/providers/http.md +++ b/docs/content/docs/user-guide/targetsource/providers/http.md @@ -7,7 +7,7 @@ description: > --- The HTTP provider discovers targets from an HTTP endpoint returning JSON, or receives webhook-based updates when push mode is enabled. - + From 3bd1c1e44893cefa44aa25686719e36b021b6700 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:02:40 +0000 Subject: [PATCH 39/76] try finding compile error --- .../examples/NetBox/Export Template/_index.md | 399 ------------------ .../docs/examples/NetBox/REST API/_index.md | 319 -------------- docs/content/docs/examples/NetBox/_index.md | 8 - .../user-guide/targetsource/providers/http.md | 4 +- 4 files changed, 2 insertions(+), 728 deletions(-) delete mode 100644 docs/content/docs/examples/NetBox/Export Template/_index.md delete mode 100644 docs/content/docs/examples/NetBox/REST API/_index.md delete mode 100644 docs/content/docs/examples/NetBox/_index.md diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md deleted file mode 100644 index 99d4d23a..00000000 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ /dev/null @@ -1,399 +0,0 @@ ---- -title: "NetBox (Export Template)" -linkTitle: "NetBox Export Template" -weight: 1 -description: > - Discover targets from NetBox using HTTP provider with NetBox Export Template ---- - -This guide shows how to use **NetBox Export Templates** with the HTTP provider to discover and sync targets. - -Export Templates offer powerful filtering, transformation, and formatting directly in NetBox, reducing the load on the operator. - -## Overview - -An **Export Template** is a Jinja2 template defined in NetBox that: - -1. **Queries** NetBox's internal database (devices, interfaces, etc.) -2. **Filters** results based on custom criteria -3. **Transforms** data into your desired output format (JSON, YAML, CSV, etc.) -4. **Returns** the formatted output via a custom REST API endpoint - -When used with gNMIc's HTTP provider, the operator simply fetches the rendered JSON template and parses the result — no additional gNMIc Operator transformation needed if done correctly. - ---- - -## Prerequisites - -- A running Kubernetes cluster with gNMIc Operator installed -- `kubectl` access to your cluster -- A reachable NetBox instance with permissions to create Export Templates -- A NetBox API token -- Familiarity with Jinja2 templates - ---- - -## Step 1: Create a NetBox API Token and Store It Securely - -### Step 1a: Create the API Token in NetBox - -Create a dedicated API token in NetBox for gNMIc Operator access. - -1. Log in to NetBox. -2. Open your user profile or go to **User > API Tokens**. -3. Click **Add** or **Add token**. -4. Enter a descriptive name such as `gNMIc Operator`. -5. Grant the minimum permissions required for read-only device discovery. -6. Copy the token value and store it safely; NetBox will not show it again. - -### Step 1b: Store the Token in a Kubernetes Secret - - - -```bash -# Substitute YOUR_NETBOX_API_TOKEN with your actual token -# Bearer Token Format (v2): nbt_. -kubectl create secret generic netbox-api-token \ - --from-literal=token=YOUR_NETBOX_API_TOKEN \ - -n gnmic-system -``` - -Verify the Secret was created: - -```bash -kubectl get secret netbox-api-token -n gnmic-system -o yaml -``` - ---- - -## Step 2: Create an Export Template in NetBox - -Log in to your NetBox instance and navigate to **Customization > Export Templates**. - -### Step 2a: Create a New Template - -Click **Add Export Template** and fill in the details: - -| Field | Value | Notes | -|-------|-------|-------| -| **Name** | `gNMIc Device Export` | Descriptive name for your template | -| **Content Type** | `dcim > device` | Export template applies to Device objects | -| **Template Code** | (see below) | Jinja2 template | -| **File Extension** | `json` | Output format | -| **Mime Type** | `application/json` | Correct MIME type for JSON | - -### Step 2b: Template Code Example - - - - - -#### Basic Template (All Devices) - -```jinja2 -[ - {% for device in queryset %} - { - "name": "{{ device.name }}", - "address": "{{ device.primary_ip4.address.ip }}", - "labels": { - "site": "{{ device.site.name }}", - "role": "{{ device.role.name }}", - "region": "{{ device.site.region.name }}", - "type": "{{ device.device_type.model }}" - } - }{{ "," if not loop.last }} - {% endfor %} -] -``` - -#### Advanced Template (Filtered by Status and Role) - -```jinja2 -[ - {% for device in queryset.filter(status='active', role__name__in=['leaf', 'spine']) %} - { - "name": "{{ device.name }}", - "address": "{{ device.primary_ip4.address.ip }}", - "labels": { - "site": "{{ device.site.name }}", - "role": "{{ device.role.name }}", - "region": "{{ device.site.region.name }}", - "model": "{{ device.device_type.model }}", - "serial": "{{ device.serial }}", - "asset_tag": "{{ device.asset_tag }}" - } - }{{ "," if not loop.last }} - {% endfor %} -] -``` - -**Key template elements:** - -- `queryset`: The filtered set of devices (all unless you add `.filter()`) -- `device.name`: Device hostname -- `device.primary_ip4.address.ip`: Primary IPv4 address -- `device.site.name`, `device.device_role.name`: NetBox relationships (site, role, etc.) -- `loop.last`: Jinja2 loop variable to avoid trailing comma on last item - -### Step 2c: Save and Access the Template - -Once saved, NetBox exposes the template via: - -``` -http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc+Device+Export -``` - -Or fetch it directly: - -```bash -# Replace with your NetBox URL and template name -# Substitute YOUR_NETBOX_API_TOKEN with your actual token -# Bearer Token Format (v2): nbt_. -curl -H "Authorization: Bearer YOUR_NETBOX_API_TOKEN" \ - "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" -``` - -The response should be a JSON array of targets ready for the gNMIc Operator. - -Sample JSON output produced by the basic export template: - -```json -[ - - { - "name": "edge-rtr-01.dc1.example.com", - "address": "203.0.113.1", - "labels": { - "site": "DC1", - "role": "edge", - "region": "eu-central-1", - "type": "router" - } - }, - -] -``` - -> Ensure the response is valid JSON and contains no hidden or invalid characters, otherwise the gNMIc Operator will fail to parse it. - -> If you instead return a JSON object with a nested array, add a mapping section such as `targetsField: "self.targets"` to the TargetSource CR. - ---- - -## Step 3: Create a TargetProfile - - - -Create a credentials Secret first, then reference it from the TargetProfile. - -```yaml -# Replace YOUR_DEVICE_USERNAME and YOUR_DEVICE_PASSWORD with your corresponding default device username and password -apiVersion: v1 -kind: Secret -metadata: - name: device-credentials - namespace: gnmic-system -type: Opaque -stringData: - username: YOUR_DEVICE_USERNAME - password: YOUR_DEVICE_PASSWORD -``` - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetProfile -metadata: - name: netbox-device - namespace: gnmic-system -spec: - credentialsRef: device-credentials - timeout: 10s -``` - -For more TargetProfile options and credential handling, see the operator documentation for `TargetProfile`. - ---- - -## Step 4: Create a TargetSource Using Export Template - -Create a `TargetSource` that references your NetBox export template endpoint: - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: netbox-export-source - namespace: gnmic-system -spec: - targetPort: 57400 - targetProfile: netbox-device - targetLabels: - inventory: netbox - sync-source: export-template - provider: - http: - url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" - method: GET - interval: 30m - timeout: 30s - authentication: - token: - scheme: Token - tokenSecretRef: - name: netbox-api-token - key: token -``` - ---- - -## Step 5: Verify Target Discovery - -Once the `TargetSource` is deployed, verify that targets are being discovered: - -```bash -# List discovered targets -kubectl get targets -n gnmic-system - -# Check TargetSource status and sync details -kubectl describe targetsource netbox-export-source -n gnmic-system -``` - -Successful sync shows: - -- `status.status`: "success" (or similar) -- `status.targetsCount`: number of devices -- `status.lastSync`: recent timestamp - ---- - -## Example: Complete Setup - -Here's a full example combining all components: - -```yaml ---- -# Secret for NetBox API token -apiVersion: v1 -kind: Secret -metadata: - name: netbox-api-token - namespace: gnmic-system -type: Opaque -data: - # base64-encoded token (echo -n "YOUR_TOKEN" | base64) - token: YOUR_BASE64_ENCODED_TOKEN - ---- -# Secret for Target Credential -apiVersion: v1 -kind: Secret -metadata: - name: device-credentials - namespace: gnmic-system -type: Opaque -stringData: - username: YOUR_DEVICE_USERNAME - password: YOUR_DEVICE_PASSWORD - ---- -# TargetProfile -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetProfile -metadata: - name: netbox-device - namespace: gnmic-system -spec: - credentialsRef: device-credentials - timeout: 10s - - ---- -# TargetSource using Export Template -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: netbox-export-source - namespace: gnmic-system -spec: - targetPort: 57400 - targetProfile: netbox-device - targetLabels: - inventory: netbox - sync-source: export-template - provider: - http: - url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" - method: GET - interval: 30m - timeout: 30s - authentication: - token: - scheme: Token - tokenSecretRef: - name: netbox-api-token - key: token -``` - ---- - -## Advantages of Export Templates - -- **Powerful Filtering**: Filter devices by site, status, role, tags, etc. directly in NetBox -- **Reduced Operator Load**: NetBox handles data transformation; operator just fetches JSON -- **Reusability**: One template can serve multiple consumers -- **Maintainability**: Update discovery logic in NetBox without changing Kubernetes manifests -- **Performance**: Avoids REST API pagination for large inventories - ---- - -## Limitations & Considerations - -### 1. Reverse Proxy and URL Path Rewriting - -If NetBox is behind a reverse proxy with URL path rewriting: - -- **Issue**: The export template endpoint uses query parameters that may not survive proxy transformation. -- **Solution**: - - Ensure the proxy preserves query strings exactly. - - Test the export URL directly: - ```bash - curl -H "Authorization: Token YOUR_TOKEN" \ - "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" - ``` - - If the proxy blocks or modifies parameters, consider using a direct NetBox endpoint without proxying. - -### 2. Large Inventory Rendering - -- Very large device counts can cause NetBox to take time rendering the template. -- **Solution**: - - Use `.filter()` in your template to limit results. - - Create separate export templates for different device groups (e.g., by site or role). - -### 3. Complex Jinja2 Logic - -- NetBox's Jinja2 sandbox restricts some Python functions for security. -- **Solution**: Keep templates simple and use NetBox's built-in filters and objects. Test the URL with curl or similar before deploying. - ---- - -## Template Troubleshooting - -### Missing Targets in Kubernetes - -- **Check**: Are all required fields populated in NetBox? (e.g., `primary_ip4` may be `None` if not set) -- **Solution**: Add conditional checks: - ```jinja2 - {% if device.primary_ip4 %} - "address": "{{ device.primary_ip4.address.ip }}" - {% endif %} - ``` - -### Authorization Fails - -If you get a 403 error: - -- Verify the token is valid and not expired. -- Ensure the API token is enabled. - ---- diff --git a/docs/content/docs/examples/NetBox/REST API/_index.md b/docs/content/docs/examples/NetBox/REST API/_index.md deleted file mode 100644 index 9502f38e..00000000 --- a/docs/content/docs/examples/NetBox/REST API/_index.md +++ /dev/null @@ -1,319 +0,0 @@ ---- -title: "NetBox (REST API)" -linkTitle: "NetBox REST" -weight: 2 -description: > - Discover targets from NetBox using the HTTP provider and NetBox REST API ---- - -This guide shows how to configure the HTTP provider to discover targets from NetBox using its REST API. - -The REST API approach is direct and straightforward — query NetBox's standard API endpoints to retrieve devices that match your criteria. - -## Prerequisites - -- A running Kubernetes cluster with gNMIc Operator installed -- `kubectl` access to your cluster -- A reachable NetBox instance (inside or outside the cluster) -- A NetBox API token - -## Overview - -The HTTP `TargetSource` loader performs these steps: - -1. **Fetch** JSON device data from a NetBox REST API endpoint (`/api/dcim/devices/`) -2. **Transform** each device record into a gNMIc target using CEL expressions -3. **Create** or **update** `Target` resources in Kubernetes with the extracted data - ---- - -## Step 1: Create a NetBox API Token and Store It Securely - -### Step 1a: Create the API Token in NetBox - -Create a dedicated API token in NetBox for gNMIc Operator access. - -1. Log in to NetBox. -2. Open your user profile or go to **User > API Tokens**. -3. Click **Add** or **Add token**. -4. Enter a descriptive name such as `gNMIc Operator`. -5. Grant the minimum permissions required for read-only device discovery. -6. Copy the token value and store it safely; NetBox will not show it again. - -### Step 1b: Store the Token in a Kubernetes Secret - - - -```bash -# Substitute YOUR_NETBOX_API_TOKEN with your actual token -# Bearer Token Format (v2): nbt_. -kubectl create secret generic netbox-api-token \ - --from-literal=token=YOUR_NETBOX_API_TOKEN \ - -n gnmic-system -``` - -Verify the Secret was created: - -```bash -kubectl get secret netbox-api-token -n gnmic-system -o yaml -``` - ---- - -## Step 2: Create a TargetProfile - - - -Create a credentials Secret first, then reference it from the profile. - -```yaml -# Replace YOUR_DEVICE_USERNAME and YOUR_DEVICE_PASSWORD with your corresponding default device username and password -apiVersion: v1 -kind: Secret -metadata: - name: device-credentials - namespace: gnmic-system -type: Opaque -stringData: - username: YOUR_DEVICE_USERNAME - password: YOUR_DEVICE_PASSWORD -``` - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetProfile -metadata: - name: netbox-device - namespace: gnmic-system -spec: - credentialsRef: device-credentials - timeout: 10s -``` - -For more TargetProfile options and credential handling, see the operator documentation for `TargetProfile`. - ---- - -## Step 3: Create a TargetSource Using REST API - -The following `TargetSource` queries NetBox's REST API to discover devices: - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: netbox-rest-source - namespace: gnmic-system -spec: - targetPort: 57400 - targetProfile: netbox-device - targetLabels: - inventory: netbox - sync-source: rest-api - provider: - http: - url: "http://netbox.example.com:8000/api/dcim/devices/?limit=1000" - method: GET - interval: 5m - timeout: 30s - authentication: - token: - scheme: Bearer - tokenSecretRef: - name: netbox-api-token - key: token - pagination: - nextField: "next" - mapping: - targetsField: "self.results" - address: "item.primary_ip4 != null ? item.primary_ip4.address.split('/')[0] : ''" - labels: | - { - "site": item.site.name, - "role": item.device_role.name, - "model": item.device_type.model, - "status": item.status.value - } -``` - - - - - -Use `self` for the full response and `item` for each candidate object. - ---- - -## Step 4: Apply and Verify Target Discovery - -Deploy the `TargetSource` and check that targets are being discovered and synced: - -```bash -# List discovered targets -kubectl apply -f /path/to/targetsource.yaml -n gnmic-system - -# List discovered targets -kubectl get targets -n gnmic-system - -# Check TargetSource status -kubectl describe targetsource netbox-rest-source -n gnmic-system -``` - -Look for: -- `status.status`: "success" (or similar) -- `status.targetsCount`: number of discovered devices -- `status.lastSync`: recent timestamp - ---- - -## Example: Complete Setup - -Here's a complete example combining all resources: - -```yaml ---- -# Secret for NetBox API token -apiVersion: v1 -kind: Secret -metadata: - name: netbox-api-token - namespace: gnmic-system -type: Opaque -data: - # base64-encoded token (echo -n "YOUR_TOKEN" | base64) - token: YOUR_BASE64_ENCODED_TOKEN - ---- -# Secret for Target Credential -apiVersion: v1 -kind: Secret -metadata: - name: device-credentials - namespace: gnmic-system -type: Opaque -stringData: - username: YOUR_DEVICE_USERNAME - password: YOUR_DEVICE_PASSWORD - ---- -# TargetProfile -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetProfile -metadata: - name: netbox-device - namespace: gnmic-system -spec: - credentialsRef: device-credentials - timeout: 10s - ---- -# TargetSource with REST API -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: netbox-rest-source - namespace: gnmic-system -spec: - targetPort: 57400 - targetProfile: netbox-device - targetLabels: - inventory: netbox - sync-source: rest-api - provider: - http: - url: "http://netbox.example.com:8000/api/dcim/devices/?limit=1000" - method: GET - interval: 5m - timeout: 30s - authentication: - token: - scheme: Bearer - tokenSecretRef: - name: netbox-api-token - key: token - pagination: - nextField: "next" - mapping: - targetsField: "self.results" - address: "item.primary_ip4 != null ? item.primary_ip4.address.split('/')[0] : ''" - labels: | - { - "site": item.site.name, - "role": item.device_role.name, - "model": item.device_type.model, - "status": item.status.value - } -``` - ---- - -## Performance Considerations & Limitations - -### REST API Query Limits - -- **Query Size**: The example uses `limit=1000`. Adjust based on your NetBox instance's pagination settings and response size limits. -- **Response Timeout**: Large device lists can take time. Set appropriate timeouts in your `TargetSource`. - -### Reverse Proxy Considerations - -If NetBox is behind a reverse proxy: - -- **Base URL**: Ensure the reverse proxy correctly handles the `/api/dcim/devices/` path. -- **Authentication**: Some proxies may require additional headers; verify with your proxy and NetBox admin. -- **HTTPS**: If using HTTPS, ensure certificates are trusted by the operator or else use the `tls` setting. - -### Large Inventories - -For inventories with thousands of devices: - - -- Implement filtering in the REST API URL (e.g., `?site=us-west&status=active`). - ---- - -## Security Considerations - -### Token and Credentials - -- **Never** embed plaintext tokens or credentials in manifests or YAML files. -- Always store tokens in Kubernetes Secrets. -- Restrict RBAC permissions on the Secret to only necessary service accounts. - -### HTTPS and Certificates - -If connecting to NetBox via HTTPS: - -- Ensure cluster DNS resolves the hostname correctly. -- Mount CA certificates if using self-signed certificates. -- Verify the operator's HTTP client configuration for certificate validation. - ---- - -## Troubleshooting - -### Show TargetSource Errors - -```bash -kubectl describe targetsource netbox-rest-source -n gnmic-system -``` - -### Targets Not Appearing - -- Check that the `TargetProfile` exists and is correctly referenced. -- Verify labels and addresses are being extracted correctly from the NetBox response. -- Review operator logs for parsing errors: - ```bash - kubectl logs -l app=gnmic-operator -n gnmic-operator-system - ``` - -### Rate Limiting or Timeouts - -Increase the sync interval in your `TargetSource` or adjust timeouts: - -```yaml -spec: - provider: - http: - interval: 1h - timeout: 1m -``` diff --git a/docs/content/docs/examples/NetBox/_index.md b/docs/content/docs/examples/NetBox/_index.md deleted file mode 100644 index 4885d60e..00000000 --- a/docs/content/docs/examples/NetBox/_index.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: "NetBox" -linkTitle: "NetBox" -weight: 6 -# draft: true -description: > - Discover targets from NetBox using the HTTP provider ---- diff --git a/docs/content/docs/user-guide/targetsource/providers/http.md b/docs/content/docs/user-guide/targetsource/providers/http.md index 44f31412..de0f3b6b 100644 --- a/docs/content/docs/user-guide/targetsource/providers/http.md +++ b/docs/content/docs/user-guide/targetsource/providers/http.md @@ -7,7 +7,7 @@ description: > --- The HTTP provider discovers targets from an HTTP endpoint returning JSON, or receives webhook-based updates when push mode is enabled. - +- Fallback to defaults when fields are missing From c456806fe4dfe412df2d1b13f3417e7e6f917adf Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:04:29 +0000 Subject: [PATCH 40/76] add example folder NetBox --- docs/content/docs/examples/NetBox/_index.md | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 docs/content/docs/examples/NetBox/_index.md diff --git a/docs/content/docs/examples/NetBox/_index.md b/docs/content/docs/examples/NetBox/_index.md new file mode 100644 index 00000000..4885d60e --- /dev/null +++ b/docs/content/docs/examples/NetBox/_index.md @@ -0,0 +1,8 @@ +--- +title: "NetBox" +linkTitle: "NetBox" +weight: 6 +# draft: true +description: > + Discover targets from NetBox using the HTTP provider +--- From ef97608a745f26fc2bcd1aef0b590130e5d328d3 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:05:32 +0000 Subject: [PATCH 41/76] add example Netbox Export Template --- .../examples/NetBox/Export Template/_index.md | 399 ++++++++++++++++++ 1 file changed, 399 insertions(+) create mode 100644 docs/content/docs/examples/NetBox/Export Template/_index.md diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md new file mode 100644 index 00000000..99d4d23a --- /dev/null +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -0,0 +1,399 @@ +--- +title: "NetBox (Export Template)" +linkTitle: "NetBox Export Template" +weight: 1 +description: > + Discover targets from NetBox using HTTP provider with NetBox Export Template +--- + +This guide shows how to use **NetBox Export Templates** with the HTTP provider to discover and sync targets. + +Export Templates offer powerful filtering, transformation, and formatting directly in NetBox, reducing the load on the operator. + +## Overview + +An **Export Template** is a Jinja2 template defined in NetBox that: + +1. **Queries** NetBox's internal database (devices, interfaces, etc.) +2. **Filters** results based on custom criteria +3. **Transforms** data into your desired output format (JSON, YAML, CSV, etc.) +4. **Returns** the formatted output via a custom REST API endpoint + +When used with gNMIc's HTTP provider, the operator simply fetches the rendered JSON template and parses the result — no additional gNMIc Operator transformation needed if done correctly. + +--- + +## Prerequisites + +- A running Kubernetes cluster with gNMIc Operator installed +- `kubectl` access to your cluster +- A reachable NetBox instance with permissions to create Export Templates +- A NetBox API token +- Familiarity with Jinja2 templates + +--- + +## Step 1: Create a NetBox API Token and Store It Securely + +### Step 1a: Create the API Token in NetBox + +Create a dedicated API token in NetBox for gNMIc Operator access. + +1. Log in to NetBox. +2. Open your user profile or go to **User > API Tokens**. +3. Click **Add** or **Add token**. +4. Enter a descriptive name such as `gNMIc Operator`. +5. Grant the minimum permissions required for read-only device discovery. +6. Copy the token value and store it safely; NetBox will not show it again. + +### Step 1b: Store the Token in a Kubernetes Secret + + + +```bash +# Substitute YOUR_NETBOX_API_TOKEN with your actual token +# Bearer Token Format (v2): nbt_. +kubectl create secret generic netbox-api-token \ + --from-literal=token=YOUR_NETBOX_API_TOKEN \ + -n gnmic-system +``` + +Verify the Secret was created: + +```bash +kubectl get secret netbox-api-token -n gnmic-system -o yaml +``` + +--- + +## Step 2: Create an Export Template in NetBox + +Log in to your NetBox instance and navigate to **Customization > Export Templates**. + +### Step 2a: Create a New Template + +Click **Add Export Template** and fill in the details: + +| Field | Value | Notes | +|-------|-------|-------| +| **Name** | `gNMIc Device Export` | Descriptive name for your template | +| **Content Type** | `dcim > device` | Export template applies to Device objects | +| **Template Code** | (see below) | Jinja2 template | +| **File Extension** | `json` | Output format | +| **Mime Type** | `application/json` | Correct MIME type for JSON | + +### Step 2b: Template Code Example + + + + + +#### Basic Template (All Devices) + +```jinja2 +[ + {% for device in queryset %} + { + "name": "{{ device.name }}", + "address": "{{ device.primary_ip4.address.ip }}", + "labels": { + "site": "{{ device.site.name }}", + "role": "{{ device.role.name }}", + "region": "{{ device.site.region.name }}", + "type": "{{ device.device_type.model }}" + } + }{{ "," if not loop.last }} + {% endfor %} +] +``` + +#### Advanced Template (Filtered by Status and Role) + +```jinja2 +[ + {% for device in queryset.filter(status='active', role__name__in=['leaf', 'spine']) %} + { + "name": "{{ device.name }}", + "address": "{{ device.primary_ip4.address.ip }}", + "labels": { + "site": "{{ device.site.name }}", + "role": "{{ device.role.name }}", + "region": "{{ device.site.region.name }}", + "model": "{{ device.device_type.model }}", + "serial": "{{ device.serial }}", + "asset_tag": "{{ device.asset_tag }}" + } + }{{ "," if not loop.last }} + {% endfor %} +] +``` + +**Key template elements:** + +- `queryset`: The filtered set of devices (all unless you add `.filter()`) +- `device.name`: Device hostname +- `device.primary_ip4.address.ip`: Primary IPv4 address +- `device.site.name`, `device.device_role.name`: NetBox relationships (site, role, etc.) +- `loop.last`: Jinja2 loop variable to avoid trailing comma on last item + +### Step 2c: Save and Access the Template + +Once saved, NetBox exposes the template via: + +``` +http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc+Device+Export +``` + +Or fetch it directly: + +```bash +# Replace with your NetBox URL and template name +# Substitute YOUR_NETBOX_API_TOKEN with your actual token +# Bearer Token Format (v2): nbt_. +curl -H "Authorization: Bearer YOUR_NETBOX_API_TOKEN" \ + "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" +``` + +The response should be a JSON array of targets ready for the gNMIc Operator. + +Sample JSON output produced by the basic export template: + +```json +[ + + { + "name": "edge-rtr-01.dc1.example.com", + "address": "203.0.113.1", + "labels": { + "site": "DC1", + "role": "edge", + "region": "eu-central-1", + "type": "router" + } + }, + +] +``` + +> Ensure the response is valid JSON and contains no hidden or invalid characters, otherwise the gNMIc Operator will fail to parse it. + +> If you instead return a JSON object with a nested array, add a mapping section such as `targetsField: "self.targets"` to the TargetSource CR. + +--- + +## Step 3: Create a TargetProfile + + + +Create a credentials Secret first, then reference it from the TargetProfile. + +```yaml +# Replace YOUR_DEVICE_USERNAME and YOUR_DEVICE_PASSWORD with your corresponding default device username and password +apiVersion: v1 +kind: Secret +metadata: + name: device-credentials + namespace: gnmic-system +type: Opaque +stringData: + username: YOUR_DEVICE_USERNAME + password: YOUR_DEVICE_PASSWORD +``` + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetProfile +metadata: + name: netbox-device + namespace: gnmic-system +spec: + credentialsRef: device-credentials + timeout: 10s +``` + +For more TargetProfile options and credential handling, see the operator documentation for `TargetProfile`. + +--- + +## Step 4: Create a TargetSource Using Export Template + +Create a `TargetSource` that references your NetBox export template endpoint: + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: netbox-export-source + namespace: gnmic-system +spec: + targetPort: 57400 + targetProfile: netbox-device + targetLabels: + inventory: netbox + sync-source: export-template + provider: + http: + url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" + method: GET + interval: 30m + timeout: 30s + authentication: + token: + scheme: Token + tokenSecretRef: + name: netbox-api-token + key: token +``` + +--- + +## Step 5: Verify Target Discovery + +Once the `TargetSource` is deployed, verify that targets are being discovered: + +```bash +# List discovered targets +kubectl get targets -n gnmic-system + +# Check TargetSource status and sync details +kubectl describe targetsource netbox-export-source -n gnmic-system +``` + +Successful sync shows: + +- `status.status`: "success" (or similar) +- `status.targetsCount`: number of devices +- `status.lastSync`: recent timestamp + +--- + +## Example: Complete Setup + +Here's a full example combining all components: + +```yaml +--- +# Secret for NetBox API token +apiVersion: v1 +kind: Secret +metadata: + name: netbox-api-token + namespace: gnmic-system +type: Opaque +data: + # base64-encoded token (echo -n "YOUR_TOKEN" | base64) + token: YOUR_BASE64_ENCODED_TOKEN + +--- +# Secret for Target Credential +apiVersion: v1 +kind: Secret +metadata: + name: device-credentials + namespace: gnmic-system +type: Opaque +stringData: + username: YOUR_DEVICE_USERNAME + password: YOUR_DEVICE_PASSWORD + +--- +# TargetProfile +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetProfile +metadata: + name: netbox-device + namespace: gnmic-system +spec: + credentialsRef: device-credentials + timeout: 10s + + +--- +# TargetSource using Export Template +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: netbox-export-source + namespace: gnmic-system +spec: + targetPort: 57400 + targetProfile: netbox-device + targetLabels: + inventory: netbox + sync-source: export-template + provider: + http: + url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" + method: GET + interval: 30m + timeout: 30s + authentication: + token: + scheme: Token + tokenSecretRef: + name: netbox-api-token + key: token +``` + +--- + +## Advantages of Export Templates + +- **Powerful Filtering**: Filter devices by site, status, role, tags, etc. directly in NetBox +- **Reduced Operator Load**: NetBox handles data transformation; operator just fetches JSON +- **Reusability**: One template can serve multiple consumers +- **Maintainability**: Update discovery logic in NetBox without changing Kubernetes manifests +- **Performance**: Avoids REST API pagination for large inventories + +--- + +## Limitations & Considerations + +### 1. Reverse Proxy and URL Path Rewriting + +If NetBox is behind a reverse proxy with URL path rewriting: + +- **Issue**: The export template endpoint uses query parameters that may not survive proxy transformation. +- **Solution**: + - Ensure the proxy preserves query strings exactly. + - Test the export URL directly: + ```bash + curl -H "Authorization: Token YOUR_TOKEN" \ + "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" + ``` + - If the proxy blocks or modifies parameters, consider using a direct NetBox endpoint without proxying. + +### 2. Large Inventory Rendering + +- Very large device counts can cause NetBox to take time rendering the template. +- **Solution**: + - Use `.filter()` in your template to limit results. + - Create separate export templates for different device groups (e.g., by site or role). + +### 3. Complex Jinja2 Logic + +- NetBox's Jinja2 sandbox restricts some Python functions for security. +- **Solution**: Keep templates simple and use NetBox's built-in filters and objects. Test the URL with curl or similar before deploying. + +--- + +## Template Troubleshooting + +### Missing Targets in Kubernetes + +- **Check**: Are all required fields populated in NetBox? (e.g., `primary_ip4` may be `None` if not set) +- **Solution**: Add conditional checks: + ```jinja2 + {% if device.primary_ip4 %} + "address": "{{ device.primary_ip4.address.ip }}" + {% endif %} + ``` + +### Authorization Fails + +If you get a 403 error: + +- Verify the token is valid and not expired. +- Ensure the API token is enabled. + +--- From 7a64601249dce5932c4f5ad6d24c67427ab306cc Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:06:39 +0000 Subject: [PATCH 42/76] minimize export template example --- .../examples/NetBox/Export Template/_index.md | 388 ------------------ 1 file changed, 388 deletions(-) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index 99d4d23a..0d76f1f7 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -9,391 +9,3 @@ description: > This guide shows how to use **NetBox Export Templates** with the HTTP provider to discover and sync targets. Export Templates offer powerful filtering, transformation, and formatting directly in NetBox, reducing the load on the operator. - -## Overview - -An **Export Template** is a Jinja2 template defined in NetBox that: - -1. **Queries** NetBox's internal database (devices, interfaces, etc.) -2. **Filters** results based on custom criteria -3. **Transforms** data into your desired output format (JSON, YAML, CSV, etc.) -4. **Returns** the formatted output via a custom REST API endpoint - -When used with gNMIc's HTTP provider, the operator simply fetches the rendered JSON template and parses the result — no additional gNMIc Operator transformation needed if done correctly. - ---- - -## Prerequisites - -- A running Kubernetes cluster with gNMIc Operator installed -- `kubectl` access to your cluster -- A reachable NetBox instance with permissions to create Export Templates -- A NetBox API token -- Familiarity with Jinja2 templates - ---- - -## Step 1: Create a NetBox API Token and Store It Securely - -### Step 1a: Create the API Token in NetBox - -Create a dedicated API token in NetBox for gNMIc Operator access. - -1. Log in to NetBox. -2. Open your user profile or go to **User > API Tokens**. -3. Click **Add** or **Add token**. -4. Enter a descriptive name such as `gNMIc Operator`. -5. Grant the minimum permissions required for read-only device discovery. -6. Copy the token value and store it safely; NetBox will not show it again. - -### Step 1b: Store the Token in a Kubernetes Secret - - - -```bash -# Substitute YOUR_NETBOX_API_TOKEN with your actual token -# Bearer Token Format (v2): nbt_. -kubectl create secret generic netbox-api-token \ - --from-literal=token=YOUR_NETBOX_API_TOKEN \ - -n gnmic-system -``` - -Verify the Secret was created: - -```bash -kubectl get secret netbox-api-token -n gnmic-system -o yaml -``` - ---- - -## Step 2: Create an Export Template in NetBox - -Log in to your NetBox instance and navigate to **Customization > Export Templates**. - -### Step 2a: Create a New Template - -Click **Add Export Template** and fill in the details: - -| Field | Value | Notes | -|-------|-------|-------| -| **Name** | `gNMIc Device Export` | Descriptive name for your template | -| **Content Type** | `dcim > device` | Export template applies to Device objects | -| **Template Code** | (see below) | Jinja2 template | -| **File Extension** | `json` | Output format | -| **Mime Type** | `application/json` | Correct MIME type for JSON | - -### Step 2b: Template Code Example - - - - - -#### Basic Template (All Devices) - -```jinja2 -[ - {% for device in queryset %} - { - "name": "{{ device.name }}", - "address": "{{ device.primary_ip4.address.ip }}", - "labels": { - "site": "{{ device.site.name }}", - "role": "{{ device.role.name }}", - "region": "{{ device.site.region.name }}", - "type": "{{ device.device_type.model }}" - } - }{{ "," if not loop.last }} - {% endfor %} -] -``` - -#### Advanced Template (Filtered by Status and Role) - -```jinja2 -[ - {% for device in queryset.filter(status='active', role__name__in=['leaf', 'spine']) %} - { - "name": "{{ device.name }}", - "address": "{{ device.primary_ip4.address.ip }}", - "labels": { - "site": "{{ device.site.name }}", - "role": "{{ device.role.name }}", - "region": "{{ device.site.region.name }}", - "model": "{{ device.device_type.model }}", - "serial": "{{ device.serial }}", - "asset_tag": "{{ device.asset_tag }}" - } - }{{ "," if not loop.last }} - {% endfor %} -] -``` - -**Key template elements:** - -- `queryset`: The filtered set of devices (all unless you add `.filter()`) -- `device.name`: Device hostname -- `device.primary_ip4.address.ip`: Primary IPv4 address -- `device.site.name`, `device.device_role.name`: NetBox relationships (site, role, etc.) -- `loop.last`: Jinja2 loop variable to avoid trailing comma on last item - -### Step 2c: Save and Access the Template - -Once saved, NetBox exposes the template via: - -``` -http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc+Device+Export -``` - -Or fetch it directly: - -```bash -# Replace with your NetBox URL and template name -# Substitute YOUR_NETBOX_API_TOKEN with your actual token -# Bearer Token Format (v2): nbt_. -curl -H "Authorization: Bearer YOUR_NETBOX_API_TOKEN" \ - "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" -``` - -The response should be a JSON array of targets ready for the gNMIc Operator. - -Sample JSON output produced by the basic export template: - -```json -[ - - { - "name": "edge-rtr-01.dc1.example.com", - "address": "203.0.113.1", - "labels": { - "site": "DC1", - "role": "edge", - "region": "eu-central-1", - "type": "router" - } - }, - -] -``` - -> Ensure the response is valid JSON and contains no hidden or invalid characters, otherwise the gNMIc Operator will fail to parse it. - -> If you instead return a JSON object with a nested array, add a mapping section such as `targetsField: "self.targets"` to the TargetSource CR. - ---- - -## Step 3: Create a TargetProfile - - - -Create a credentials Secret first, then reference it from the TargetProfile. - -```yaml -# Replace YOUR_DEVICE_USERNAME and YOUR_DEVICE_PASSWORD with your corresponding default device username and password -apiVersion: v1 -kind: Secret -metadata: - name: device-credentials - namespace: gnmic-system -type: Opaque -stringData: - username: YOUR_DEVICE_USERNAME - password: YOUR_DEVICE_PASSWORD -``` - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetProfile -metadata: - name: netbox-device - namespace: gnmic-system -spec: - credentialsRef: device-credentials - timeout: 10s -``` - -For more TargetProfile options and credential handling, see the operator documentation for `TargetProfile`. - ---- - -## Step 4: Create a TargetSource Using Export Template - -Create a `TargetSource` that references your NetBox export template endpoint: - -```yaml -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: netbox-export-source - namespace: gnmic-system -spec: - targetPort: 57400 - targetProfile: netbox-device - targetLabels: - inventory: netbox - sync-source: export-template - provider: - http: - url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" - method: GET - interval: 30m - timeout: 30s - authentication: - token: - scheme: Token - tokenSecretRef: - name: netbox-api-token - key: token -``` - ---- - -## Step 5: Verify Target Discovery - -Once the `TargetSource` is deployed, verify that targets are being discovered: - -```bash -# List discovered targets -kubectl get targets -n gnmic-system - -# Check TargetSource status and sync details -kubectl describe targetsource netbox-export-source -n gnmic-system -``` - -Successful sync shows: - -- `status.status`: "success" (or similar) -- `status.targetsCount`: number of devices -- `status.lastSync`: recent timestamp - ---- - -## Example: Complete Setup - -Here's a full example combining all components: - -```yaml ---- -# Secret for NetBox API token -apiVersion: v1 -kind: Secret -metadata: - name: netbox-api-token - namespace: gnmic-system -type: Opaque -data: - # base64-encoded token (echo -n "YOUR_TOKEN" | base64) - token: YOUR_BASE64_ENCODED_TOKEN - ---- -# Secret for Target Credential -apiVersion: v1 -kind: Secret -metadata: - name: device-credentials - namespace: gnmic-system -type: Opaque -stringData: - username: YOUR_DEVICE_USERNAME - password: YOUR_DEVICE_PASSWORD - ---- -# TargetProfile -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetProfile -metadata: - name: netbox-device - namespace: gnmic-system -spec: - credentialsRef: device-credentials - timeout: 10s - - ---- -# TargetSource using Export Template -apiVersion: operator.gnmic.dev/v1alpha1 -kind: TargetSource -metadata: - name: netbox-export-source - namespace: gnmic-system -spec: - targetPort: 57400 - targetProfile: netbox-device - targetLabels: - inventory: netbox - sync-source: export-template - provider: - http: - url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" - method: GET - interval: 30m - timeout: 30s - authentication: - token: - scheme: Token - tokenSecretRef: - name: netbox-api-token - key: token -``` - ---- - -## Advantages of Export Templates - -- **Powerful Filtering**: Filter devices by site, status, role, tags, etc. directly in NetBox -- **Reduced Operator Load**: NetBox handles data transformation; operator just fetches JSON -- **Reusability**: One template can serve multiple consumers -- **Maintainability**: Update discovery logic in NetBox without changing Kubernetes manifests -- **Performance**: Avoids REST API pagination for large inventories - ---- - -## Limitations & Considerations - -### 1. Reverse Proxy and URL Path Rewriting - -If NetBox is behind a reverse proxy with URL path rewriting: - -- **Issue**: The export template endpoint uses query parameters that may not survive proxy transformation. -- **Solution**: - - Ensure the proxy preserves query strings exactly. - - Test the export URL directly: - ```bash - curl -H "Authorization: Token YOUR_TOKEN" \ - "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" - ``` - - If the proxy blocks or modifies parameters, consider using a direct NetBox endpoint without proxying. - -### 2. Large Inventory Rendering - -- Very large device counts can cause NetBox to take time rendering the template. -- **Solution**: - - Use `.filter()` in your template to limit results. - - Create separate export templates for different device groups (e.g., by site or role). - -### 3. Complex Jinja2 Logic - -- NetBox's Jinja2 sandbox restricts some Python functions for security. -- **Solution**: Keep templates simple and use NetBox's built-in filters and objects. Test the URL with curl or similar before deploying. - ---- - -## Template Troubleshooting - -### Missing Targets in Kubernetes - -- **Check**: Are all required fields populated in NetBox? (e.g., `primary_ip4` may be `None` if not set) -- **Solution**: Add conditional checks: - ```jinja2 - {% if device.primary_ip4 %} - "address": "{{ device.primary_ip4.address.ip }}" - {% endif %} - ``` - -### Authorization Fails - -If you get a 403 error: - -- Verify the token is valid and not expired. -- Ensure the API token is enabled. - ---- From 3cd9666603697977612c6336ace1462e0eded375 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:08:09 +0000 Subject: [PATCH 43/76] add step 1 --- .../examples/NetBox/Export Template/_index.md | 54 +++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index 0d76f1f7..f8f58253 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -9,3 +9,57 @@ description: > This guide shows how to use **NetBox Export Templates** with the HTTP provider to discover and sync targets. Export Templates offer powerful filtering, transformation, and formatting directly in NetBox, reducing the load on the operator. + +## Overview + +An **Export Template** is a Jinja2 template defined in NetBox that: + +1. **Queries** NetBox's internal database (devices, interfaces, etc.) +2. **Filters** results based on custom criteria +3. **Transforms** data into your desired output format (JSON, YAML, CSV, etc.) +4. **Returns** the formatted output via a custom REST API endpoint + +When used with gNMIc's HTTP provider, the operator simply fetches the rendered JSON template and parses the result — no additional gNMIc Operator transformation needed if done correctly. + +--- + +## Prerequisites + +- A running Kubernetes cluster with gNMIc Operator installed +- `kubectl` access to your cluster +- A reachable NetBox instance with permissions to create Export Templates +- A NetBox API token +- Familiarity with Jinja2 templates + +--- + +## Step 1: Create a NetBox API Token and Store It Securely + +### Step 1a: Create the API Token in NetBox + +Create a dedicated API token in NetBox for gNMIc Operator access. + +1. Log in to NetBox. +2. Open your user profile or go to **User > API Tokens**. +3. Click **Add** or **Add token**. +4. Enter a descriptive name such as `gNMIc Operator`. +5. Grant the minimum permissions required for read-only device discovery. +6. Copy the token value and store it safely; NetBox will not show it again. + +### Step 1b: Store the Token in a Kubernetes Secret + +Create a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing the token. + +```bash +# Substitute YOUR_NETBOX_API_TOKEN with your actual token +# Bearer Token Format (v2): nbt_. +kubectl create secret generic netbox-api-token \ + --from-literal=token=YOUR_NETBOX_API_TOKEN \ + -n gnmic-system +``` + +Verify the Secret was created: + +```bash +kubectl get secret netbox-api-token -n gnmic-system -o yaml +``` From 9fb54c341139e093021dc96731eff2981e62fffc Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:10:13 +0000 Subject: [PATCH 44/76] add step 2 & 3 --- .../examples/NetBox/Export Template/_index.md | 86 +++++++++++++++++++ 1 file changed, 86 insertions(+) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index f8f58253..8e736ea5 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -63,3 +63,89 @@ Verify the Secret was created: ```bash kubectl get secret netbox-api-token -n gnmic-system -o yaml ``` + +--- + +## Step 2: Create a TargetProfile + +Define how discovered targets should be configured. The `TargetProfile` points to a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing device credentials, such as username/password or client certificates. + +Create a credentials Secret first, then reference it from the profile. + +```yaml +# Replace YOUR_DEVICE_USERNAME and YOUR_DEVICE_PASSWORD with your corresponding default device username and password +apiVersion: v1 +kind: Secret +metadata: + name: device-credentials + namespace: gnmic-system +type: Opaque +stringData: + username: YOUR_DEVICE_USERNAME + password: YOUR_DEVICE_PASSWORD +``` + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetProfile +metadata: + name: netbox-device + namespace: gnmic-system +spec: + credentialsRef: device-credentials + timeout: 10s +``` + +For more TargetProfile options and credential handling, see the operator documentation for `TargetProfile`. + +--- + +## Step 3: Create a TargetSource Using REST API + +The following `TargetSource` queries NetBox's REST API to discover devices: + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: netbox-rest-source + namespace: gnmic-system +spec: + targetPort: 57400 + targetProfile: netbox-device + targetLabels: + inventory: netbox + sync-source: rest-api + provider: + http: + url: "http://netbox.example.com:8000/api/dcim/devices/?limit=1000" + method: GET + interval: 5m + timeout: 30s + authentication: + token: + scheme: Bearer + tokenSecretRef: + name: netbox-api-token + key: token + pagination: + nextField: "next" + mapping: + targetsField: "self.results" + address: "item.primary_ip4 != null ? item.primary_ip4.address.split('/')[0] : ''" + labels: | + { + "site": item.site.name, + "role": item.device_role.name, + "model": item.device_type.model, + "status": item.status.value + } +``` + +> This mapping only works for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox API details, see the [NetBox REST API](https://netboxlabs.com/docs/netbox/integrations/rest-api/) documentation. + +The HTTP loader supports `targetsField` and individual CEL expressions for `name`, `address`, `port`, `labels`, and `targetProfile`. See the HTTP provider docs "Response Mapping via CEL" section for more details: {{< relref "user-guide/targetsource/providers/http.md" >}}. + +Use `self` for the full response and `item` for each candidate object. + +--- \ No newline at end of file From fcbd33970e1a8cc4db3f87350e2a9bd3b385eb79 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:11:13 +0000 Subject: [PATCH 45/76] remove relref --- docs/content/docs/examples/NetBox/Export Template/_index.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index 8e736ea5..648c1bc3 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -143,9 +143,3 @@ spec: ``` > This mapping only works for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox API details, see the [NetBox REST API](https://netboxlabs.com/docs/netbox/integrations/rest-api/) documentation. - -The HTTP loader supports `targetsField` and individual CEL expressions for `name`, `address`, `port`, `labels`, and `targetProfile`. See the HTTP provider docs "Response Mapping via CEL" section for more details: {{< relref "user-guide/targetsource/providers/http.md" >}}. - -Use `self` for the full response and `item` for each candidate object. - ---- \ No newline at end of file From 1dd827c998aaaf25510c82c091537c74cac31426 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:14:20 +0000 Subject: [PATCH 46/76] complete example without relrefs --- .../examples/NetBox/Export Template/_index.md | 218 ++++++++++++++++-- 1 file changed, 196 insertions(+), 22 deletions(-) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index 648c1bc3..b719597a 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -1,35 +1,29 @@ --- -title: "NetBox (Export Template)" -linkTitle: "NetBox Export Template" -weight: 1 +title: "NetBox (REST API)" +linkTitle: "NetBox REST" +weight: 2 description: > - Discover targets from NetBox using HTTP provider with NetBox Export Template + Discover targets from NetBox using the HTTP provider and NetBox REST API --- -This guide shows how to use **NetBox Export Templates** with the HTTP provider to discover and sync targets. +This guide shows how to configure the HTTP provider to discover targets from NetBox using its REST API. -Export Templates offer powerful filtering, transformation, and formatting directly in NetBox, reducing the load on the operator. - -## Overview - -An **Export Template** is a Jinja2 template defined in NetBox that: - -1. **Queries** NetBox's internal database (devices, interfaces, etc.) -2. **Filters** results based on custom criteria -3. **Transforms** data into your desired output format (JSON, YAML, CSV, etc.) -4. **Returns** the formatted output via a custom REST API endpoint - -When used with gNMIc's HTTP provider, the operator simply fetches the rendered JSON template and parses the result — no additional gNMIc Operator transformation needed if done correctly. - ---- +The REST API approach is direct and straightforward — query NetBox's standard API endpoints to retrieve devices that match your criteria. ## Prerequisites - A running Kubernetes cluster with gNMIc Operator installed - `kubectl` access to your cluster -- A reachable NetBox instance with permissions to create Export Templates +- A reachable NetBox instance (inside or outside the cluster) - A NetBox API token -- Familiarity with Jinja2 templates + +## Overview + +The HTTP `TargetSource` loader performs these steps: + +1. **Fetch** JSON device data from a NetBox REST API endpoint (`/api/dcim/devices/`) +2. **Transform** each device record into a gNMIc target using CEL expressions +3. **Create** or **update** `Target` resources in Kubernetes with the extracted data --- @@ -48,7 +42,7 @@ Create a dedicated API token in NetBox for gNMIc Operator access. ### Step 1b: Store the Token in a Kubernetes Secret -Create a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing the token. +Create a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing the token so it is not embedded in manifests. ```bash # Substitute YOUR_NETBOX_API_TOKEN with your actual token @@ -143,3 +137,183 @@ spec: ``` > This mapping only works for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox API details, see the [NetBox REST API](https://netboxlabs.com/docs/netbox/integrations/rest-api/) documentation. + +The HTTP loader supports `targetsField` and individual CEL expressions for `name`, `address`, `port`, `labels`, and `targetProfile`. See the HTTP provider docs "Response Mapping via CEL" section for more details:. + +Use `self` for the full response and `item` for each candidate object. + +--- + +## Step 4: Apply and Verify Target Discovery + +Deploy the `TargetSource` and check that targets are being discovered and synced: + +```bash +# List discovered targets +kubectl apply -f /path/to/targetsource.yaml -n gnmic-system + +# List discovered targets +kubectl get targets -n gnmic-system + +# Check TargetSource status +kubectl describe targetsource netbox-rest-source -n gnmic-system +``` + +Look for: +- `status.status`: "success" (or similar) +- `status.targetsCount`: number of discovered devices +- `status.lastSync`: recent timestamp + +--- + +## Example: Complete Setup + +Here's a complete example combining all resources: + +```yaml +--- +# Secret for NetBox API token +apiVersion: v1 +kind: Secret +metadata: + name: netbox-api-token + namespace: gnmic-system +type: Opaque +data: + # base64-encoded token (echo -n "YOUR_TOKEN" | base64) + token: YOUR_BASE64_ENCODED_TOKEN + +--- +# Secret for Target Credential +apiVersion: v1 +kind: Secret +metadata: + name: device-credentials + namespace: gnmic-system +type: Opaque +stringData: + username: YOUR_DEVICE_USERNAME + password: YOUR_DEVICE_PASSWORD + +--- +# TargetProfile +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetProfile +metadata: + name: netbox-device + namespace: gnmic-system +spec: + credentialsRef: device-credentials + timeout: 10s + +--- +# TargetSource with REST API +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: netbox-rest-source + namespace: gnmic-system +spec: + targetPort: 57400 + targetProfile: netbox-device + targetLabels: + inventory: netbox + sync-source: rest-api + provider: + http: + url: "http://netbox.example.com:8000/api/dcim/devices/?limit=1000" + method: GET + interval: 5m + timeout: 30s + authentication: + token: + scheme: Bearer + tokenSecretRef: + name: netbox-api-token + key: token + pagination: + nextField: "next" + mapping: + targetsField: "self.results" + address: "item.primary_ip4 != null ? item.primary_ip4.address.split('/')[0] : ''" + labels: | + { + "site": item.site.name, + "role": item.device_role.name, + "model": item.device_type.model, + "status": item.status.value + } +``` + +--- + +## Performance Considerations & Limitations + +### REST API Query Limits + +- **Query Size**: The example uses `limit=1000`. Adjust based on your NetBox instance's pagination settings and response size limits. +- **Response Timeout**: Large device lists can take time. Set appropriate timeouts in your `TargetSource`. + +### Reverse Proxy Considerations + +If NetBox is behind a reverse proxy: + +- **Base URL**: Ensure the reverse proxy correctly handles the `/api/dcim/devices/` path. +- **Authentication**: Some proxies may require additional headers; verify with your proxy and NetBox admin. +- **HTTPS**: If using HTTPS, ensure certificates are trusted by the operator or else use the `tls` setting. + +### Large Inventories + +For inventories with thousands of devices: + + +- Implement filtering in the REST API URL (e.g., `?site=us-west&status=active`). + +--- + +## Security Considerations + +### Token and Credentials + +- **Never** embed plaintext tokens or credentials in manifests or YAML files. +- Always store tokens in Kubernetes Secrets. +- Restrict RBAC permissions on the Secret to only necessary service accounts. + +### HTTPS and Certificates + +If connecting to NetBox via HTTPS: + +- Ensure cluster DNS resolves the hostname correctly. +- Mount CA certificates if using self-signed certificates. +- Verify the operator's HTTP client configuration for certificate validation. + +--- + +## Troubleshooting + +### Show TargetSource Errors + +```bash +kubectl describe targetsource netbox-rest-source -n gnmic-system +``` + +### Targets Not Appearing + +- Check that the `TargetProfile` exists and is correctly referenced. +- Verify labels and addresses are being extracted correctly from the NetBox response. +- Review operator logs for parsing errors: + ```bash + kubectl logs -l app=gnmic-operator -n gnmic-operator-system + ``` + +### Rate Limiting or Timeouts + +Increase the sync interval in your `TargetSource` or adjust timeouts: + +```yaml +spec: + provider: + http: + interval: 1h + timeout: 1m +``` From 0da0fb087a4402a4c02836b47173328328f88527 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:16:56 +0000 Subject: [PATCH 47/76] add relrefs --- docs/content/docs/examples/NetBox/Export Template/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index b719597a..2fe0c277 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -138,7 +138,7 @@ spec: > This mapping only works for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox API details, see the [NetBox REST API](https://netboxlabs.com/docs/netbox/integrations/rest-api/) documentation. -The HTTP loader supports `targetsField` and individual CEL expressions for `name`, `address`, `port`, `labels`, and `targetProfile`. See the HTTP provider docs "Response Mapping via CEL" section for more details:. +The HTTP loader supports `targetsField` and individual CEL expressions for `name`, `address`, `port`, `labels`, and `targetProfile`. See the HTTP provider docs "Response Mapping via CEL" section for more details: {{< relref "../../user-guide/targetsource/providers/http.md" >}}. Use `self` for the full response and `item` for each candidate object. @@ -266,7 +266,7 @@ If NetBox is behind a reverse proxy: For inventories with thousands of devices: - +- Consider using **Export Templates** (see [NetBox Export Templates]({{< relref "../Export Template" >}})) for better filtering and performance. - Implement filtering in the REST API URL (e.g., `?site=us-west&status=active`). --- From 20accaa260fef79486cf45e8f8da9d2eedfa2867 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:18:51 +0000 Subject: [PATCH 48/76] add one relref --- docs/content/docs/examples/NetBox/Export Template/_index.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index 2fe0c277..afefbe50 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -138,7 +138,6 @@ spec: > This mapping only works for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox API details, see the [NetBox REST API](https://netboxlabs.com/docs/netbox/integrations/rest-api/) documentation. -The HTTP loader supports `targetsField` and individual CEL expressions for `name`, `address`, `port`, `labels`, and `targetProfile`. See the HTTP provider docs "Response Mapping via CEL" section for more details: {{< relref "../../user-guide/targetsource/providers/http.md" >}}. Use `self` for the full response and `item` for each candidate object. From 47918d7068ab40217547ffd53ade5cddea8ad005 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:35:12 +0000 Subject: [PATCH 49/76] update relref --- docs/content/docs/examples/NetBox/Export Template/_index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index afefbe50..5ffbdfd4 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -138,6 +138,7 @@ spec: > This mapping only works for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox API details, see the [NetBox REST API](https://netboxlabs.com/docs/netbox/integrations/rest-api/) documentation. +The HTTP loader supports `targetsField` and individual CEL expressions for `name`, `address`, `port`, `labels`, and `targetProfile`. See the HTTP provider docs "Response Mapping via CEL" section for more details: {{< relref "user-guide/targetsource/providers/http.md" >}}. Use `self` for the full response and `item` for each candidate object. From e2d60000cee769e783f38b226f0541c8b24a60e2 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:37:01 +0000 Subject: [PATCH 50/76] replace relref --- docs/content/docs/examples/NetBox/Export Template/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index 5ffbdfd4..251d6b9a 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -138,7 +138,7 @@ spec: > This mapping only works for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox API details, see the [NetBox REST API](https://netboxlabs.com/docs/netbox/integrations/rest-api/) documentation. -The HTTP loader supports `targetsField` and individual CEL expressions for `name`, `address`, `port`, `labels`, and `targetProfile`. See the HTTP provider docs "Response Mapping via CEL" section for more details: {{< relref "user-guide/targetsource/providers/http.md" >}}. +The HTTP loader supports `targetsField` and individual CEL expressions for `name`, `address`, `port`, `labels`, and `targetProfile`. See the [HTTP provider docs](../../user-guide/targetsource/providers/http.md) "Response Mapping via CEL" section for more details. Use `self` for the full response and `item` for each candidate object. From 1d3f57799f3e50160923ba4dbc2dcb80b981cfe3 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Thu, 4 Jun 2026 14:41:35 +0000 Subject: [PATCH 51/76] add rest api examples --- .../examples/NetBox/Export Template/_index.md | 324 +++++++++++------- .../docs/examples/NetBox/REST API/_index.md | 319 +++++++++++++++++ 2 files changed, 521 insertions(+), 122 deletions(-) create mode 100644 docs/content/docs/examples/NetBox/REST API/_index.md diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index 251d6b9a..2cc48b3b 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -1,29 +1,35 @@ --- -title: "NetBox (REST API)" -linkTitle: "NetBox REST" -weight: 2 +title: "NetBox (Export Template)" +linkTitle: "NetBox Export Template" +weight: 1 description: > - Discover targets from NetBox using the HTTP provider and NetBox REST API + Discover targets from NetBox using HTTP provider with NetBox Export Template --- -This guide shows how to configure the HTTP provider to discover targets from NetBox using its REST API. +This guide shows how to use **NetBox Export Templates** with the HTTP provider to discover and sync targets. -The REST API approach is direct and straightforward — query NetBox's standard API endpoints to retrieve devices that match your criteria. +Export Templates offer powerful filtering, transformation, and formatting directly in NetBox, reducing the load on the operator. + +## Overview + +An **Export Template** is a Jinja2 template defined in NetBox that: + +1. **Queries** NetBox's internal database (devices, interfaces, etc.) +2. **Filters** results based on custom criteria +3. **Transforms** data into your desired output format (JSON, YAML, CSV, etc.) +4. **Returns** the formatted output via a custom REST API endpoint + +When used with gNMIc's HTTP provider, the operator simply fetches the rendered JSON template and parses the result — no additional gNMIc Operator transformation needed if done correctly. + +--- ## Prerequisites - A running Kubernetes cluster with gNMIc Operator installed - `kubectl` access to your cluster -- A reachable NetBox instance (inside or outside the cluster) +- A reachable NetBox instance with permissions to create Export Templates - A NetBox API token - -## Overview - -The HTTP `TargetSource` loader performs these steps: - -1. **Fetch** JSON device data from a NetBox REST API endpoint (`/api/dcim/devices/`) -2. **Transform** each device record into a gNMIc target using CEL expressions -3. **Create** or **update** `Target` resources in Kubernetes with the extracted data +- Familiarity with Jinja2 templates --- @@ -42,7 +48,7 @@ Create a dedicated API token in NetBox for gNMIc Operator access. ### Step 1b: Store the Token in a Kubernetes Secret -Create a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing the token so it is not embedded in manifests. +Create a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing the token. ```bash # Substitute YOUR_NETBOX_API_TOKEN with your actual token @@ -60,11 +66,126 @@ kubectl get secret netbox-api-token -n gnmic-system -o yaml --- -## Step 2: Create a TargetProfile +## Step 2: Create an Export Template in NetBox + +Log in to your NetBox instance and navigate to **Customization > Export Templates**. + +### Step 2a: Create a New Template + +Click **Add Export Template** and fill in the details: + +| Field | Value | Notes | +|-------|-------|-------| +| **Name** | `gNMIc Device Export` | Descriptive name for your template | +| **Content Type** | `dcim > device` | Export template applies to Device objects | +| **Template Code** | (see below) | Jinja2 template | +| **File Extension** | `json` | Output format | +| **Mime Type** | `application/json` | Correct MIME type for JSON | + +### Step 2b: Template Code Example + +The following Export Templates only work for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox data model details, see the [NetBox Devices Data Model](https://netboxlabs.com/docs/netbox/models/dcim/device/) documentation. + +See the HTTP provider's "Default Response Format" section for the expected JSON structure: [HTTP provider docs](../../user-guide/targetsource/providers/http.md) + +#### Basic Template (All Devices) + +```jinja2 +[ + {% for device in queryset %} + { + "name": "{{ device.name }}", + "address": "{{ device.primary_ip4.address.ip }}", + "labels": { + "site": "{{ device.site.name }}", + "role": "{{ device.role.name }}", + "region": "{{ device.site.region.name }}", + "type": "{{ device.device_type.model }}" + } + }{{ "," if not loop.last }} + {% endfor %} +] +``` + +#### Advanced Template (Filtered by Status and Role) + +```jinja2 +[ + {% for device in queryset.filter(status='active', role__name__in=['leaf', 'spine']) %} + { + "name": "{{ device.name }}", + "address": "{{ device.primary_ip4.address.ip }}", + "labels": { + "site": "{{ device.site.name }}", + "role": "{{ device.role.name }}", + "region": "{{ device.site.region.name }}", + "model": "{{ device.device_type.model }}", + "serial": "{{ device.serial }}", + "asset_tag": "{{ device.asset_tag }}" + } + }{{ "," if not loop.last }} + {% endfor %} +] +``` + +**Key template elements:** + +- `queryset`: The filtered set of devices (all unless you add `.filter()`) +- `device.name`: Device hostname +- `device.primary_ip4.address.ip`: Primary IPv4 address +- `device.site.name`, `device.device_role.name`: NetBox relationships (site, role, etc.) +- `loop.last`: Jinja2 loop variable to avoid trailing comma on last item + +### Step 2c: Save and Access the Template + +Once saved, NetBox exposes the template via: + +``` +http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc+Device+Export +``` + +Or fetch it directly: + +```bash +# Replace with your NetBox URL and template name +# Substitute YOUR_NETBOX_API_TOKEN with your actual token +# Bearer Token Format (v2): nbt_. +curl -H "Authorization: Bearer YOUR_NETBOX_API_TOKEN" \ + "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" +``` + +The response should be a JSON array of targets ready for the gNMIc Operator. + +Sample JSON output produced by the basic export template: + +```json +[ + + { + "name": "edge-rtr-01.dc1.example.com", + "address": "203.0.113.1", + "labels": { + "site": "DC1", + "role": "edge", + "region": "eu-central-1", + "type": "router" + } + }, + +] +``` + +> Ensure the response is valid JSON and contains no hidden or invalid characters, otherwise the gNMIc Operator will fail to parse it. + +> If you instead return a JSON object with a nested array, add a mapping section such as `targetsField: "self.targets"` to the TargetSource CR. + +--- + +## Step 3: Create a TargetProfile Define how discovered targets should be configured. The `TargetProfile` points to a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing device credentials, such as username/password or client certificates. -Create a credentials Secret first, then reference it from the profile. +Create a credentials Secret first, then reference it from the TargetProfile. ```yaml # Replace YOUR_DEVICE_USERNAME and YOUR_DEVICE_PASSWORD with your corresponding default device username and password @@ -94,81 +215,61 @@ For more TargetProfile options and credential handling, see the operator documen --- -## Step 3: Create a TargetSource Using REST API +## Step 4: Create a TargetSource Using Export Template -The following `TargetSource` queries NetBox's REST API to discover devices: +Create a `TargetSource` that references your NetBox export template endpoint: ```yaml apiVersion: operator.gnmic.dev/v1alpha1 kind: TargetSource metadata: - name: netbox-rest-source + name: netbox-export-source namespace: gnmic-system spec: targetPort: 57400 targetProfile: netbox-device targetLabels: inventory: netbox - sync-source: rest-api + sync-source: export-template provider: http: - url: "http://netbox.example.com:8000/api/dcim/devices/?limit=1000" + url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" method: GET - interval: 5m + interval: 30m timeout: 30s authentication: token: - scheme: Bearer + scheme: Token tokenSecretRef: name: netbox-api-token key: token - pagination: - nextField: "next" - mapping: - targetsField: "self.results" - address: "item.primary_ip4 != null ? item.primary_ip4.address.split('/')[0] : ''" - labels: | - { - "site": item.site.name, - "role": item.device_role.name, - "model": item.device_type.model, - "status": item.status.value - } ``` -> This mapping only works for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox API details, see the [NetBox REST API](https://netboxlabs.com/docs/netbox/integrations/rest-api/) documentation. - -The HTTP loader supports `targetsField` and individual CEL expressions for `name`, `address`, `port`, `labels`, and `targetProfile`. See the [HTTP provider docs](../../user-guide/targetsource/providers/http.md) "Response Mapping via CEL" section for more details. - -Use `self` for the full response and `item` for each candidate object. - --- -## Step 4: Apply and Verify Target Discovery +## Step 5: Verify Target Discovery -Deploy the `TargetSource` and check that targets are being discovered and synced: +Once the `TargetSource` is deployed, verify that targets are being discovered: ```bash -# List discovered targets -kubectl apply -f /path/to/targetsource.yaml -n gnmic-system - # List discovered targets kubectl get targets -n gnmic-system -# Check TargetSource status -kubectl describe targetsource netbox-rest-source -n gnmic-system +# Check TargetSource status and sync details +kubectl describe targetsource netbox-export-source -n gnmic-system ``` -Look for: +Successful sync shows: + - `status.status`: "success" (or similar) -- `status.targetsCount`: number of discovered devices +- `status.targetsCount`: number of devices - `status.lastSync`: recent timestamp --- ## Example: Complete Setup -Here's a complete example combining all resources: +Here's a full example combining all components: ```yaml --- @@ -206,114 +307,93 @@ spec: credentialsRef: device-credentials timeout: 10s + --- -# TargetSource with REST API +# TargetSource using Export Template apiVersion: operator.gnmic.dev/v1alpha1 kind: TargetSource metadata: - name: netbox-rest-source + name: netbox-export-source namespace: gnmic-system spec: targetPort: 57400 targetProfile: netbox-device targetLabels: inventory: netbox - sync-source: rest-api + sync-source: export-template provider: http: - url: "http://netbox.example.com:8000/api/dcim/devices/?limit=1000" + url: "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" method: GET - interval: 5m + interval: 30m timeout: 30s authentication: token: - scheme: Bearer + scheme: Token tokenSecretRef: name: netbox-api-token key: token - pagination: - nextField: "next" - mapping: - targetsField: "self.results" - address: "item.primary_ip4 != null ? item.primary_ip4.address.split('/')[0] : ''" - labels: | - { - "site": item.site.name, - "role": item.device_role.name, - "model": item.device_type.model, - "status": item.status.value - } ``` --- -## Performance Considerations & Limitations - -### REST API Query Limits +## Advantages of Export Templates -- **Query Size**: The example uses `limit=1000`. Adjust based on your NetBox instance's pagination settings and response size limits. -- **Response Timeout**: Large device lists can take time. Set appropriate timeouts in your `TargetSource`. +- **Powerful Filtering**: Filter devices by site, status, role, tags, etc. directly in NetBox +- **Reduced Operator Load**: NetBox handles data transformation; operator just fetches JSON +- **Reusability**: One template can serve multiple consumers +- **Maintainability**: Update discovery logic in NetBox without changing Kubernetes manifests +- **Performance**: Avoids REST API pagination for large inventories -### Reverse Proxy Considerations - -If NetBox is behind a reverse proxy: - -- **Base URL**: Ensure the reverse proxy correctly handles the `/api/dcim/devices/` path. -- **Authentication**: Some proxies may require additional headers; verify with your proxy and NetBox admin. -- **HTTPS**: If using HTTPS, ensure certificates are trusted by the operator or else use the `tls` setting. - -### Large Inventories +--- -For inventories with thousands of devices: +## Limitations & Considerations -- Consider using **Export Templates** (see [NetBox Export Templates]({{< relref "../Export Template" >}})) for better filtering and performance. -- Implement filtering in the REST API URL (e.g., `?site=us-west&status=active`). +### 1. Reverse Proxy and URL Path Rewriting ---- +If NetBox is behind a reverse proxy with URL path rewriting: -## Security Considerations +- **Issue**: The export template endpoint uses query parameters that may not survive proxy transformation. +- **Solution**: + - Ensure the proxy preserves query strings exactly. + - Test the export URL directly: + ```bash + curl -H "Authorization: Token YOUR_TOKEN" \ + "http://netbox.example.com:8000/api/dcim/devices/?export=gNMIc%20Device%20Export" + ``` + - If the proxy blocks or modifies parameters, consider using a direct NetBox endpoint without proxying. -### Token and Credentials +### 2. Large Inventory Rendering -- **Never** embed plaintext tokens or credentials in manifests or YAML files. -- Always store tokens in Kubernetes Secrets. -- Restrict RBAC permissions on the Secret to only necessary service accounts. +- Very large device counts can cause NetBox to take time rendering the template. +- **Solution**: + - Use `.filter()` in your template to limit results. + - Create separate export templates for different device groups (e.g., by site or role). -### HTTPS and Certificates +### 3. Complex Jinja2 Logic -If connecting to NetBox via HTTPS: - -- Ensure cluster DNS resolves the hostname correctly. -- Mount CA certificates if using self-signed certificates. -- Verify the operator's HTTP client configuration for certificate validation. +- NetBox's Jinja2 sandbox restricts some Python functions for security. +- **Solution**: Keep templates simple and use NetBox's built-in filters and objects. Test the URL with curl or similar before deploying. --- -## Troubleshooting - -### Show TargetSource Errors +## Template Troubleshooting -```bash -kubectl describe targetsource netbox-rest-source -n gnmic-system -``` +### Missing Targets in Kubernetes -### Targets Not Appearing - -- Check that the `TargetProfile` exists and is correctly referenced. -- Verify labels and addresses are being extracted correctly from the NetBox response. -- Review operator logs for parsing errors: - ```bash - kubectl logs -l app=gnmic-operator -n gnmic-operator-system +- **Check**: Are all required fields populated in NetBox? (e.g., `primary_ip4` may be `None` if not set) +- **Solution**: Add conditional checks: + ```jinja2 + {% if device.primary_ip4 %} + "address": "{{ device.primary_ip4.address.ip }}" + {% endif %} ``` -### Rate Limiting or Timeouts +### Authorization Fails -Increase the sync interval in your `TargetSource` or adjust timeouts: +If you get a 403 error: -```yaml -spec: - provider: - http: - interval: 1h - timeout: 1m -``` +- Verify the token is valid and not expired. +- Ensure the API token is enabled. + +--- diff --git a/docs/content/docs/examples/NetBox/REST API/_index.md b/docs/content/docs/examples/NetBox/REST API/_index.md new file mode 100644 index 00000000..18111d75 --- /dev/null +++ b/docs/content/docs/examples/NetBox/REST API/_index.md @@ -0,0 +1,319 @@ +--- +title: "NetBox (REST API)" +linkTitle: "NetBox REST" +weight: 2 +description: > + Discover targets from NetBox using the HTTP provider and NetBox REST API +--- + +This guide shows how to configure the HTTP provider to discover targets from NetBox using its REST API. + +The REST API approach is direct and straightforward — query NetBox's standard API endpoints to retrieve devices that match your criteria. + +## Prerequisites + +- A running Kubernetes cluster with gNMIc Operator installed +- `kubectl` access to your cluster +- A reachable NetBox instance (inside or outside the cluster) +- A NetBox API token + +## Overview + +The HTTP `TargetSource` loader performs these steps: + +1. **Fetch** JSON device data from a NetBox REST API endpoint (`/api/dcim/devices/`) +2. **Transform** each device record into a gNMIc target using CEL expressions +3. **Create** or **update** `Target` resources in Kubernetes with the extracted data + +--- + +## Step 1: Create a NetBox API Token and Store It Securely + +### Step 1a: Create the API Token in NetBox + +Create a dedicated API token in NetBox for gNMIc Operator access. + +1. Log in to NetBox. +2. Open your user profile or go to **User > API Tokens**. +3. Click **Add** or **Add token**. +4. Enter a descriptive name such as `gNMIc Operator`. +5. Grant the minimum permissions required for read-only device discovery. +6. Copy the token value and store it safely; NetBox will not show it again. + +### Step 1b: Store the Token in a Kubernetes Secret + +Create a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing the token so it is not embedded in manifests. + +```bash +# Substitute YOUR_NETBOX_API_TOKEN with your actual token +# Bearer Token Format (v2): nbt_. +kubectl create secret generic netbox-api-token \ + --from-literal=token=YOUR_NETBOX_API_TOKEN \ + -n gnmic-system +``` + +Verify the Secret was created: + +```bash +kubectl get secret netbox-api-token -n gnmic-system -o yaml +``` + +--- + +## Step 2: Create a TargetProfile + +Define how discovered targets should be configured. The `TargetProfile` points to a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) containing device credentials, such as username/password or client certificates. + +Create a credentials Secret first, then reference it from the profile. + +```yaml +# Replace YOUR_DEVICE_USERNAME and YOUR_DEVICE_PASSWORD with your corresponding default device username and password +apiVersion: v1 +kind: Secret +metadata: + name: device-credentials + namespace: gnmic-system +type: Opaque +stringData: + username: YOUR_DEVICE_USERNAME + password: YOUR_DEVICE_PASSWORD +``` + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetProfile +metadata: + name: netbox-device + namespace: gnmic-system +spec: + credentialsRef: device-credentials + timeout: 10s +``` + +For more TargetProfile options and credential handling, see the operator documentation for `TargetProfile`. + +--- + +## Step 3: Create a TargetSource Using REST API + +The following `TargetSource` queries NetBox's REST API to discover devices: + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: netbox-rest-source + namespace: gnmic-system +spec: + targetPort: 57400 + targetProfile: netbox-device + targetLabels: + inventory: netbox + sync-source: rest-api + provider: + http: + url: "http://netbox.example.com:8000/api/dcim/devices/?limit=1000" + method: GET + interval: 5m + timeout: 30s + authentication: + token: + scheme: Bearer + tokenSecretRef: + name: netbox-api-token + key: token + pagination: + nextField: "next" + mapping: + targetsField: "self.results" + address: "item.primary_ip4 != null ? item.primary_ip4.address.split('/')[0] : ''" + labels: | + { + "site": item.site.name, + "role": item.device_role.name, + "model": item.device_type.model, + "status": item.status.value + } +``` + +> This mapping only works for devices that have a primary IPv4 address set in NetBox. If primary_ip4 is missing, the expression returns '', so those devices will not yield a valid target address. For NetBox API details, see the [NetBox REST API](https://netboxlabs.com/docs/netbox/integrations/rest-api/) documentation. + +The HTTP loader supports `targetsField` and individual CEL expressions for `name`, `address`, `port`, `labels`, and `targetProfile`. See the HTTP Provider docs "Response Mapping via CEL" section for more details: [HTTP provider docs](../../user-guide/targetsource/providers/http.md) + +Use `self` for the full response and `item` for each candidate object. + +--- + +## Step 4: Apply and Verify Target Discovery + +Deploy the `TargetSource` and check that targets are being discovered and synced: + +```bash +# List discovered targets +kubectl apply -f /path/to/targetsource.yaml -n gnmic-system + +# List discovered targets +kubectl get targets -n gnmic-system + +# Check TargetSource status +kubectl describe targetsource netbox-rest-source -n gnmic-system +``` + +Look for: +- `status.status`: "success" (or similar) +- `status.targetsCount`: number of discovered devices +- `status.lastSync`: recent timestamp + +--- + +## Example: Complete Setup + +Here's a complete example combining all resources: + +```yaml +--- +# Secret for NetBox API token +apiVersion: v1 +kind: Secret +metadata: + name: netbox-api-token + namespace: gnmic-system +type: Opaque +data: + # base64-encoded token (echo -n "YOUR_TOKEN" | base64) + token: YOUR_BASE64_ENCODED_TOKEN + +--- +# Secret for Target Credential +apiVersion: v1 +kind: Secret +metadata: + name: device-credentials + namespace: gnmic-system +type: Opaque +stringData: + username: YOUR_DEVICE_USERNAME + password: YOUR_DEVICE_PASSWORD + +--- +# TargetProfile +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetProfile +metadata: + name: netbox-device + namespace: gnmic-system +spec: + credentialsRef: device-credentials + timeout: 10s + +--- +# TargetSource with REST API +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: netbox-rest-source + namespace: gnmic-system +spec: + targetPort: 57400 + targetProfile: netbox-device + targetLabels: + inventory: netbox + sync-source: rest-api + provider: + http: + url: "http://netbox.example.com:8000/api/dcim/devices/?limit=1000" + method: GET + interval: 5m + timeout: 30s + authentication: + token: + scheme: Bearer + tokenSecretRef: + name: netbox-api-token + key: token + pagination: + nextField: "next" + mapping: + targetsField: "self.results" + address: "item.primary_ip4 != null ? item.primary_ip4.address.split('/')[0] : ''" + labels: | + { + "site": item.site.name, + "role": item.device_role.name, + "model": item.device_type.model, + "status": item.status.value + } +``` + +--- + +## Performance Considerations & Limitations + +### REST API Query Limits + +- **Query Size**: The example uses `limit=1000`. Adjust based on your NetBox instance's pagination settings and response size limits. +- **Response Timeout**: Large device lists can take time. Set appropriate timeouts in your `TargetSource`. + +### Reverse Proxy Considerations + +If NetBox is behind a reverse proxy: + +- **Base URL**: Ensure the reverse proxy correctly handles the `/api/dcim/devices/` path. +- **Authentication**: Some proxies may require additional headers; verify with your proxy and NetBox admin. +- **HTTPS**: If using HTTPS, ensure certificates are trusted by the operator or else use the `tls` setting. + +### Large Inventories + +For inventories with thousands of devices: + +- Consider using **Export Templates** (see [NetBox Export Templates]({{< relref "../Export Template" >}})) for better filtering and performance. +- Implement filtering in the REST API URL (e.g., `?site=us-west&status=active`). + +--- + +## Security Considerations + +### Token and Credentials + +- **Never** embed plaintext tokens or credentials in manifests or YAML files. +- Always store tokens in Kubernetes Secrets. +- Restrict RBAC permissions on the Secret to only necessary service accounts. + +### HTTPS and Certificates + +If connecting to NetBox via HTTPS: + +- Ensure cluster DNS resolves the hostname correctly. +- Mount CA certificates if using self-signed certificates. +- Verify the operator's HTTP client configuration for certificate validation. + +--- + +## Troubleshooting + +### Show TargetSource Errors + +```bash +kubectl describe targetsource netbox-rest-source -n gnmic-system +``` + +### Targets Not Appearing + +- Check that the `TargetProfile` exists and is correctly referenced. +- Verify labels and addresses are being extracted correctly from the NetBox response. +- Review operator logs for parsing errors: + ```bash + kubectl logs -l app=gnmic-operator -n gnmic-operator-system + ``` + +### Rate Limiting or Timeouts + +Increase the sync interval in your `TargetSource` or adjust timeouts: + +```yaml +spec: + provider: + http: + interval: 1h + timeout: 1m +``` From 00555b4458bd81484a6a4787f41d78610aa1fccc Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Mon, 8 Jun 2026 11:31:35 +0200 Subject: [PATCH 52/76] generate docs --- .../docs/advanced/.openapi-generator-ignore | 23 ++++++ .../docs/advanced/.openapi-generator/FILES | 4 + .../docs/advanced/.openapi-generator/VERSION | 1 + docs/content/docs/advanced/Apis/DefaultApi.md | 57 +++++++++++++ docs/content/docs/advanced/Models/Target.md | 14 ++++ docs/content/docs/advanced/README.md | 27 ++++++ docs/content/docs/advanced/openapi.yaml | 82 +++++++++++++++++++ 7 files changed, 208 insertions(+) create mode 100644 docs/content/docs/advanced/.openapi-generator-ignore create mode 100644 docs/content/docs/advanced/.openapi-generator/FILES create mode 100644 docs/content/docs/advanced/.openapi-generator/VERSION create mode 100644 docs/content/docs/advanced/Apis/DefaultApi.md create mode 100644 docs/content/docs/advanced/Models/Target.md create mode 100644 docs/content/docs/advanced/README.md create mode 100644 docs/content/docs/advanced/openapi.yaml diff --git a/docs/content/docs/advanced/.openapi-generator-ignore b/docs/content/docs/advanced/.openapi-generator-ignore new file mode 100644 index 00000000..7484ee59 --- /dev/null +++ b/docs/content/docs/advanced/.openapi-generator-ignore @@ -0,0 +1,23 @@ +# OpenAPI Generator Ignore +# Generated by openapi-generator https://github.com/openapitools/openapi-generator + +# Use this file to prevent files from being overwritten by the generator. +# The patterns follow closely to .gitignore or .dockerignore. + +# As an example, the C# client generator defines ApiClient.cs. +# You can make changes and tell OpenAPI Generator to ignore just this file by uncommenting the following line: +#ApiClient.cs + +# You can match any string of characters against a directory, file or extension with a single asterisk (*): +#foo/*/qux +# The above matches foo/bar/qux and foo/baz/qux, but not foo/bar/baz/qux + +# You can recursively match patterns against a directory, file or extension with a double asterisk (**): +#foo/**/qux +# This matches foo/bar/qux, foo/baz/qux, and foo/bar/baz/qux + +# You can also negate patterns with an exclamation (!). +# For example, you can ignore all files in a docs folder with the file extension .md: +#docs/*.md +# Then explicitly reverse the ignore rule for a single file: +#!docs/README.md diff --git a/docs/content/docs/advanced/.openapi-generator/FILES b/docs/content/docs/advanced/.openapi-generator/FILES new file mode 100644 index 00000000..dd678403 --- /dev/null +++ b/docs/content/docs/advanced/.openapi-generator/FILES @@ -0,0 +1,4 @@ +.openapi-generator-ignore +Apis/DefaultApi.md +Models/Target.md +README.md diff --git a/docs/content/docs/advanced/.openapi-generator/VERSION b/docs/content/docs/advanced/.openapi-generator/VERSION new file mode 100644 index 00000000..ca7bf6e4 --- /dev/null +++ b/docs/content/docs/advanced/.openapi-generator/VERSION @@ -0,0 +1 @@ +7.23.0-SNAPSHOT diff --git a/docs/content/docs/advanced/Apis/DefaultApi.md b/docs/content/docs/advanced/Apis/DefaultApi.md new file mode 100644 index 00000000..a69288cd --- /dev/null +++ b/docs/content/docs/advanced/Apis/DefaultApi.md @@ -0,0 +1,57 @@ +# DefaultApi + +All URIs are relative to *http://localhost* + +| Method | HTTP request | Description | +|------------- | ------------- | -------------| +| [**applyTargets**](DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Targets received are applied in gNMIc Operator. | +| [**getClusterPlan**](DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | + + + +# **applyTargets** +> List applyTargets(Target) + +Targets received are applied in gNMIc Operator. + +### Parameters + +|Name | Type | Description | Notes | +|------------- | ------------- | ------------- | -------------| +| **Target** | [**List**](../Models/Target.md)| | | + +### Return type + +[**List**](../Models/Target.md) + +### Authorization + +[bearerAuth](../README.md#bearerAuth) + +### HTTP request headers + +- **Content-Type**: application/json +- **Accept**: application/json + + +# **getClusterPlan** +> getClusterPlan() + +Get cluster plan. + +### Parameters +This endpoint does not need any parameter. + +### Return type + +null (empty response body) + +### Authorization + +No authorization required + +### HTTP request headers + +- **Content-Type**: Not defined +- **Accept**: Not defined + diff --git a/docs/content/docs/advanced/Models/Target.md b/docs/content/docs/advanced/Models/Target.md new file mode 100644 index 00000000..5f784eba --- /dev/null +++ b/docs/content/docs/advanced/Models/Target.md @@ -0,0 +1,14 @@ +# Target +## Properties + +| Name | Type | Description | Notes | +|------------ | ------------- | ------------- | -------------| +| **name** | **String** | Routername | [default to null] | +| **address** | **String** | IPv4 or IPv6 | [optional] [default to null] | +| **port** | **Integer** | gNMIc port | [optional] [default to null] | +| **targetProfile** | **String** | TargetProfile applied to specific router | [optional] [default to null] | +| **labels** | [**List**](map.md) | Input of labels through key:value pair | [optional] [default to null] | +| **operation** | **String** | Either created, updated or deleted. created and updated internally is the same operation (apply) | [default to null] | + +[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md) + diff --git a/docs/content/docs/advanced/README.md b/docs/content/docs/advanced/README.md new file mode 100644 index 00000000..4a0ddd35 --- /dev/null +++ b/docs/content/docs/advanced/README.md @@ -0,0 +1,27 @@ +# Documentation for gNMIc Operator REST API + + +## Documentation for API Endpoints + +All URIs are relative to *http://localhost* + +| Class | Method | HTTP request | Description | +|------------ | ------------- | ------------- | -------------| +| *DefaultApi* | [**applyTargets**](Apis/DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Targets received are applied in gNMIc Operator. | +*DefaultApi* | [**getClusterPlan**](Apis/DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | + + + +## Documentation for Models + + - [Target](./Models/Target.md) + + + +## Documentation for Authorization + + +### bearerAuth + +- **Type**: HTTP Bearer Token authentication + diff --git a/docs/content/docs/advanced/openapi.yaml b/docs/content/docs/advanced/openapi.yaml new file mode 100644 index 00000000..b26e69de --- /dev/null +++ b/docs/content/docs/advanced/openapi.yaml @@ -0,0 +1,82 @@ +openapi: 3.0.3 +info: + title: "gNMIc Operator REST API" + version: "0.0.1" +paths: + /clusters/:namespace/:name/plan: + get: + summary: "Get cluster plan." + operationId: "getClusterPlan" + responses: + '200': + description: "ClusterPlan returned" + /api/v1/:namespace/target-source/:name/applyTargets: + post: + summary: "Targets received are applied in gNMIc Operator." + operationId: "applyTargets" + security: + - bearerAuth: [] + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/Targets' + responses: + '201': + description: "Targets applied successfully" + content: + application/json: + schema: + $ref: '#/components/schemas/Targets' + '401': + description: Access token is missing or invalid + +components: + schemas: + Targets: + type: array + items: + $ref: '#/components/schemas/Target' + Label: + description: TBD + type: object + additionalProperties: + description: Label must be passed as key:value pair. Multiple values per key possible + type: string + Target: + type: object + required: + - name + - ip + - operation + properties: + name: + type: string + description: Routername + address: + type: string + description: IPv4 or IPv6 + port: + type: integer + description: gNMIc port + targetProfile: + type: string + description: TargetProfile applied to specific router + labels: + type: array + description: Input of labels through key:value pair + items: + $ref: '#/components/schemas/Label' + operation: + type: string + enum: + - created + - updated + - deleted + description: Either created, updated or deleted. created and updated internally is the same operation (apply) + securitySchemes: + bearerAuth: + type: http + scheme: bearer + \ No newline at end of file From ad2b377b95b3a84c424c8745b2b2d7e360c6f2b4 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Mon, 8 Jun 2026 11:39:20 +0200 Subject: [PATCH 53/76] add header to generated md --- docs/content/docs/advanced/README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/content/docs/advanced/README.md b/docs/content/docs/advanced/README.md index 4a0ddd35..1985a6e5 100644 --- a/docs/content/docs/advanced/README.md +++ b/docs/content/docs/advanced/README.md @@ -1,3 +1,11 @@ +--- +title: "Documentation for gNMIc Operator REST API" +linkTitle: "REST API interface" +weight: 3 +description: > + Documentation of REST API interface according to openAPI standard. +--- + # Documentation for gNMIc Operator REST API From 3a0f5c8a13ab939b1042c050aca33898cf861b36 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Mon, 8 Jun 2026 13:34:08 +0200 Subject: [PATCH 54/76] tests with markdown templates --- docs/content/docs/advanced/Apis/DefaultApi.md | 6 +- docs/content/docs/advanced/Models/Target.md | 24 ++++--- .../advanced/openapi-generator-config.yaml | 7 ++ .../markdown-documentation/README.mustache | 67 +++++++++++++++++++ .../markdown-documentation/api.mustache | 42 ++++++++++++ .../markdown-documentation/model.mustache | 27 ++++++++ docs/content/docs/advanced/openapi.yaml | 19 +++--- .../docs/advanced/rest-api-documentation.md | 33 +++++++++ 8 files changed, 205 insertions(+), 20 deletions(-) create mode 100644 docs/content/docs/advanced/openapi-generator-config.yaml create mode 100644 docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache create mode 100644 docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache create mode 100644 docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache create mode 100644 docs/content/docs/advanced/rest-api-documentation.md diff --git a/docs/content/docs/advanced/Apis/DefaultApi.md b/docs/content/docs/advanced/Apis/DefaultApi.md index a69288cd..aeb9cdbd 100644 --- a/docs/content/docs/advanced/Apis/DefaultApi.md +++ b/docs/content/docs/advanced/Apis/DefaultApi.md @@ -4,7 +4,7 @@ All URIs are relative to *http://localhost* | Method | HTTP request | Description | |------------- | ------------- | -------------| -| [**applyTargets**](DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Targets received are applied in gNMIc Operator. | +| [**applyTargets**](DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | | [**getClusterPlan**](DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | @@ -12,7 +12,7 @@ All URIs are relative to *http://localhost* # **applyTargets** > List applyTargets(Target) -Targets received are applied in gNMIc Operator. +Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. ### Parameters @@ -26,7 +26,7 @@ Targets received are applied in gNMIc Operator. ### Authorization -[bearerAuth](../README.md#bearerAuth) +[bearerAuth](../rest-api-documentation.md#bearerAuth) ### HTTP request headers diff --git a/docs/content/docs/advanced/Models/Target.md b/docs/content/docs/advanced/Models/Target.md index 5f784eba..f3e5de65 100644 --- a/docs/content/docs/advanced/Models/Target.md +++ b/docs/content/docs/advanced/Models/Target.md @@ -1,14 +1,22 @@ +--- +title: "Model" +linkTitle: "Model" +weight: 4 +description: > + Todo +--- + # Target ## Properties | Name | Type | Description | Notes | |------------ | ------------- | ------------- | -------------| -| **name** | **String** | Routername | [default to null] | -| **address** | **String** | IPv4 or IPv6 | [optional] [default to null] | -| **port** | **Integer** | gNMIc port | [optional] [default to null] | -| **targetProfile** | **String** | TargetProfile applied to specific router | [optional] [default to null] | -| **labels** | [**List**](map.md) | Input of labels through key:value pair | [optional] [default to null] | -| **operation** | **String** | Either created, updated or deleted. created and updated internally is the same operation (apply) | [default to null] | - -[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md) +| **name** | **String** | Name of device to be monitored. | [default to null] | +| **address** | **String** | IPv4/IPv6 address or hostname. | [default to null] | +| **port** | **Integer** | gNMIc port. | [optional] [default to null] | +| **targetProfile** | **String** | TargetProfile applied to apply to this router. | [optional] [default to null] | +| **labels** | [**List**](map.md) | Input of labels as key:value pair. | [optional] [default to null] | +| **operation** | **String** | Either `created`, `updated` or `deleted`. `created` and `updated` are identical and both apply the target. | [default to null] | + +[[Back to Model list]](../rest-api-documentation.md#documentation-for-models) [[Back to API list]](../rest-api-documentation.md#documentation-for-api-endpoints) [[Back to README]](../rest-api-documentation.md) diff --git a/docs/content/docs/advanced/openapi-generator-config.yaml b/docs/content/docs/advanced/openapi-generator-config.yaml new file mode 100644 index 00000000..20fa9d53 --- /dev/null +++ b/docs/content/docs/advanced/openapi-generator-config.yaml @@ -0,0 +1,7 @@ +generatorName: markdown +inputSpec: /local/docs/content/docs/advanced/openapi.yaml +outputDir: /local/docs/content/docs/advanced +templateDir: /local/docs/content/docs/advanced/openapi-templates/markdown-documentation +files: + README.mustache: + destinationFilename: rest-api-documentation.md \ No newline at end of file diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache new file mode 100644 index 00000000..f231ec03 --- /dev/null +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache @@ -0,0 +1,67 @@ +--- +title: "Documentation for gNMIc Operator REST API" +linkTitle: "REST API interface" +weight: 3 +description: > + Documentation of REST API interface according to openAPI standard. +--- + +{{#generateApiDocs}} + +## Documentation for API Endpoints + +All URIs are relative to *{{{basePath}}}* + +| Class | Method | HTTP request | Description | +|------------ | ------------- | ------------- | -------------| +{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{classname}}* | [**{{operationId}}**](Apis/{{apiDocPath}}{{classname}}.md#{{operationId}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} | +{{/operation}}{{/operations}}{{/apis}}{{/apiInfo}} +{{/generateApiDocs}} + +{{#generateModelDocs}} + +## Documentation for Models + +{{#modelPackage}} +{{#models}}{{#model}} - [{{{classname}}}](./{{{modelPackage}}}/{{modelDocPath}}{{{classFilename}}}.md) +{{/model}}{{/models}} +{{/modelPackage}} +{{^modelPackage}} +No model defined in this package +{{/modelPackage}} +{{/generateModelDocs}} + +{{! TODO: optional documentation for authorization? }} +## Documentation for Authorization + +{{^authMethods}} +All endpoints do not require authorization. +{{/authMethods}} +{{#authMethods}} +{{#last}} + Authentication schemes defined for the API: +{{/last}} +{{/authMethods}} +{{#authMethods}} + +### {{name}} + +{{#isApiKey}}- **Type**: API key +- **API key parameter name**: {{keyParamName}} +- **Location**: {{#isKeyInQuery}}URL query string{{/isKeyInQuery}}{{#isKeyInHeader}}HTTP header{{/isKeyInHeader}} +{{/isApiKey}} +{{#isBasicBasic}}- **Type**: HTTP basic authentication +{{/isBasicBasic}} +{{#isBasicBearer}}- **Type**: HTTP Bearer Token authentication{{#bearerFormat}} ({{{.}}}){{/bearerFormat}} +{{/isBasicBearer}} +{{#isHttpSignature}}- **Type**: HTTP signature authentication +{{/isHttpSignature}} +{{#isOAuth}}- **Type**: OAuth +- **Flow**: {{flow}} +- **Authorization URL**: {{authorizationUrl}} +- **Scopes**: {{^scopes}}N/A{{/scopes}} +{{#scopes}} - {{scope}}: {{description}} +{{/scopes}} +{{/isOAuth}} + +{{/authMethods}} \ No newline at end of file diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache new file mode 100644 index 00000000..cbb8d19e --- /dev/null +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache @@ -0,0 +1,42 @@ +# {{classname}}{{#description}} + {{.}}{{/description}} + +All URIs are relative to *{{basePath}}* + +| Method | HTTP request | Description | +|------------- | ------------- | -------------| +{{#operations}}{{#operation}}| [**{{operationId}}**]({{classname}}.md#{{operationId}}) | **{{httpMethod}}** {{path}} | {{summary}} | +{{/operation}}{{/operations}} + +{{#operations}} +{{#operation}} + +# **{{operationId}}** +> {{#returnType}}{{.}} {{/returnType}}{{operationId}}({{#allParams}}{{{paramName}}}{{^-last}}, {{/-last}}{{/allParams}}) + +{{summary}}{{#notes}} + + {{.}}{{/notes}} + +### Parameters +{{^allParams}}This endpoint does not need any parameter.{{/allParams}}{{#allParams}}{{#-last}} +|Name | Type | Description | Notes | +|------------- | ------------- | ------------- | -------------|{{/-last}}{{/allParams}} +{{#allParams}}| **{{paramName}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}{{#isFile}}**{{dataType}}**{{/isFile}}{{^isFile}}{{#generateModelDocs}}[**{{dataType}}**](../{{modelPackage}}/{{baseType}}.md){{/generateModelDocs}}{{^generateModelDocs}}**{{dataType}}**{{/generateModelDocs}}{{/isFile}}{{/isPrimitiveType}}| {{description}} |{{^required}} [optional]{{/required}}{{#defaultValue}} [default to {{.}}]{{/defaultValue}}{{#allowableValues}} [enum: {{#values}}{{{.}}}{{^-last}}, {{/-last}}{{/values}}]{{/allowableValues}} | +{{/allParams}} + +### Return type + +{{#returnType}}{{#returnTypeIsPrimitive}}**{{returnType}}**{{/returnTypeIsPrimitive}}{{^returnTypeIsPrimitive}}{{#generateModelDocs}}[**{{returnType}}**](../{{modelPackage}}/{{returnBaseType}}.md){{/generateModelDocs}}{{^generateModelDocs}}**{{returnType}}**{{/generateModelDocs}}{{/returnTypeIsPrimitive}}{{/returnType}}{{^returnType}}null (empty response body){{/returnType}} + +### Authorization + +{{^authMethods}}No authorization required{{/authMethods}}{{#authMethods}}[{{name}}](../rest-api-documentation.md#{{name}}){{^-last}}, {{/-last}}{{/authMethods}} + +### HTTP request headers + +- **Content-Type**: {{#consumes}}{{{mediaType}}}{{^-last}}, {{/-last}}{{/consumes}}{{^consumes}}Not defined{{/consumes}} +- **Accept**: {{#produces}}{{{mediaType}}}{{^-last}}, {{/-last}}{{/produces}}{{^produces}}Not defined{{/produces}} + +{{/operation}} +{{/operations}} \ No newline at end of file diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache new file mode 100644 index 00000000..a5b5e851 --- /dev/null +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache @@ -0,0 +1,27 @@ +--- +title: "Model" +linkTitle: "Model" +weight: 4 +description: > + Todo +--- + +{{#models}} +{{#model}} +# {{{classname}}} +## Properties + +| Name | Type | Description | Notes | +|------------ | ------------- | ------------- | -------------| +{{#parent}} +{{#parentVars}} +| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}[**{{dataType}}**]({{complexType}}.md){{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} | +{{/parentVars}} +{{/parent}} +{{#vars}}| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}[**{{dataType}}**]({{complexType}}.md){{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} | +{{/vars}} + +[[Back to Model list]](../rest-api-documentation.md#documentation-for-models) [[Back to API list]](../rest-api-documentation.md#documentation-for-api-endpoints) [[Back to README]](../rest-api-documentation.md) + + {{/model}} +{{/models}} \ No newline at end of file diff --git a/docs/content/docs/advanced/openapi.yaml b/docs/content/docs/advanced/openapi.yaml index b26e69de..3883f465 100644 --- a/docs/content/docs/advanced/openapi.yaml +++ b/docs/content/docs/advanced/openapi.yaml @@ -12,7 +12,7 @@ paths: description: "ClusterPlan returned" /api/v1/:namespace/target-source/:name/applyTargets: post: - summary: "Targets received are applied in gNMIc Operator." + summary: "Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator." operationId: "applyTargets" security: - bearerAuth: [] @@ -42,30 +42,31 @@ components: description: TBD type: object additionalProperties: - description: Label must be passed as key:value pair. Multiple values per key possible + description: Label must be passed as key:value pair, multiple values per key possible. type: string Target: + description: Network device to be monitored. Properties not marked as optional must be in JSON body. type: object required: - name - - ip + - address - operation properties: name: type: string - description: Routername + description: Name of device to be monitored. address: type: string - description: IPv4 or IPv6 + description: IPv4/IPv6 address or hostname. port: type: integer - description: gNMIc port + description: gNMIc port. targetProfile: type: string - description: TargetProfile applied to specific router + description: TargetProfile applied to apply to this router. labels: type: array - description: Input of labels through key:value pair + description: Input of labels as key:value pair. items: $ref: '#/components/schemas/Label' operation: @@ -74,7 +75,7 @@ components: - created - updated - deleted - description: Either created, updated or deleted. created and updated internally is the same operation (apply) + description: "Either `created`, `updated` or `deleted`. `created` and `updated` are identical and both apply the target." securitySchemes: bearerAuth: type: http diff --git a/docs/content/docs/advanced/rest-api-documentation.md b/docs/content/docs/advanced/rest-api-documentation.md new file mode 100644 index 00000000..ddc20bfb --- /dev/null +++ b/docs/content/docs/advanced/rest-api-documentation.md @@ -0,0 +1,33 @@ +--- +title: "Documentation for gNMIc Operator REST API" +linkTitle: "REST API interface" +weight: 3 +description: > + Documentation of REST API interface according to openAPI standard. +--- + + +## Documentation for API Endpoints + +All URIs are relative to *http://localhost* + +| Class | Method | HTTP request | Description | +|------------ | ------------- | ------------- | -------------| +| *DefaultApi* | [**applyTargets**](Apis/DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | +*DefaultApi* | [**getClusterPlan**](Apis/DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | + + + +## Documentation for Models + + - [Target](./Models/Target.md) + + + +## Documentation for Authorization + + +### bearerAuth + +- **Type**: HTTP Bearer Token authentication + From 131d838a8f39ac337f52cedc513a9530f0baf91a Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Mon, 8 Jun 2026 13:41:03 +0200 Subject: [PATCH 55/76] fix links --- docs/content/docs/advanced/README.md | 35 ------------------- .../markdown-documentation/README.mustache | 4 +-- .../docs/advanced/rest-api-documentation.md | 6 ++-- 3 files changed, 5 insertions(+), 40 deletions(-) delete mode 100644 docs/content/docs/advanced/README.md diff --git a/docs/content/docs/advanced/README.md b/docs/content/docs/advanced/README.md deleted file mode 100644 index 1985a6e5..00000000 --- a/docs/content/docs/advanced/README.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -title: "Documentation for gNMIc Operator REST API" -linkTitle: "REST API interface" -weight: 3 -description: > - Documentation of REST API interface according to openAPI standard. ---- - -# Documentation for gNMIc Operator REST API - - -## Documentation for API Endpoints - -All URIs are relative to *http://localhost* - -| Class | Method | HTTP request | Description | -|------------ | ------------- | ------------- | -------------| -| *DefaultApi* | [**applyTargets**](Apis/DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Targets received are applied in gNMIc Operator. | -*DefaultApi* | [**getClusterPlan**](Apis/DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | - - - -## Documentation for Models - - - [Target](./Models/Target.md) - - - -## Documentation for Authorization - - -### bearerAuth - -- **Type**: HTTP Bearer Token authentication - diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache index f231ec03..70619729 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache @@ -14,7 +14,7 @@ All URIs are relative to *{{{basePath}}}* | Class | Method | HTTP request | Description | |------------ | ------------- | ------------- | -------------| -{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{classname}}* | [**{{operationId}}**](Apis/{{apiDocPath}}{{classname}}.md#{{operationId}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} | +{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{classname}}* | [**{{operationId}}**](../Apis/{{apiDocPath}}{{classname}}.md#{{operationId}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} | {{/operation}}{{/operations}}{{/apis}}{{/apiInfo}} {{/generateApiDocs}} @@ -23,7 +23,7 @@ All URIs are relative to *{{{basePath}}}* ## Documentation for Models {{#modelPackage}} -{{#models}}{{#model}} - [{{{classname}}}](./{{{modelPackage}}}/{{modelDocPath}}{{{classFilename}}}.md) +{{#models}}{{#model}} - [{{{classname}}}](../{{{modelPackage}}}/{{modelDocPath}}{{{classFilename}}}.md) {{/model}}{{/models}} {{/modelPackage}} {{^modelPackage}} diff --git a/docs/content/docs/advanced/rest-api-documentation.md b/docs/content/docs/advanced/rest-api-documentation.md index ddc20bfb..edba35f0 100644 --- a/docs/content/docs/advanced/rest-api-documentation.md +++ b/docs/content/docs/advanced/rest-api-documentation.md @@ -13,14 +13,14 @@ All URIs are relative to *http://localhost* | Class | Method | HTTP request | Description | |------------ | ------------- | ------------- | -------------| -| *DefaultApi* | [**applyTargets**](Apis/DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | -*DefaultApi* | [**getClusterPlan**](Apis/DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | +| *DefaultApi* | [**applyTargets**](../Apis/DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | +*DefaultApi* | [**getClusterPlan**](../Apis/DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | ## Documentation for Models - - [Target](./Models/Target.md) + - [Target](../Models/Target.md) From a7c632111091f6476f69cbaef7e87dc12b87e1cf Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Mon, 8 Jun 2026 13:56:54 +0200 Subject: [PATCH 56/76] udpate folder structure --- .gitignore | 4 ++- .../docs/advanced/.openapi-generator/FILES | 4 --- .../docs/advanced/.openapi-generator/VERSION | 1 - .../advanced/openapi-generator-config.yaml | 4 +-- .../markdown-documentation/README.mustache | 4 +-- .../markdown-documentation/api.mustache | 10 +++++- .../markdown-documentation/model.mustache | 2 +- .../Apis/DefaultApi.md | 10 +++++- .../Models/Target.md | 2 +- .../advanced/rest-api-documentation/_index.md | 33 +++++++++++++++++++ 10 files changed, 60 insertions(+), 14 deletions(-) delete mode 100644 docs/content/docs/advanced/.openapi-generator/FILES delete mode 100644 docs/content/docs/advanced/.openapi-generator/VERSION rename docs/content/docs/advanced/{ => rest-api-documentation}/Apis/DefaultApi.md (87%) rename docs/content/docs/advanced/{ => rest-api-documentation}/Models/Target.md (79%) create mode 100644 docs/content/docs/advanced/rest-api-documentation/_index.md diff --git a/.gitignore b/.gitignore index 29d31af3..743dab8a 100644 --- a/.gitignore +++ b/.gitignore @@ -33,4 +33,6 @@ notes/ docs/public docs/resources/_gen/ docs/.hugo_build.lock -test/integration/clab-* \ No newline at end of file +test/integration/clab-* +docs/content/docs/advanced/rest-api-documentation/.openapi-generator +docs/content/docs/advanced/rest-api-documentation/.openapi-generator-ignore diff --git a/docs/content/docs/advanced/.openapi-generator/FILES b/docs/content/docs/advanced/.openapi-generator/FILES deleted file mode 100644 index dd678403..00000000 --- a/docs/content/docs/advanced/.openapi-generator/FILES +++ /dev/null @@ -1,4 +0,0 @@ -.openapi-generator-ignore -Apis/DefaultApi.md -Models/Target.md -README.md diff --git a/docs/content/docs/advanced/.openapi-generator/VERSION b/docs/content/docs/advanced/.openapi-generator/VERSION deleted file mode 100644 index ca7bf6e4..00000000 --- a/docs/content/docs/advanced/.openapi-generator/VERSION +++ /dev/null @@ -1 +0,0 @@ -7.23.0-SNAPSHOT diff --git a/docs/content/docs/advanced/openapi-generator-config.yaml b/docs/content/docs/advanced/openapi-generator-config.yaml index 20fa9d53..9ab8481d 100644 --- a/docs/content/docs/advanced/openapi-generator-config.yaml +++ b/docs/content/docs/advanced/openapi-generator-config.yaml @@ -1,7 +1,7 @@ generatorName: markdown inputSpec: /local/docs/content/docs/advanced/openapi.yaml -outputDir: /local/docs/content/docs/advanced +outputDir: /local/docs/content/docs/advanced/rest-api-documentation templateDir: /local/docs/content/docs/advanced/openapi-templates/markdown-documentation files: README.mustache: - destinationFilename: rest-api-documentation.md \ No newline at end of file + destinationFilename: _index.md \ No newline at end of file diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache index 70619729..f231ec03 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache @@ -14,7 +14,7 @@ All URIs are relative to *{{{basePath}}}* | Class | Method | HTTP request | Description | |------------ | ------------- | ------------- | -------------| -{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{classname}}* | [**{{operationId}}**](../Apis/{{apiDocPath}}{{classname}}.md#{{operationId}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} | +{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{classname}}* | [**{{operationId}}**](Apis/{{apiDocPath}}{{classname}}.md#{{operationId}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} | {{/operation}}{{/operations}}{{/apis}}{{/apiInfo}} {{/generateApiDocs}} @@ -23,7 +23,7 @@ All URIs are relative to *{{{basePath}}}* ## Documentation for Models {{#modelPackage}} -{{#models}}{{#model}} - [{{{classname}}}](../{{{modelPackage}}}/{{modelDocPath}}{{{classFilename}}}.md) +{{#models}}{{#model}} - [{{{classname}}}](./{{{modelPackage}}}/{{modelDocPath}}{{{classFilename}}}.md) {{/model}}{{/models}} {{/modelPackage}} {{^modelPackage}} diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache index cbb8d19e..f8b29ef5 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache @@ -1,3 +1,11 @@ +--- +title: "REST API definition" +linkTitle: "REST API definition" +weight: 4 +description: > + Gives REST API definition with available options. +--- + # {{classname}}{{#description}} {{.}}{{/description}} @@ -31,7 +39,7 @@ All URIs are relative to *{{basePath}}* ### Authorization -{{^authMethods}}No authorization required{{/authMethods}}{{#authMethods}}[{{name}}](../rest-api-documentation.md#{{name}}){{^-last}}, {{/-last}}{{/authMethods}} +{{^authMethods}}No authorization required{{/authMethods}}{{#authMethods}}[{{name}}](../_index.md#{{name}}){{^-last}}, {{/-last}}{{/authMethods}} ### HTTP request headers diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache index a5b5e851..844cde03 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache @@ -21,7 +21,7 @@ description: > {{#vars}}| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}[**{{dataType}}**]({{complexType}}.md){{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} | {{/vars}} -[[Back to Model list]](../rest-api-documentation.md#documentation-for-models) [[Back to API list]](../rest-api-documentation.md#documentation-for-api-endpoints) [[Back to README]](../rest-api-documentation.md) +[[Back to Model list]](../_index.md#documentation-for-models) [[Back to API list]](../_index.md#documentation-for-api-endpoints) [[Back to README]](../_index.md) {{/model}} {{/models}} \ No newline at end of file diff --git a/docs/content/docs/advanced/Apis/DefaultApi.md b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md similarity index 87% rename from docs/content/docs/advanced/Apis/DefaultApi.md rename to docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md index aeb9cdbd..2432fc73 100644 --- a/docs/content/docs/advanced/Apis/DefaultApi.md +++ b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md @@ -1,3 +1,11 @@ +--- +title: "REST API definition" +linkTitle: "REST API definition" +weight: 4 +description: > + Gives REST API definition with available options. +--- + # DefaultApi All URIs are relative to *http://localhost* @@ -26,7 +34,7 @@ Interface for real-time target updates, usually using a webhook. Targets are app ### Authorization -[bearerAuth](../rest-api-documentation.md#bearerAuth) +[bearerAuth](../_index.md#bearerAuth) ### HTTP request headers diff --git a/docs/content/docs/advanced/Models/Target.md b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md similarity index 79% rename from docs/content/docs/advanced/Models/Target.md rename to docs/content/docs/advanced/rest-api-documentation/Models/Target.md index f3e5de65..106dea2c 100644 --- a/docs/content/docs/advanced/Models/Target.md +++ b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md @@ -18,5 +18,5 @@ description: > | **labels** | [**List**](map.md) | Input of labels as key:value pair. | [optional] [default to null] | | **operation** | **String** | Either `created`, `updated` or `deleted`. `created` and `updated` are identical and both apply the target. | [default to null] | -[[Back to Model list]](../rest-api-documentation.md#documentation-for-models) [[Back to API list]](../rest-api-documentation.md#documentation-for-api-endpoints) [[Back to README]](../rest-api-documentation.md) +[[Back to Model list]](../_index.md#documentation-for-models) [[Back to API list]](../_index.md#documentation-for-api-endpoints) [[Back to README]](../_index.md) diff --git a/docs/content/docs/advanced/rest-api-documentation/_index.md b/docs/content/docs/advanced/rest-api-documentation/_index.md new file mode 100644 index 00000000..ddc20bfb --- /dev/null +++ b/docs/content/docs/advanced/rest-api-documentation/_index.md @@ -0,0 +1,33 @@ +--- +title: "Documentation for gNMIc Operator REST API" +linkTitle: "REST API interface" +weight: 3 +description: > + Documentation of REST API interface according to openAPI standard. +--- + + +## Documentation for API Endpoints + +All URIs are relative to *http://localhost* + +| Class | Method | HTTP request | Description | +|------------ | ------------- | ------------- | -------------| +| *DefaultApi* | [**applyTargets**](Apis/DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | +*DefaultApi* | [**getClusterPlan**](Apis/DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | + + + +## Documentation for Models + + - [Target](./Models/Target.md) + + + +## Documentation for Authorization + + +### bearerAuth + +- **Type**: HTTP Bearer Token authentication + From 7aec4c2c0cf5afb6031646e751141f56e04925c2 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Mon, 8 Jun 2026 14:10:31 +0200 Subject: [PATCH 57/76] fix links, cleanup some text --- .../markdown-documentation/README.mustache | 10 +++++----- .../markdown-documentation/api.mustache | 6 +++--- .../markdown-documentation/model.mustache | 8 ++++++-- .../rest-api-documentation/Apis/DefaultApi.md | 6 +++--- .../advanced/rest-api-documentation/Models/Target.md | 6 ++++-- .../docs/advanced/rest-api-documentation/_index.md | 12 ++++++------ 6 files changed, 27 insertions(+), 21 deletions(-) diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache index f231ec03..6ecbdb00 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache @@ -1,20 +1,20 @@ --- -title: "Documentation for gNMIc Operator REST API" +title: "REST API interface" linkTitle: "REST API interface" weight: 3 description: > - Documentation of REST API interface according to openAPI standard. + The gNMIc Operator has a REST API endpoint. This documentation explains what routes are avaiable and how to use them. --- {{#generateApiDocs}} ## Documentation for API Endpoints -All URIs are relative to *{{{basePath}}}* +All URIs are relative to *{{{basePath}}}:8082* | Class | Method | HTTP request | Description | |------------ | ------------- | ------------- | -------------| -{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{classname}}* | [**{{operationId}}**](Apis/{{apiDocPath}}{{classname}}.md#{{operationId}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} | +{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{classname}}* | [**{{operationId}}**](/docs/advanced/rest-api-documentation/Apis/{{apiDocPath}}{{classname}}/#{{operationId}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} | {{/operation}}{{/operations}}{{/apis}}{{/apiInfo}} {{/generateApiDocs}} @@ -23,7 +23,7 @@ All URIs are relative to *{{{basePath}}}* ## Documentation for Models {{#modelPackage}} -{{#models}}{{#model}} - [{{{classname}}}](./{{{modelPackage}}}/{{modelDocPath}}{{{classFilename}}}.md) +{{#models}}{{#model}} - [{{{classname}}}](/docs/advanced/rest-api-documentation/{{{modelPackage}}}/{{modelDocPath}}{{{classFilename}}}/) {{/model}}{{/models}} {{/modelPackage}} {{^modelPackage}} diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache index f8b29ef5..9ac12fd6 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache @@ -30,16 +30,16 @@ All URIs are relative to *{{basePath}}* {{^allParams}}This endpoint does not need any parameter.{{/allParams}}{{#allParams}}{{#-last}} |Name | Type | Description | Notes | |------------- | ------------- | ------------- | -------------|{{/-last}}{{/allParams}} -{{#allParams}}| **{{paramName}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}{{#isFile}}**{{dataType}}**{{/isFile}}{{^isFile}}{{#generateModelDocs}}[**{{dataType}}**](../{{modelPackage}}/{{baseType}}.md){{/generateModelDocs}}{{^generateModelDocs}}**{{dataType}}**{{/generateModelDocs}}{{/isFile}}{{/isPrimitiveType}}| {{description}} |{{^required}} [optional]{{/required}}{{#defaultValue}} [default to {{.}}]{{/defaultValue}}{{#allowableValues}} [enum: {{#values}}{{{.}}}{{^-last}}, {{/-last}}{{/values}}]{{/allowableValues}} | +{{#allParams}}| **{{paramName}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}{{#isFile}}**{{dataType}}**{{/isFile}}{{^isFile}}{{#generateModelDocs}}[**{{dataType}}**](/docs/advanced/rest-api-documentation/{{modelPackage}}/{{baseType}}/){{/generateModelDocs}}{{^generateModelDocs}}**{{dataType}}**{{/generateModelDocs}}{{/isFile}}{{/isPrimitiveType}}| {{description}} |{{^required}} [optional]{{/required}}{{#defaultValue}} [default to {{.}}]{{/defaultValue}}{{#allowableValues}} [enum: {{#values}}{{{.}}}{{^-last}}, {{/-last}}{{/values}}]{{/allowableValues}} | {{/allParams}} ### Return type -{{#returnType}}{{#returnTypeIsPrimitive}}**{{returnType}}**{{/returnTypeIsPrimitive}}{{^returnTypeIsPrimitive}}{{#generateModelDocs}}[**{{returnType}}**](../{{modelPackage}}/{{returnBaseType}}.md){{/generateModelDocs}}{{^generateModelDocs}}**{{returnType}}**{{/generateModelDocs}}{{/returnTypeIsPrimitive}}{{/returnType}}{{^returnType}}null (empty response body){{/returnType}} +{{#returnType}}{{#returnTypeIsPrimitive}}**{{returnType}}**{{/returnTypeIsPrimitive}}{{^returnTypeIsPrimitive}}{{#generateModelDocs}}[**{{returnType}}**](/docs/advanced/rest-api-documentation/{{modelPackage}}/{{returnBaseType}}/){{/generateModelDocs}}{{^generateModelDocs}}**{{returnType}}**{{/generateModelDocs}}{{/returnTypeIsPrimitive}}{{/returnType}}{{^returnType}}null (empty response body){{/returnType}} ### Authorization -{{^authMethods}}No authorization required{{/authMethods}}{{#authMethods}}[{{name}}](../_index.md#{{name}}){{^-last}}, {{/-last}}{{/authMethods}} +{{^authMethods}}No authorization required{{/authMethods}}{{#authMethods}}[{{name}}](/docs/advanced/rest-api-documentation/#{{name}}){{^-last}}, {{/-last}}{{/authMethods}} ### HTTP request headers diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache index 844cde03..d9a711c5 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache @@ -3,12 +3,16 @@ title: "Model" linkTitle: "Model" weight: 4 description: > - Todo + Documentation for OpenAPI models and their schema-defined properties. --- {{#models}} {{#model}} # {{{classname}}} +{{#description}} +{{{description}}} +{{/description}} + ## Properties | Name | Type | Description | Notes | @@ -21,7 +25,7 @@ description: > {{#vars}}| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}[**{{dataType}}**]({{complexType}}.md){{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} | {{/vars}} -[[Back to Model list]](../_index.md#documentation-for-models) [[Back to API list]](../_index.md#documentation-for-api-endpoints) [[Back to README]](../_index.md) +[[Back to Model list]](/docs/advanced/rest-api-documentation/#documentation-for-models) [[Back to API list]](/docs/advanced/rest-api-documentation/#documentation-for-api-endpoints) [[Back to README]](/docs/advanced/rest-api-documentation/) {{/model}} {{/models}} \ No newline at end of file diff --git a/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md index 2432fc73..bf27be18 100644 --- a/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md +++ b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md @@ -26,15 +26,15 @@ Interface for real-time target updates, usually using a webhook. Targets are app |Name | Type | Description | Notes | |------------- | ------------- | ------------- | -------------| -| **Target** | [**List**](../Models/Target.md)| | | +| **Target** | [**List**](/docs/advanced/rest-api-documentation/Models/Target/)| | | ### Return type -[**List**](../Models/Target.md) +[**List**](/docs/advanced/rest-api-documentation/Models/Target/) ### Authorization -[bearerAuth](../_index.md#bearerAuth) +[bearerAuth](/docs/advanced/rest-api-documentation/#bearerAuth) ### HTTP request headers diff --git a/docs/content/docs/advanced/rest-api-documentation/Models/Target.md b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md index 106dea2c..4292d45c 100644 --- a/docs/content/docs/advanced/rest-api-documentation/Models/Target.md +++ b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md @@ -3,10 +3,12 @@ title: "Model" linkTitle: "Model" weight: 4 description: > - Todo + Documentation for OpenAPI models and their schema-defined properties. --- # Target +Network device to be monitored. Properties not marked as optional must be in JSON body. + ## Properties | Name | Type | Description | Notes | @@ -18,5 +20,5 @@ description: > | **labels** | [**List**](map.md) | Input of labels as key:value pair. | [optional] [default to null] | | **operation** | **String** | Either `created`, `updated` or `deleted`. `created` and `updated` are identical and both apply the target. | [default to null] | -[[Back to Model list]](../_index.md#documentation-for-models) [[Back to API list]](../_index.md#documentation-for-api-endpoints) [[Back to README]](../_index.md) +[[Back to Model list]](/docs/advanced/rest-api-documentation/#documentation-for-models) [[Back to API list]](/docs/advanced/rest-api-documentation/#documentation-for-api-endpoints) [[Back to README]](/docs/advanced/rest-api-documentation/) diff --git a/docs/content/docs/advanced/rest-api-documentation/_index.md b/docs/content/docs/advanced/rest-api-documentation/_index.md index ddc20bfb..379f851c 100644 --- a/docs/content/docs/advanced/rest-api-documentation/_index.md +++ b/docs/content/docs/advanced/rest-api-documentation/_index.md @@ -1,26 +1,26 @@ --- -title: "Documentation for gNMIc Operator REST API" +title: "REST API interface" linkTitle: "REST API interface" weight: 3 description: > - Documentation of REST API interface according to openAPI standard. + The gNMIc Operator has a REST API endpoint. This documentation explains what routes are avaiable and how to use them. --- ## Documentation for API Endpoints -All URIs are relative to *http://localhost* +All URIs are relative to *http://localhost:8082* | Class | Method | HTTP request | Description | |------------ | ------------- | ------------- | -------------| -| *DefaultApi* | [**applyTargets**](Apis/DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | -*DefaultApi* | [**getClusterPlan**](Apis/DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | +| *DefaultApi* | [**applyTargets**](/docs/advanced/rest-api-documentation/Apis/DefaultApi/#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | +*DefaultApi* | [**getClusterPlan**](/docs/advanced/rest-api-documentation/Apis/DefaultApi/#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | ## Documentation for Models - - [Target](./Models/Target.md) + - [Target](/docs/advanced/rest-api-documentation/Models/Target/) From 7732cb4db8422363e388c6de3fd6b1129d2f7b8a Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Mon, 8 Jun 2026 14:19:47 +0200 Subject: [PATCH 58/76] update inline md links --- .../markdown-documentation/README.mustache | 4 ++-- .../openapi-templates/markdown-documentation/api.mustache | 8 ++++---- .../advanced/rest-api-documentation/Apis/DefaultApi.md | 8 ++++---- .../docs/advanced/rest-api-documentation/_index.md | 6 +++--- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache index 6ecbdb00..38ee13ca 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache @@ -3,7 +3,7 @@ title: "REST API interface" linkTitle: "REST API interface" weight: 3 description: > - The gNMIc Operator has a REST API endpoint. This documentation explains what routes are avaiable and how to use them. + The gNMIc Operator has a REST API endpoint. This documentation explains what routes are available and how to use them. --- {{#generateApiDocs}} @@ -14,7 +14,7 @@ All URIs are relative to *{{{basePath}}}:8082* | Class | Method | HTTP request | Description | |------------ | ------------- | ------------- | -------------| -{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{classname}}* | [**{{operationId}}**](/docs/advanced/rest-api-documentation/Apis/{{apiDocPath}}{{classname}}/#{{operationId}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} | +{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{classname}}* | [**{{operationId}}**](/docs/advanced/rest-api-documentation/{{apiDocPath}}{{classname}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} | {{/operation}}{{/operations}}{{/apis}}{{/apiInfo}} {{/generateApiDocs}} diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache index 9ac12fd6..0935152a 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache @@ -1,15 +1,15 @@ --- -title: "REST API definition" -linkTitle: "REST API definition" +title: "Routes" +linkTitle: "Routes" weight: 4 description: > - Gives REST API definition with available options. + Available HTTP routes on the gNMIc Operator API interface. --- # {{classname}}{{#description}} {{.}}{{/description}} -All URIs are relative to *{{basePath}}* +All URIs are relative to *{{basePath}}:8082* | Method | HTTP request | Description | |------------- | ------------- | -------------| diff --git a/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md index bf27be18..5b43e25c 100644 --- a/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md +++ b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md @@ -1,14 +1,14 @@ --- -title: "REST API definition" -linkTitle: "REST API definition" +title: "Routes" +linkTitle: "Routes" weight: 4 description: > - Gives REST API definition with available options. + Available HTTP routes on the gNMIc Operator API interface. --- # DefaultApi -All URIs are relative to *http://localhost* +All URIs are relative to *http://localhost:8082* | Method | HTTP request | Description | |------------- | ------------- | -------------| diff --git a/docs/content/docs/advanced/rest-api-documentation/_index.md b/docs/content/docs/advanced/rest-api-documentation/_index.md index 379f851c..b47a39d6 100644 --- a/docs/content/docs/advanced/rest-api-documentation/_index.md +++ b/docs/content/docs/advanced/rest-api-documentation/_index.md @@ -3,7 +3,7 @@ title: "REST API interface" linkTitle: "REST API interface" weight: 3 description: > - The gNMIc Operator has a REST API endpoint. This documentation explains what routes are avaiable and how to use them. + The gNMIc Operator has a REST API endpoint. This documentation explains what routes are available and how to use them. --- @@ -13,8 +13,8 @@ All URIs are relative to *http://localhost:8082* | Class | Method | HTTP request | Description | |------------ | ------------- | ------------- | -------------| -| *DefaultApi* | [**applyTargets**](/docs/advanced/rest-api-documentation/Apis/DefaultApi/#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | -*DefaultApi* | [**getClusterPlan**](/docs/advanced/rest-api-documentation/Apis/DefaultApi/#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | +| *DefaultApi* | [**applyTargets**](/docs/advanced/rest-api-documentation/DefaultApi) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | +*DefaultApi* | [**getClusterPlan**](/docs/advanced/rest-api-documentation/DefaultApi) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | From 6d6c7209e24095de42cd7e9fef5df78ec1449d68 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Mon, 8 Jun 2026 14:41:22 +0200 Subject: [PATCH 59/76] add signature --- docs/content/docs/Apis/DefaultApi.md | 2 ++ .../markdown-documentation/README.mustache | 4 ++-- docs/content/docs/advanced/openapi.yaml | 7 +++++++ .../rest-api-documentation/Apis/DefaultApi.md | 2 +- .../docs/advanced/rest-api-documentation/_index.md | 13 ++++++++++--- 5 files changed, 22 insertions(+), 6 deletions(-) create mode 100644 docs/content/docs/Apis/DefaultApi.md diff --git a/docs/content/docs/Apis/DefaultApi.md b/docs/content/docs/Apis/DefaultApi.md new file mode 100644 index 00000000..513f6da6 --- /dev/null +++ b/docs/content/docs/Apis/DefaultApi.md @@ -0,0 +1,2 @@ +# DefaultApi + diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache index 38ee13ca..70ef54a6 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache @@ -3,7 +3,7 @@ title: "REST API interface" linkTitle: "REST API interface" weight: 3 description: > - The gNMIc Operator has a REST API endpoint. This documentation explains what routes are available and how to use them. + This document describes the REST API exposed by the gNMIc Operator, including the available endpoints, request formats, and usage examples. --- {{#generateApiDocs}} @@ -14,7 +14,7 @@ All URIs are relative to *{{{basePath}}}:8082* | Class | Method | HTTP request | Description | |------------ | ------------- | ------------- | -------------| -{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{classname}}* | [**{{operationId}}**](/docs/advanced/rest-api-documentation/{{apiDocPath}}{{classname}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} | +{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{classname}}* | [**{{operationId}}**](/docs/advanced/rest-api-documentation/apis/{{apiDocPath}}{{classname}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} | {{/operation}}{{/operations}}{{/apis}}{{/apiInfo}} {{/generateApiDocs}} diff --git a/docs/content/docs/advanced/openapi.yaml b/docs/content/docs/advanced/openapi.yaml index 3883f465..5f6ea35b 100644 --- a/docs/content/docs/advanced/openapi.yaml +++ b/docs/content/docs/advanced/openapi.yaml @@ -16,6 +16,7 @@ paths: operationId: "applyTargets" security: - bearerAuth: [] + signature: [] requestBody: required: true content: @@ -80,4 +81,10 @@ components: bearerAuth: type: http scheme: bearer + description: HTTP authentication using Bearer token + signature: + name: X-Hook-Signature + type: apiKey + in: header + description: HMAC signature of the request payload \ No newline at end of file diff --git a/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md index 5b43e25c..e0d65613 100644 --- a/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md +++ b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md @@ -34,7 +34,7 @@ Interface for real-time target updates, usually using a webhook. Targets are app ### Authorization -[bearerAuth](/docs/advanced/rest-api-documentation/#bearerAuth) +[signature](/docs/advanced/rest-api-documentation/#signature), [bearerAuth](/docs/advanced/rest-api-documentation/#bearerAuth) ### HTTP request headers diff --git a/docs/content/docs/advanced/rest-api-documentation/_index.md b/docs/content/docs/advanced/rest-api-documentation/_index.md index b47a39d6..a47cb911 100644 --- a/docs/content/docs/advanced/rest-api-documentation/_index.md +++ b/docs/content/docs/advanced/rest-api-documentation/_index.md @@ -3,7 +3,7 @@ title: "REST API interface" linkTitle: "REST API interface" weight: 3 description: > - The gNMIc Operator has a REST API endpoint. This documentation explains what routes are available and how to use them. + This document describes the REST API exposed by the gNMIc Operator, including the available endpoints, request formats, and usage examples. --- @@ -13,8 +13,8 @@ All URIs are relative to *http://localhost:8082* | Class | Method | HTTP request | Description | |------------ | ------------- | ------------- | -------------| -| *DefaultApi* | [**applyTargets**](/docs/advanced/rest-api-documentation/DefaultApi) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | -*DefaultApi* | [**getClusterPlan**](/docs/advanced/rest-api-documentation/DefaultApi) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | +| *DefaultApi* | [**applyTargets**](/docs/advanced/rest-api-documentation/apis/DefaultApi) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | +*DefaultApi* | [**getClusterPlan**](/docs/advanced/rest-api-documentation/apis/DefaultApi) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | @@ -31,3 +31,10 @@ All URIs are relative to *http://localhost:8082* - **Type**: HTTP Bearer Token authentication + +### signature + +- **Type**: API key +- **API key parameter name**: X-Hook-Signature +- **Location**: HTTP header + From 04244041f31841970a3401a29d897cb36af1c3c9 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Mon, 8 Jun 2026 15:00:39 +0200 Subject: [PATCH 60/76] paths again and small corrections --- .../markdown-documentation/README.mustache | 4 ++-- .../markdown-documentation/api.mustache | 8 ++++---- .../markdown-documentation/model.mustache | 6 +++--- docs/content/docs/advanced/openapi.yaml | 1 + .../advanced/rest-api-documentation/Apis/DefaultApi.md | 10 +++++----- .../advanced/rest-api-documentation/Models/Target.md | 2 +- .../docs/advanced/rest-api-documentation/_index.md | 6 +++--- 7 files changed, 19 insertions(+), 18 deletions(-) diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache index 70ef54a6..9bf976ad 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache @@ -14,7 +14,7 @@ All URIs are relative to *{{{basePath}}}:8082* | Class | Method | HTTP request | Description | |------------ | ------------- | ------------- | -------------| -{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{classname}}* | [**{{operationId}}**](/docs/advanced/rest-api-documentation/apis/{{apiDocPath}}{{classname}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} | +{{#apiInfo}}{{#apis}}{{#operations}}| {{#operation}}*{{#lambda.lowercase}}{{classname}}{{/lambda.lowercase}}* | [**{{operationId}}**](/docs/advanced/rest-api-documentation/apis/{{#lambda.lowercase}}{{classname}}{{/lambda.lowercase}}) | **{{httpMethod}}** {{path}} | {{{summary}}}{{^summary}}{{{notes}}}{{/summary}} | {{/operation}}{{/operations}}{{/apis}}{{/apiInfo}} {{/generateApiDocs}} @@ -23,7 +23,7 @@ All URIs are relative to *{{{basePath}}}:8082* ## Documentation for Models {{#modelPackage}} -{{#models}}{{#model}} - [{{{classname}}}](/docs/advanced/rest-api-documentation/{{{modelPackage}}}/{{modelDocPath}}{{{classFilename}}}/) +{{#models}}{{#model}} - [{{#lambda.lowercase}}{{{classname}}}{{/lambda.lowercase}}](/docs/advanced/rest-api-documentation/models/{{#lambda.lowercase}}{{{classFilename}}}{{/lambda.lowercase}}/) {{/model}}{{/models}} {{/modelPackage}} {{^modelPackage}} diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache index 0935152a..5fb1b8e1 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache @@ -6,14 +6,14 @@ description: > Available HTTP routes on the gNMIc Operator API interface. --- -# {{classname}}{{#description}} +# {{#lambda.lowercase}}{{classname}}{{/lambda.lowercase}}{{#description}} {{.}}{{/description}} All URIs are relative to *{{basePath}}:8082* | Method | HTTP request | Description | |------------- | ------------- | -------------| -{{#operations}}{{#operation}}| [**{{operationId}}**]({{classname}}.md#{{operationId}}) | **{{httpMethod}}** {{path}} | {{summary}} | +{{#operations}}{{#operation}}| [**{{operationId}}**]({{#lambda.lowercase}}{{classname}}{{/lambda.lowercase}}.md#{{operationId}}) | **{{httpMethod}}** {{path}} | {{summary}} | {{/operation}}{{/operations}} {{#operations}} @@ -30,12 +30,12 @@ All URIs are relative to *{{basePath}}:8082* {{^allParams}}This endpoint does not need any parameter.{{/allParams}}{{#allParams}}{{#-last}} |Name | Type | Description | Notes | |------------- | ------------- | ------------- | -------------|{{/-last}}{{/allParams}} -{{#allParams}}| **{{paramName}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}{{#isFile}}**{{dataType}}**{{/isFile}}{{^isFile}}{{#generateModelDocs}}[**{{dataType}}**](/docs/advanced/rest-api-documentation/{{modelPackage}}/{{baseType}}/){{/generateModelDocs}}{{^generateModelDocs}}**{{dataType}}**{{/generateModelDocs}}{{/isFile}}{{/isPrimitiveType}}| {{description}} |{{^required}} [optional]{{/required}}{{#defaultValue}} [default to {{.}}]{{/defaultValue}}{{#allowableValues}} [enum: {{#values}}{{{.}}}{{^-last}}, {{/-last}}{{/values}}]{{/allowableValues}} | +{{#allParams}}| **{{paramName}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}{{#isFile}}**{{dataType}}**{{/isFile}}{{^isFile}}{{#generateModelDocs}}[**{{dataType}}**](/docs/advanced/rest-api-documentation/models/{{#lambda.lowercase}}{{baseType}}{{/lambda.lowercase}}/){{/generateModelDocs}}{{^generateModelDocs}}**{{dataType}}**{{/generateModelDocs}}{{/isFile}}{{/isPrimitiveType}}| {{description}} |{{^required}} [optional]{{/required}}{{#defaultValue}} [default to {{.}}]{{/defaultValue}}{{#allowableValues}} [enum: {{#values}}{{{.}}}{{^-last}}, {{/-last}}{{/values}}]{{/allowableValues}} | {{/allParams}} ### Return type -{{#returnType}}{{#returnTypeIsPrimitive}}**{{returnType}}**{{/returnTypeIsPrimitive}}{{^returnTypeIsPrimitive}}{{#generateModelDocs}}[**{{returnType}}**](/docs/advanced/rest-api-documentation/{{modelPackage}}/{{returnBaseType}}/){{/generateModelDocs}}{{^generateModelDocs}}**{{returnType}}**{{/generateModelDocs}}{{/returnTypeIsPrimitive}}{{/returnType}}{{^returnType}}null (empty response body){{/returnType}} +{{#returnType}}{{#returnTypeIsPrimitive}}**{{returnType}}**{{/returnTypeIsPrimitive}}{{^returnTypeIsPrimitive}}{{#generateModelDocs}}[**{{returnType}}**](/docs/advanced/rest-api-documentation/models/{{#lambda.lowercase}}{{returnBaseType}}{{/lambda.lowercase}}/){{/generateModelDocs}}{{^generateModelDocs}}**{{returnType}}**{{/generateModelDocs}}{{/returnTypeIsPrimitive}}{{/returnType}}{{^returnType}}null (empty response body){{/returnType}} ### Authorization diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache index d9a711c5..1905a91e 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache @@ -8,7 +8,7 @@ description: > {{#models}} {{#model}} -# {{{classname}}} +# {{#lambda.lowercase}}{{{classname}}}{{/lambda.lowercase}} {{#description}} {{{description}}} {{/description}} @@ -19,10 +19,10 @@ description: > |------------ | ------------- | ------------- | -------------| {{#parent}} {{#parentVars}} -| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}[**{{dataType}}**]({{complexType}}.md){{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} | +| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}[**{{dataType}}**]({{#lambda.lowercase}}{{complexType}}{{/lambda.lowercase}}.md){{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} | {{/parentVars}} {{/parent}} -{{#vars}}| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}[**{{dataType}}**]({{complexType}}.md){{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} | +{{#vars}}| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}[**{{dataType}}**]({{#lambda.lowercase}}{{complexType}}{{/lambda.lowercase}}.md){{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} | {{/vars}} [[Back to Model list]](/docs/advanced/rest-api-documentation/#documentation-for-models) [[Back to API list]](/docs/advanced/rest-api-documentation/#documentation-for-api-endpoints) [[Back to README]](/docs/advanced/rest-api-documentation/) diff --git a/docs/content/docs/advanced/openapi.yaml b/docs/content/docs/advanced/openapi.yaml index 5f6ea35b..7dfbc24b 100644 --- a/docs/content/docs/advanced/openapi.yaml +++ b/docs/content/docs/advanced/openapi.yaml @@ -19,6 +19,7 @@ paths: signature: [] requestBody: required: true + description: Target must be passed as a list, multiple targets possible. content: application/json: schema: diff --git a/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md index e0d65613..2415c558 100644 --- a/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md +++ b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md @@ -6,14 +6,14 @@ description: > Available HTTP routes on the gNMIc Operator API interface. --- -# DefaultApi +# defaultapi All URIs are relative to *http://localhost:8082* | Method | HTTP request | Description | |------------- | ------------- | -------------| -| [**applyTargets**](DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | -| [**getClusterPlan**](DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | +| [**applyTargets**](defaultapi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | +| [**getClusterPlan**](defaultapi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | @@ -26,11 +26,11 @@ Interface for real-time target updates, usually using a webhook. Targets are app |Name | Type | Description | Notes | |------------- | ------------- | ------------- | -------------| -| **Target** | [**List**](/docs/advanced/rest-api-documentation/Models/Target/)| | | +| **Target** | [**List**](/docs/advanced/rest-api-documentation/models/target/)| Target must be passed as a list, multiple targets possible. | | ### Return type -[**List**](/docs/advanced/rest-api-documentation/Models/Target/) +[**List**](/docs/advanced/rest-api-documentation/models/target/) ### Authorization diff --git a/docs/content/docs/advanced/rest-api-documentation/Models/Target.md b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md index 4292d45c..17967253 100644 --- a/docs/content/docs/advanced/rest-api-documentation/Models/Target.md +++ b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md @@ -6,7 +6,7 @@ description: > Documentation for OpenAPI models and their schema-defined properties. --- -# Target +# target Network device to be monitored. Properties not marked as optional must be in JSON body. ## Properties diff --git a/docs/content/docs/advanced/rest-api-documentation/_index.md b/docs/content/docs/advanced/rest-api-documentation/_index.md index a47cb911..a0ad838b 100644 --- a/docs/content/docs/advanced/rest-api-documentation/_index.md +++ b/docs/content/docs/advanced/rest-api-documentation/_index.md @@ -13,14 +13,14 @@ All URIs are relative to *http://localhost:8082* | Class | Method | HTTP request | Description | |------------ | ------------- | ------------- | -------------| -| *DefaultApi* | [**applyTargets**](/docs/advanced/rest-api-documentation/apis/DefaultApi) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | -*DefaultApi* | [**getClusterPlan**](/docs/advanced/rest-api-documentation/apis/DefaultApi) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | +| *defaultapi* | [**applyTargets**](/docs/advanced/rest-api-documentation/apis/defaultapi) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | +*defaultapi* | [**getClusterPlan**](/docs/advanced/rest-api-documentation/apis/defaultapi) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | ## Documentation for Models - - [Target](/docs/advanced/rest-api-documentation/Models/Target/) + - [target](/docs/advanced/rest-api-documentation/models/target/) From ede85b6aeda3bde6f7dfcb864240e2add432d3c6 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Mon, 8 Jun 2026 16:00:20 +0200 Subject: [PATCH 61/76] remove #section hyperlink --- .../openapi-templates/markdown-documentation/api.mustache | 2 +- .../openapi-templates/markdown-documentation/model.mustache | 2 -- .../docs/advanced/rest-api-documentation/Apis/DefaultApi.md | 4 ++-- .../docs/advanced/rest-api-documentation/Models/Target.md | 2 -- 4 files changed, 3 insertions(+), 7 deletions(-) diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache index 5fb1b8e1..8a75a8a9 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache @@ -13,7 +13,7 @@ All URIs are relative to *{{basePath}}:8082* | Method | HTTP request | Description | |------------- | ------------- | -------------| -{{#operations}}{{#operation}}| [**{{operationId}}**]({{#lambda.lowercase}}{{classname}}{{/lambda.lowercase}}.md#{{operationId}}) | **{{httpMethod}}** {{path}} | {{summary}} | +{{#operations}}{{#operation}}| **{{operationId}}** | **{{httpMethod}}** {{path}} | {{summary}} | {{/operation}}{{/operations}} {{#operations}} diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache index 1905a91e..d5d3f74a 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache @@ -25,7 +25,5 @@ description: > {{#vars}}| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}[**{{dataType}}**]({{#lambda.lowercase}}{{complexType}}{{/lambda.lowercase}}.md){{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} | {{/vars}} -[[Back to Model list]](/docs/advanced/rest-api-documentation/#documentation-for-models) [[Back to API list]](/docs/advanced/rest-api-documentation/#documentation-for-api-endpoints) [[Back to README]](/docs/advanced/rest-api-documentation/) - {{/model}} {{/models}} \ No newline at end of file diff --git a/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md index 2415c558..ec3b4c08 100644 --- a/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md +++ b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md @@ -12,8 +12,8 @@ All URIs are relative to *http://localhost:8082* | Method | HTTP request | Description | |------------- | ------------- | -------------| -| [**applyTargets**](defaultapi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | -| [**getClusterPlan**](defaultapi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | +| **applyTargets** | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | +| **getClusterPlan** | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | diff --git a/docs/content/docs/advanced/rest-api-documentation/Models/Target.md b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md index 17967253..ec4df149 100644 --- a/docs/content/docs/advanced/rest-api-documentation/Models/Target.md +++ b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md @@ -20,5 +20,3 @@ Network device to be monitored. Properties not marked as optional must be in JSO | **labels** | [**List**](map.md) | Input of labels as key:value pair. | [optional] [default to null] | | **operation** | **String** | Either `created`, `updated` or `deleted`. `created` and `updated` are identical and both apply the target. | [default to null] | -[[Back to Model list]](/docs/advanced/rest-api-documentation/#documentation-for-models) [[Back to API list]](/docs/advanced/rest-api-documentation/#documentation-for-api-endpoints) [[Back to README]](/docs/advanced/rest-api-documentation/) - From 8e8e25189c139b6aea8cddee2dc1d636c834ba9f Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Mon, 8 Jun 2026 16:46:17 +0200 Subject: [PATCH 62/76] add push-interface document structure --- docs/content/docs/advanced/openapi-generator-config.yaml | 2 ++ docs/content/docs/examples/NetBox/webhook/_index.md | 9 +++++++++ .../docs/user-guide/targetsource/push-interface.md | 9 +++++++++ 3 files changed, 20 insertions(+) create mode 100644 docs/content/docs/examples/NetBox/webhook/_index.md create mode 100644 docs/content/docs/user-guide/targetsource/push-interface.md diff --git a/docs/content/docs/advanced/openapi-generator-config.yaml b/docs/content/docs/advanced/openapi-generator-config.yaml index 9ab8481d..fb047ff3 100644 --- a/docs/content/docs/advanced/openapi-generator-config.yaml +++ b/docs/content/docs/advanced/openapi-generator-config.yaml @@ -1,3 +1,5 @@ +# docker run --rm -v ${PWD}:/local openapitools/openapi-generator-cli generate -c /local/docs/content/docs/advanced/openapi-generator-config.yaml + generatorName: markdown inputSpec: /local/docs/content/docs/advanced/openapi.yaml outputDir: /local/docs/content/docs/advanced/rest-api-documentation diff --git a/docs/content/docs/examples/NetBox/webhook/_index.md b/docs/content/docs/examples/NetBox/webhook/_index.md new file mode 100644 index 00000000..1b58a1df --- /dev/null +++ b/docs/content/docs/examples/NetBox/webhook/_index.md @@ -0,0 +1,9 @@ +--- +title: "Real-time target udpate with webhook" +linkTitle: "Real-time target udpate with webhook" +weight: 2 +description: > + Configure a webhook in Netbox to update targets in the gNMIc Operator real-time. +--- + +## Implemented soon \ No newline at end of file diff --git a/docs/content/docs/user-guide/targetsource/push-interface.md b/docs/content/docs/user-guide/targetsource/push-interface.md new file mode 100644 index 00000000..a8a4f75e --- /dev/null +++ b/docs/content/docs/user-guide/targetsource/push-interface.md @@ -0,0 +1,9 @@ +--- +title: "Push-based interface" +linkTitle: "Push-based interface" +weight: 4 +description: > + Enables REST API interface that accepts real-time target updates. +--- + +# Titel 1 From 7bc8886d19acaa58edcc5bdb8388249bae8ff452 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Mon, 8 Jun 2026 18:33:31 +0200 Subject: [PATCH 63/76] first version push --- .../docs/examples/NetBox/webhook/_index.md | 4 +- .../user-guide/targetsource/providers/http.md | 3 - .../user-guide/targetsource/push-interface.md | 9 -- .../docs/user-guide/targetsource/push.md | 105 ++++++++++++++++++ 4 files changed, 107 insertions(+), 14 deletions(-) delete mode 100644 docs/content/docs/user-guide/targetsource/push-interface.md create mode 100644 docs/content/docs/user-guide/targetsource/push.md diff --git a/docs/content/docs/examples/NetBox/webhook/_index.md b/docs/content/docs/examples/NetBox/webhook/_index.md index 1b58a1df..70cf240e 100644 --- a/docs/content/docs/examples/NetBox/webhook/_index.md +++ b/docs/content/docs/examples/NetBox/webhook/_index.md @@ -1,6 +1,6 @@ --- -title: "Real-time target udpate with webhook" -linkTitle: "Real-time target udpate with webhook" +title: "Real-time target update with webhook" +linkTitle: "Real-time target update with webhook" weight: 2 description: > Configure a webhook in Netbox to update targets in the gNMIc Operator real-time. diff --git a/docs/content/docs/user-guide/targetsource/providers/http.md b/docs/content/docs/user-guide/targetsource/providers/http.md index de0f3b6b..99bc1b2d 100644 --- a/docs/content/docs/user-guide/targetsource/providers/http.md +++ b/docs/content/docs/user-guide/targetsource/providers/http.md @@ -6,8 +6,6 @@ description: > The HTTP provider discovers targets from an HTTP endpoint returning JSON, or receives webhook-based updates when push mode is enabled. --- -The HTTP provider discovers targets from an HTTP endpoint returning JSON, or receives webhook-based updates when push mode is enabled. - ## Basic Configuration ```yaml @@ -16,7 +14,6 @@ kind: TargetSource metadata: name: targetsource-1 spec: - provider: provider: http: url: http://inventory-service:8080/targets diff --git a/docs/content/docs/user-guide/targetsource/push-interface.md b/docs/content/docs/user-guide/targetsource/push-interface.md deleted file mode 100644 index a8a4f75e..00000000 --- a/docs/content/docs/user-guide/targetsource/push-interface.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -title: "Push-based interface" -linkTitle: "Push-based interface" -weight: 4 -description: > - Enables REST API interface that accepts real-time target updates. ---- - -# Titel 1 diff --git a/docs/content/docs/user-guide/targetsource/push.md b/docs/content/docs/user-guide/targetsource/push.md new file mode 100644 index 00000000..18199694 --- /dev/null +++ b/docs/content/docs/user-guide/targetsource/push.md @@ -0,0 +1,105 @@ +--- +title: "Push mode" +linkTitle: "Push mode" +weight: 4 +description: > + Enables REST API interface that accepts real-time target updates. +--- + +## Basic configuration + +This CR enables the push interface with no authentication. + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: targetsource-1 +spec: + provider: + http: + push: + enabled: true +``` + +## Spec Fields + +| Field | Type | Required | Default | Description | +|-------|------|----------|---------|-------------| +| `push` | object | No | - | Push interface config | +| `enabled` | bool | Yes | False | Whether the push interface is active | +| `auth` | object | No | - | Bearer token authentication | +| `signature` | object | No | - | HTTP body verification using HMAC | +| `algorithm` | string | No | sha512 | Algorithm for signature verification(`sha256`or`sha512`) | + +## Address + +The REST API endpoint runs on `http://cluster-address:8082/api/v1/:namespace/target-source/:name/applyTargets`. + +- `cluster-address`: Adress of your cluster, localhost during development. +- `:namespace`: Namespace the gNMIc controller runs in. +- `:name`: Name of the TargetSource. + +See [real-time target update with webhook](/docs/examples/netbox/webhook) for an on how to configure the URI. + +## REST API + +Refer to the [REST API documentation](/docs/advanced/rest-api-documentation/) for the expected request schema and payload format. Any system or automation capable of sending HTTP POST requests can integrate with this interface. Compatibility has been validated using a [Netbox webhook](/docs/examples/netbox/webhook). + +## Security + +The API supports Bearer Token authentication and X-Hook-Signature, both are optional and turned off by default. They can be used in combination and are enabled by adding them to the specification. + +### Bearer Authentication + +Bearer authentication compares a token stored in Kubernetes with the one sent in the HTTP header. The Kubernetes secret is referenced as `tokenSecretRef`. + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: targetsource-1 +spec: + provider: + http: + push: + enabled: true + auth: + bearer: + tokenSecretRef: + name: gnmic-api-auth # secret name + key: bearer-token # secret key +``` + +This requires the [creation](https://kubernetes.ltd/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_generic/) of an Opaque Kuberentes secret: + +- Must be in the same namespace the gNMIc controller runs in. +- `name`: refers to the secret name +- `key`: key of the secret +- Example: `kubectl create secret generic gnmic-api-auth --from-literal=bearer-token=Secret...` + +#### Authorization Header + +HTTP request must contain the Bearer token in the header in the format: + +```yaml +Authorization: Bearer Secret... +``` + +### Signature + +Signature verification requires an Opaque Kubernetes secret that stores the shared key (see Bearer Authentication). For each request, the HMAC generated from the request body and shared key must be provided in the `X-Hook-Signature` header. Refer to the [Netbox webhook](/docs/examples/netbox/webhook) example for a configuration reference. + +```yaml +spec: + provider: + http: + push: + enabled: true + auth: + signature: + algorithm: sha512 + secretRef: + name: gnmic-signature + key: signature +``` From a4cdf035f9b43b010fc408f96ce44c21c259e0de Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Mon, 8 Jun 2026 23:10:07 +0200 Subject: [PATCH 64/76] add Netbox webhook example --- .../docs/examples/NetBox/webhook/_index.md | 120 +++++++++++++++++- .../docs/user-guide/targetsource/push.md | 6 +- 2 files changed, 124 insertions(+), 2 deletions(-) diff --git a/docs/content/docs/examples/NetBox/webhook/_index.md b/docs/content/docs/examples/NetBox/webhook/_index.md index 70cf240e..3e499d3b 100644 --- a/docs/content/docs/examples/NetBox/webhook/_index.md +++ b/docs/content/docs/examples/NetBox/webhook/_index.md @@ -6,4 +6,122 @@ description: > Configure a webhook in Netbox to update targets in the gNMIc Operator real-time. --- -## Implemented soon \ No newline at end of file +## Netbox Webhook Configuration + +This example will run you through the configuration of a webhook in Netbox. This allows for real-time target updates from Netbox into the gNMIc Operator. The configuration steps are: + +1. Apply TargetSource +2. Create Kubernetes Secrets +3. Configure Webhook +4. Create Event Rule +5. Verification + +### Prerequisites + +- Kubernetes cluster with gNMIc Operator installed +- kubectl access to your cluster +- Running Netbox instance +- Netbox can send HTTP requests to the gNMIc Operator + +### 1. Apply TargetSource + +Apply the targetSource with `kubectl apply -f netbox.yaml -n default`. + +- `enabled` set to `true`, otherwise target updates are rejected +- Bearer authentication and Signature activated + +```yaml +# netbox.yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: netbox +spec: + provider: + http: + push: + enabled: true + auth: + bearer: + tokenSecretRef: + name: gnmic-api-auth + key: bearer-token + signature: + secretRef: + name: gnmic-signature + key: signature + targetLabels: + integrationtest: http + targetProfile: default +``` + +### 2. Create Kubernetes Secrets + +Authentication and the signature both require a Kubernetes secret. These must: + +- Be in the same namespace as the TargetSource, in this case `default`. +- `Name` and `Key` align with the TargetSource spec. + +```bash +kubectl create secret generic gnmic-api-auth --from-literal=bearer-token=thisIsASecureToken -n default +kubectl create secret generic gnmic-signature --from-literal=signature=SecretSignature -n default +``` + +### 3. Configure Webhook + +Now we switch to Netbox and configure a webhook. The webhook gets triggered by events like `device update` and sends a HTTP POST request to the gNMIc Operator. + +Configure the Webhook under `Operations > Webhooks` and create a new Webhook with the following settings: + +- *Name*: GNMIc operator push +- *URL*: `http://gnmic-controller-manager-api.gnmic-system.svc.cluster.local:8082/api/v1/default/target-source/netbox/applyTargets` + - The cluster-address might be `http://localhost:8082/` or `http://servername:8082/`, depending on your setup. +- *HTTP method*: POST +- *HTTP content type*: application/json +- *SSL Verification*: true +- *Additional headers:* `Authorization: Bearer thisIsASecureToken` +- *Body Template*: + + ```json + [ + { + "name": "{{ data.name }}", + "address": "{{ data.primary_ip4.address.split('/')[0] }}", + "operation": "{{ event }}", + "targetProfile": "{{ data.custom_fields.target_profile | default('', true) }}", + "port": {{ data.custom_fields.gnmic_port | default(57400, true) }}, + "labels": [ + { + "key": "vendor", + "value": "{{ data.device_type.manufacturer.name }}" + } + ] + } + ] + ``` + +- *Secret*: `SecretSignature` + +### 4. Create Event Rule + +The webhook just created needs a trigger, which is created as an event rule under `Operations > Event Rules`. + +- *Name*: gNMIc Operator push target change +- *Object types*: DCIM > Device +- *Event types*: "Object Created", "Object Updated", "Object Deleted" +- *Action type*: Webhook +- *Webhook*: gNMIc Operator push + +### 5. Verification + +Updating a device in Netbox will now trigger the webhook, verify this with these commands: + +```bash +kubectl get targets +kubectl get targets -o yaml + +# Check logs of incoming POST requests: +kubectl logs -n gnmic-system deploy/gnmic-controller-manager -f +``` + +Every POST request received will write logs, even if rejected. If no POST request are being logged, the request is not received. diff --git a/docs/content/docs/user-guide/targetsource/push.md b/docs/content/docs/user-guide/targetsource/push.md index 18199694..5ed69e4a 100644 --- a/docs/content/docs/user-guide/targetsource/push.md +++ b/docs/content/docs/user-guide/targetsource/push.md @@ -37,7 +37,7 @@ spec: The REST API endpoint runs on `http://cluster-address:8082/api/v1/:namespace/target-source/:name/applyTargets`. - `cluster-address`: Adress of your cluster, localhost during development. -- `:namespace`: Namespace the gNMIc controller runs in. +- `:namespace`: Namespace the TargetSource is created in. - `:name`: Name of the TargetSource. See [real-time target update with webhook](/docs/examples/netbox/webhook) for an on how to configure the URI. @@ -86,6 +86,10 @@ HTTP request must contain the Bearer token in the header in the format: Authorization: Bearer Secret... ``` +#### Reverse Proxy + +Use one. + ### Signature Signature verification requires an Opaque Kubernetes secret that stores the shared key (see Bearer Authentication). For each request, the HMAC generated from the request body and shared key must be provided in the `X-Hook-Signature` header. Refer to the [Netbox webhook](/docs/examples/netbox/webhook) example for a configuration reference. From b17bf93ef606028be3fa102e8a52ff4041791032 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Tue, 9 Jun 2026 00:07:13 +0200 Subject: [PATCH 65/76] small grammar changes --- .../markdown-documentation/README.mustache | 2 + .../markdown-documentation/model.mustache | 2 +- .../rest-api-documentation/Models/Target.md | 2 +- .../advanced/rest-api-documentation/_index.md | 2 + .../examples/NetBox/Export Template/_index.md | 12 ++--- .../docs/examples/NetBox/REST API/_index.md | 4 +- .../docs/examples/NetBox/webhook/_index.md | 45 +++++++++++-------- .../docs/user-guide/targetsource/push.md | 18 +++++--- 8 files changed, 51 insertions(+), 36 deletions(-) diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache index 9bf976ad..7b3df486 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache @@ -34,6 +34,8 @@ No model defined in this package {{! TODO: optional documentation for authorization? }} ## Documentation for Authorization +For a detailed explanation on how to configure the required secrets within the gNMIc Operator, refer to [TargetSource > Push mode](/docs/user-guide/targetsource/push/). + {{^authMethods}} All endpoints do not require authorization. {{/authMethods}} diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache index d5d3f74a..3ee88fd2 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache @@ -8,7 +8,7 @@ description: > {{#models}} {{#model}} -# {{#lambda.lowercase}}{{{classname}}}{{/lambda.lowercase}} +# {{#lambda}}{{{classname}}}{{/lambda}} {{#description}} {{{description}}} {{/description}} diff --git a/docs/content/docs/advanced/rest-api-documentation/Models/Target.md b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md index ec4df149..58c08caa 100644 --- a/docs/content/docs/advanced/rest-api-documentation/Models/Target.md +++ b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md @@ -6,7 +6,7 @@ description: > Documentation for OpenAPI models and their schema-defined properties. --- -# target +# Target Network device to be monitored. Properties not marked as optional must be in JSON body. ## Properties diff --git a/docs/content/docs/advanced/rest-api-documentation/_index.md b/docs/content/docs/advanced/rest-api-documentation/_index.md index a0ad838b..93b2d678 100644 --- a/docs/content/docs/advanced/rest-api-documentation/_index.md +++ b/docs/content/docs/advanced/rest-api-documentation/_index.md @@ -26,6 +26,8 @@ All URIs are relative to *http://localhost:8082* ## Documentation for Authorization +For a detailed explanation on how to configure the required secrets within the gNMIc Operator, refer to [TargetSource > Push mode](/docs/user-guide/targetsource/push/). + ### bearerAuth diff --git a/docs/content/docs/examples/NetBox/Export Template/_index.md b/docs/content/docs/examples/NetBox/Export Template/_index.md index 2cc48b3b..fd481841 100644 --- a/docs/content/docs/examples/NetBox/Export Template/_index.md +++ b/docs/content/docs/examples/NetBox/Export Template/_index.md @@ -1,6 +1,6 @@ --- -title: "NetBox (Export Template)" -linkTitle: "NetBox Export Template" +title: "Pull with Export Template" +linkTitle: "Pull with Export Template" weight: 1 description: > Discover targets from NetBox using HTTP provider with NetBox Export Template @@ -12,14 +12,14 @@ Export Templates offer powerful filtering, transformation, and formatting direct ## Overview -An **Export Template** is a Jinja2 template defined in NetBox that: +An **Export Template** is a Jinja2 template that: 1. **Queries** NetBox's internal database (devices, interfaces, etc.) 2. **Filters** results based on custom criteria -3. **Transforms** data into your desired output format (JSON, YAML, CSV, etc.) -4. **Returns** the formatted output via a custom REST API endpoint +3. **Transforms** data into your desired output format +4. **Returns** the formatted output via REST API endpoint -When used with gNMIc's HTTP provider, the operator simply fetches the rendered JSON template and parses the result — no additional gNMIc Operator transformation needed if done correctly. +When used with gNMIc's HTTP provider, the operator fetches the rendered JSON template and parses the result with no further transformation needed by the gNMIc Operator. --- diff --git a/docs/content/docs/examples/NetBox/REST API/_index.md b/docs/content/docs/examples/NetBox/REST API/_index.md index 18111d75..bb4be0e4 100644 --- a/docs/content/docs/examples/NetBox/REST API/_index.md +++ b/docs/content/docs/examples/NetBox/REST API/_index.md @@ -1,6 +1,6 @@ --- -title: "NetBox (REST API)" -linkTitle: "NetBox REST" +title: "Pull with REST API" +linkTitle: "Pull with REST API"" weight: 2 description: > Discover targets from NetBox using the HTTP provider and NetBox REST API diff --git a/docs/content/docs/examples/NetBox/webhook/_index.md b/docs/content/docs/examples/NetBox/webhook/_index.md index 3e499d3b..b8b11467 100644 --- a/docs/content/docs/examples/NetBox/webhook/_index.md +++ b/docs/content/docs/examples/NetBox/webhook/_index.md @@ -1,14 +1,14 @@ --- -title: "Real-time target update with webhook" -linkTitle: "Real-time target update with webhook" +title: "Push mode with webhook" +linkTitle: "Push mode with webhook" weight: 2 description: > - Configure a webhook in Netbox to update targets in the gNMIc Operator real-time. + Configure a webhook in NetBox to update targets in the gNMIc Operator in real time. --- ## Netbox Webhook Configuration -This example will run you through the configuration of a webhook in Netbox. This allows for real-time target updates from Netbox into the gNMIc Operator. The configuration steps are: +This tutorial walks through configuring a webhook in NetBox to push real-time target updates to the gNMIc Operator. The workflow includes the following steps: 1. Apply TargetSource 2. Create Kubernetes Secrets @@ -19,16 +19,18 @@ This example will run you through the configuration of a webhook in Netbox. This ### Prerequisites - Kubernetes cluster with gNMIc Operator installed -- kubectl access to your cluster -- Running Netbox instance -- Netbox can send HTTP requests to the gNMIc Operator +- `kubectl` access to your cluster +- Running NetBox instance +- Network connectivity from NetBox to the gNMIc Operator API endpoint + +--- ### 1. Apply TargetSource -Apply the targetSource with `kubectl apply -f netbox.yaml -n default`. +Apply the TargetSource manifest: `kubectl apply -f netbox.yaml -n default` -- `enabled` set to `true`, otherwise target updates are rejected -- Bearer authentication and Signature activated +- `enabled` must be set to `true`, otherwise updates are rejected. +- Bearer authentication and signature verification are enabled. ```yaml # netbox.yaml @@ -55,12 +57,14 @@ spec: targetProfile: default ``` +> Namespace is `default`, the name of the TargetSource is `netbox`. These values will be in the URL in step 3. + ### 2. Create Kubernetes Secrets -Authentication and the signature both require a Kubernetes secret. These must: +Bearer authentication and signature verification both require Kubernetes secrets. Ensure that the secrets: -- Be in the same namespace as the TargetSource, in this case `default`. -- `Name` and `Key` align with the TargetSource spec. +- Are created in the same namespace as the TargetSource (`default` in this example). +- Use `name` and `key` values that match the TargetSource spec. ```bash kubectl create secret generic gnmic-api-auth --from-literal=bearer-token=thisIsASecureToken -n default @@ -69,13 +73,14 @@ kubectl create secret generic gnmic-signature --from-literal=signature=SecretSig ### 3. Configure Webhook -Now we switch to Netbox and configure a webhook. The webhook gets triggered by events like `device update` and sends a HTTP POST request to the gNMIc Operator. +Next, configure a webhook in NetBox. The webhook is triggered by device events (for example, updates) and sends an HTTP POST request to the gNMIc Operator. -Configure the Webhook under `Operations > Webhooks` and create a new Webhook with the following settings: +In NetBox, go to `Operations > Webhooks` and create a webhook with the following settings: - *Name*: GNMIc operator push - *URL*: `http://gnmic-controller-manager-api.gnmic-system.svc.cluster.local:8082/api/v1/default/target-source/netbox/applyTargets` - - The cluster-address might be `http://localhost:8082/` or `http://servername:8082/`, depending on your setup. + - Depending on your environment, the cluster address may instead be `http://localhost:8082/` or `http://servername:8082/`. + - URL contains the namespace `default` and TargetSource name `netbox`. - *HTTP method*: POST - *HTTP content type*: application/json - *SSL Verification*: true @@ -104,7 +109,7 @@ Configure the Webhook under `Operations > Webhooks` and create a new Webhook wit ### 4. Create Event Rule -The webhook just created needs a trigger, which is created as an event rule under `Operations > Event Rules`. +The webhook requires a trigger, configured as an event rule under `Operations > Event Rules`. - *Name*: gNMIc Operator push target change - *Object types*: DCIM > Device @@ -112,9 +117,11 @@ The webhook just created needs a trigger, which is created as an event rule unde - *Action type*: Webhook - *Webhook*: gNMIc Operator push +--- + ### 5. Verification -Updating a device in Netbox will now trigger the webhook, verify this with these commands: +Updating a device in NetBox should now trigger the webhook. Verify this with the following commands: ```bash kubectl get targets @@ -124,4 +131,4 @@ kubectl get targets -o yaml kubectl logs -n gnmic-system deploy/gnmic-controller-manager -f ``` -Every POST request received will write logs, even if rejected. If no POST request are being logged, the request is not received. +Every incoming POST request is logged, including rejected requests. If no POST requests appear in the logs, the webhook request is not reaching the gNMIc Operator. diff --git a/docs/content/docs/user-guide/targetsource/push.md b/docs/content/docs/user-guide/targetsource/push.md index 5ed69e4a..9fa35fdf 100644 --- a/docs/content/docs/user-guide/targetsource/push.md +++ b/docs/content/docs/user-guide/targetsource/push.md @@ -40,16 +40,20 @@ The REST API endpoint runs on `http://cluster-address:8082/api/v1/:namespace/tar - `:namespace`: Namespace the TargetSource is created in. - `:name`: Name of the TargetSource. -See [real-time target update with webhook](/docs/examples/netbox/webhook) for an on how to configure the URI. +See [Push mode with webhook](/docs/examples/netbox/webhook) for an example on how to configure the URL. ## REST API -Refer to the [REST API documentation](/docs/advanced/rest-api-documentation/) for the expected request schema and payload format. Any system or automation capable of sending HTTP POST requests can integrate with this interface. Compatibility has been validated using a [Netbox webhook](/docs/examples/netbox/webhook). +Refer to the [REST API documentation](/docs/advanced/rest-api-documentation/) for the expected request schema and payload format. + +Any system or script capable of sending HTTP POST requests can integrate with this interface. ## Security The API supports Bearer Token authentication and X-Hook-Signature, both are optional and turned off by default. They can be used in combination and are enabled by adding them to the specification. +An example configuration of both is documented in the [Netbox webhook](/docs/examples/netbox/webhook) example. + ### Bearer Authentication Bearer authentication compares a token stored in Kubernetes with the one sent in the HTTP header. The Kubernetes secret is referenced as `tokenSecretRef`. @@ -86,13 +90,9 @@ HTTP request must contain the Bearer token in the header in the format: Authorization: Bearer Secret... ``` -#### Reverse Proxy - -Use one. - ### Signature -Signature verification requires an Opaque Kubernetes secret that stores the shared key (see Bearer Authentication). For each request, the HMAC generated from the request body and shared key must be provided in the `X-Hook-Signature` header. Refer to the [Netbox webhook](/docs/examples/netbox/webhook) example for a configuration reference. +Signature verification requires an Opaque Kubernetes secret that stores the shared key (see Bearer Authentication). For each request, the HMAC generated from the request body and shared key must be provided in the `X-Hook-Signature` header. ```yaml spec: @@ -107,3 +107,7 @@ spec: name: gnmic-signature key: signature ``` + +#### Reverse Proxy + +In order to have a secure setup, the HTTP post requests must be sent using TLS. The REST API interface does not support HTTPS, at least not directly. It is recommended to terminate the TLS connection at the reverse proxy and forward a plain HTTP request to the gNMIc Operator. From 5e5cca72d925fdfe0fdfe01bcd68cef2be894ac0 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Tue, 9 Jun 2026 00:23:31 +0200 Subject: [PATCH 66/76] fix pages build --- docs/content/docs/examples/NetBox/REST API/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/docs/examples/NetBox/REST API/_index.md b/docs/content/docs/examples/NetBox/REST API/_index.md index bb4be0e4..fcbd990a 100644 --- a/docs/content/docs/examples/NetBox/REST API/_index.md +++ b/docs/content/docs/examples/NetBox/REST API/_index.md @@ -1,6 +1,6 @@ --- title: "Pull with REST API" -linkTitle: "Pull with REST API"" +linkTitle: "Pull with REST API" weight: 2 description: > Discover targets from NetBox using the HTTP provider and NetBox REST API From a9ad35f002a1a385d9d87473a7f3a9261ebc39c2 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Tue, 9 Jun 2026 00:29:03 +0200 Subject: [PATCH 67/76] remove unneeded files, cleanup --- .../docs/advanced/.openapi-generator-ignore | 23 ----- .../advanced/openapi-generator-config.yaml | 2 +- docs/content/docs/advanced/openapi.yaml | 91 ------------------- 3 files changed, 1 insertion(+), 115 deletions(-) delete mode 100644 docs/content/docs/advanced/.openapi-generator-ignore delete mode 100644 docs/content/docs/advanced/openapi.yaml diff --git a/docs/content/docs/advanced/.openapi-generator-ignore b/docs/content/docs/advanced/.openapi-generator-ignore deleted file mode 100644 index 7484ee59..00000000 --- a/docs/content/docs/advanced/.openapi-generator-ignore +++ /dev/null @@ -1,23 +0,0 @@ -# OpenAPI Generator Ignore -# Generated by openapi-generator https://github.com/openapitools/openapi-generator - -# Use this file to prevent files from being overwritten by the generator. -# The patterns follow closely to .gitignore or .dockerignore. - -# As an example, the C# client generator defines ApiClient.cs. -# You can make changes and tell OpenAPI Generator to ignore just this file by uncommenting the following line: -#ApiClient.cs - -# You can match any string of characters against a directory, file or extension with a single asterisk (*): -#foo/*/qux -# The above matches foo/bar/qux and foo/baz/qux, but not foo/bar/baz/qux - -# You can recursively match patterns against a directory, file or extension with a double asterisk (**): -#foo/**/qux -# This matches foo/bar/qux, foo/baz/qux, and foo/bar/baz/qux - -# You can also negate patterns with an exclamation (!). -# For example, you can ignore all files in a docs folder with the file extension .md: -#docs/*.md -# Then explicitly reverse the ignore rule for a single file: -#!docs/README.md diff --git a/docs/content/docs/advanced/openapi-generator-config.yaml b/docs/content/docs/advanced/openapi-generator-config.yaml index fb047ff3..5f041408 100644 --- a/docs/content/docs/advanced/openapi-generator-config.yaml +++ b/docs/content/docs/advanced/openapi-generator-config.yaml @@ -1,7 +1,7 @@ # docker run --rm -v ${PWD}:/local openapitools/openapi-generator-cli generate -c /local/docs/content/docs/advanced/openapi-generator-config.yaml generatorName: markdown -inputSpec: /local/docs/content/docs/advanced/openapi.yaml +inputSpec: /local/internal/apiserver/openapi.yaml outputDir: /local/docs/content/docs/advanced/rest-api-documentation templateDir: /local/docs/content/docs/advanced/openapi-templates/markdown-documentation files: diff --git a/docs/content/docs/advanced/openapi.yaml b/docs/content/docs/advanced/openapi.yaml deleted file mode 100644 index 7dfbc24b..00000000 --- a/docs/content/docs/advanced/openapi.yaml +++ /dev/null @@ -1,91 +0,0 @@ -openapi: 3.0.3 -info: - title: "gNMIc Operator REST API" - version: "0.0.1" -paths: - /clusters/:namespace/:name/plan: - get: - summary: "Get cluster plan." - operationId: "getClusterPlan" - responses: - '200': - description: "ClusterPlan returned" - /api/v1/:namespace/target-source/:name/applyTargets: - post: - summary: "Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator." - operationId: "applyTargets" - security: - - bearerAuth: [] - signature: [] - requestBody: - required: true - description: Target must be passed as a list, multiple targets possible. - content: - application/json: - schema: - $ref: '#/components/schemas/Targets' - responses: - '201': - description: "Targets applied successfully" - content: - application/json: - schema: - $ref: '#/components/schemas/Targets' - '401': - description: Access token is missing or invalid - -components: - schemas: - Targets: - type: array - items: - $ref: '#/components/schemas/Target' - Label: - description: TBD - type: object - additionalProperties: - description: Label must be passed as key:value pair, multiple values per key possible. - type: string - Target: - description: Network device to be monitored. Properties not marked as optional must be in JSON body. - type: object - required: - - name - - address - - operation - properties: - name: - type: string - description: Name of device to be monitored. - address: - type: string - description: IPv4/IPv6 address or hostname. - port: - type: integer - description: gNMIc port. - targetProfile: - type: string - description: TargetProfile applied to apply to this router. - labels: - type: array - description: Input of labels as key:value pair. - items: - $ref: '#/components/schemas/Label' - operation: - type: string - enum: - - created - - updated - - deleted - description: "Either `created`, `updated` or `deleted`. `created` and `updated` are identical and both apply the target." - securitySchemes: - bearerAuth: - type: http - scheme: bearer - description: HTTP authentication using Bearer token - signature: - name: X-Hook-Signature - type: apiKey - in: header - description: HMAC signature of the request payload - \ No newline at end of file From 5394f55297652cfb7fa60345264f9a73d90d9ea8 Mon Sep 17 00:00:00 2001 From: Daniel Schatzmann Date: Tue, 9 Jun 2026 08:31:14 +0000 Subject: [PATCH 68/76] update recommended patterns of CEL expressions --- .../user-guide/targetsource/providers/http.md | 31 +++++++++---------- 1 file changed, 15 insertions(+), 16 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource/providers/http.md b/docs/content/docs/user-guide/targetsource/providers/http.md index de0f3b6b..a6a21055 100644 --- a/docs/content/docs/user-guide/targetsource/providers/http.md +++ b/docs/content/docs/user-guide/targetsource/providers/http.md @@ -442,22 +442,7 @@ When writing more complex CEL expressions, it is recommended to use YAML’s pip This is especially useful for expressions that span multiple lines or contain nested logic. -## Push Mode - -The HTTP provider supports webhook-based target updates via `spec.provider.http.push`. - -```yaml -spec: - provider: - http: - push: - enabled: true -``` - -When `push.enabled` is true, the operator accepts incoming webhook notifications and can update targets without polling a remote endpoint. The `url` field is optional when push mode is enabled, but can still be used for polling and fallback behavior. - -#### Recommended pattern (labels example) - +**Labels example:** ```yaml mapping: labels: | @@ -474,6 +459,20 @@ mapping: - **Maintainability**: Complex CEL expressions don't require escaping - **YAML best practice**: Literal blocks handle special characters naturally +## Push Mode + +The HTTP provider supports webhook-based target updates via `spec.provider.http.push`. + +```yaml +spec: + provider: + http: + push: + enabled: true +``` + +When `push.enabled` is true, the operator accepts incoming webhook notifications and can update targets without polling a remote endpoint. The `url` field is optional when push mode is enabled, but can still be used for polling and fallback behavior. + ## Recommended Production Settings When deploying HTTP TargetSource providers in production networks, follow these guidelines to ensure reliable and efficient target discovery: From 89841097071125f028f5e4c4b0017db414315df6 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Wed, 10 Jun 2026 10:17:45 +0200 Subject: [PATCH 69/76] update webhook example --- .../docs/examples/NetBox/webhook/_index.md | 158 ++++++++++++++---- 1 file changed, 130 insertions(+), 28 deletions(-) diff --git a/docs/content/docs/examples/NetBox/webhook/_index.md b/docs/content/docs/examples/NetBox/webhook/_index.md index b8b11467..f53e8bb3 100644 --- a/docs/content/docs/examples/NetBox/webhook/_index.md +++ b/docs/content/docs/examples/NetBox/webhook/_index.md @@ -8,14 +8,18 @@ description: > ## Netbox Webhook Configuration -This tutorial walks through configuring a webhook in NetBox to push real-time target updates to the gNMIc Operator. The workflow includes the following steps: +This example walks through configuring a webhook in NetBox to push real-time target updates to the gNMIc Operator. It covers the configuration in the gNMIc Operator (Step 1-3), and the configuration within Netbox (step 4). -1. Apply TargetSource +1. Create Targetprofile 2. Create Kubernetes Secrets -3. Configure Webhook -4. Create Event Rule +3. Apply TargetSource +4. Netbox setup + a: Configure Webhook + b: Create Event Rule 5. Verification +At the end, the logs will show the incoming POST requests and the targets are in status `READY`. + ### Prerequisites - Kubernetes cluster with gNMIc Operator installed @@ -25,9 +29,57 @@ This tutorial walks through configuring a webhook in NetBox to push real-time ta --- -### 1. Apply TargetSource +### 1. Create TargetProfile + +Define how discovered targets should be configured. The `TargetProfile` contains device credentials, such as username/password or client certificates. These are either defined inline strings or stored in a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/). + +```yaml +# Replace YOUR_DEVICE_USERNAME and YOUR_DEVICE_PASSWORD with your corresponding default device username and password +apiVersion: v1 +kind: Secret +metadata: + name: device-credentials + namespace: gnmic-system +type: Opaque +stringData: + username: YOUR_DEVICE_USERNAME + password: YOUR_DEVICE_PASSWORD +``` + +When using a secret, create a credentials Secret first, then reference it from the profile. + +```yaml +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetProfile +metadata: + name: netbox-device + namespace: gnmic-system +spec: + credentialsRef: device-credentials + timeout: 10s +``` + +For more TargetProfile options and credential handling, see the operator documentation for `TargetProfile`. + +--- + +### 2. Create Kubernetes Secrets + +Bearer authentication and signature verification both require Kubernetes secrets. Ensure that the secrets: + +- Are created in the same namespace as the TargetSource (`gnmic-system` in this example). +- Use `name` and `key` values that match the TargetSource spec. + +```bash +kubectl create secret generic gnmic-api-auth --from-literal=bearer-token=YOUR_SECURE_TOKEN -n gnmic-system +kubectl create secret generic gnmic-signature --from-literal=signature=YOUR_SIGNATURE -n gnmic-system +``` + +--- + +### 3. Apply TargetSource -Apply the TargetSource manifest: `kubectl apply -f netbox.yaml -n default` +Apply the TargetSource manifest: `kubectl apply -f netbox.yaml -n gnmic-system` - `enabled` must be set to `true`, otherwise updates are rejected. - Bearer authentication and signature verification are enabled. @@ -38,7 +90,13 @@ apiVersion: operator.gnmic.dev/v1alpha1 kind: TargetSource metadata: name: netbox + namespace: gnmic-system spec: + targetPort: 57400 + targetProfile: netbox-device + targetLabels: + inventory: netbox + sync-source: rest-api provider: http: push: @@ -52,33 +110,23 @@ spec: secretRef: name: gnmic-signature key: signature - targetLabels: - integrationtest: http - targetProfile: default ``` -> Namespace is `default`, the name of the TargetSource is `netbox`. These values will be in the URL in step 3. - -### 2. Create Kubernetes Secrets +> Namespace is `gnmic-system`, the name of the TargetSource is `netbox`. These values will be in the URL in step 4. -Bearer authentication and signature verification both require Kubernetes secrets. Ensure that the secrets: +--- -- Are created in the same namespace as the TargetSource (`default` in this example). -- Use `name` and `key` values that match the TargetSource spec. +### 4. Netbox Setup -```bash -kubectl create secret generic gnmic-api-auth --from-literal=bearer-token=thisIsASecureToken -n default -kubectl create secret generic gnmic-signature --from-literal=signature=SecretSignature -n default -``` +Next, configure a webhook in NetBox. The webhook is triggered by device events (for example, updates) and sends an HTTP POST request to the gNMIc Operator. -### 3. Configure Webhook +#### Configure Webhook -Next, configure a webhook in NetBox. The webhook is triggered by device events (for example, updates) and sends an HTTP POST request to the gNMIc Operator. In NetBox, go to `Operations > Webhooks` and create a webhook with the following settings: - *Name*: GNMIc operator push -- *URL*: `http://gnmic-controller-manager-api.gnmic-system.svc.cluster.local:8082/api/v1/default/target-source/netbox/applyTargets` +- *URL*: `http://gnmic-controller-manager-api.gnmic-system.svc.cluster.local:8082/api/v1/gnmic-system/target-source/netbox/applyTargets` - Depending on your environment, the cluster address may instead be `http://localhost:8082/` or `http://servername:8082/`. - URL contains the namespace `default` and TargetSource name `netbox`. - *HTTP method*: POST @@ -96,18 +144,15 @@ In NetBox, go to `Operations > Webhooks` and create a webhook with the following "targetProfile": "{{ data.custom_fields.target_profile | default('', true) }}", "port": {{ data.custom_fields.gnmic_port | default(57400, true) }}, "labels": [ - { - "key": "vendor", - "value": "{{ data.device_type.manufacturer.name }}" - } - ] + "vendor":"{{ data.device_type.manufacturer.name }}" + ] } ] ``` - *Secret*: `SecretSignature` -### 4. Create Event Rule +#### Create Event Rule The webhook requires a trigger, configured as an event rule under `Operations > Event Rules`. @@ -132,3 +177,60 @@ kubectl logs -n gnmic-system deploy/gnmic-controller-manager -f ``` Every incoming POST request is logged, including rejected requests. If no POST requests appear in the logs, the webhook request is not reaching the gNMIc Operator. + +--- + +### Example: Complete Setup + +Here's a complete example combining all resources: + + ```yaml +--- +# Secret for Target Credential +apiVersion: v1 +kind: Secret +metadata: + name: device-credentials + namespace: gnmic-system +type: Opaque +stringData: + username: YOUR_DEVICE_USERNAME + password: YOUR_DEVICE_PASSWORD + +--- +# TargetProfile +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetProfile +metadata: + name: netbox-device + namespace: gnmic-system +spec: + credentialsRef: device-credentials + timeout: 10s +--- +# Apply Targetsource +apiVersion: operator.gnmic.dev/v1alpha1 +kind: TargetSource +metadata: + name: netbox + namespace: gnmic-system +spec: + targetPort: 57400 + targetProfile: netbox-device + targetLabels: + inventory: netbox + sync-source: rest-api + provider: + http: + push: + enabled: true + auth: + bearer: + tokenSecretRef: + name: gnmic-api-auth + key: bearer-token + signature: + secretRef: + name: gnmic-signature + key: signature +``` From b2a27d7091fe034c1ace497ac0568f1f1a345bfe Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Wed, 10 Jun 2026 12:57:48 +0200 Subject: [PATCH 70/76] remove link --- .../markdown-documentation/model.mustache | 4 ++-- .../rest-api-documentation/Models/Target.md | 2 +- .../docs/examples/NetBox/webhook/_index.md | 23 ++++++++++--------- .../user-guide/targetsource/providers/http.md | 2 ++ .../docs/user-guide/targetsource/push.md | 14 +++++++---- 5 files changed, 26 insertions(+), 19 deletions(-) diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache index 3ee88fd2..a110b934 100644 --- a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache +++ b/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache @@ -19,10 +19,10 @@ description: > |------------ | ------------- | ------------- | -------------| {{#parent}} {{#parentVars}} -| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}[**{{dataType}}**]({{#lambda.lowercase}}{{complexType}}{{/lambda.lowercase}}.md){{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} | +| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} | {{/parentVars}} {{/parent}} -{{#vars}}| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}[**{{dataType}}**]({{#lambda.lowercase}}{{complexType}}{{/lambda.lowercase}}.md){{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} | +{{#vars}}| **{{name}}** | {{#isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}}{{^isPrimitiveType}}**{{dataType}}**{{/isPrimitiveType}} | {{{description}}} | {{^required}}[optional] {{/required}}{{#readOnly}}[readonly] {{/readOnly}}{{#defaultValue}}[default to {{{.}}}]{{/defaultValue}} | {{/vars}} {{/model}} diff --git a/docs/content/docs/advanced/rest-api-documentation/Models/Target.md b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md index 58c08caa..86e318a4 100644 --- a/docs/content/docs/advanced/rest-api-documentation/Models/Target.md +++ b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md @@ -17,6 +17,6 @@ Network device to be monitored. Properties not marked as optional must be in JSO | **address** | **String** | IPv4/IPv6 address or hostname. | [default to null] | | **port** | **Integer** | gNMIc port. | [optional] [default to null] | | **targetProfile** | **String** | TargetProfile applied to apply to this router. | [optional] [default to null] | -| **labels** | [**List**](map.md) | Input of labels as key:value pair. | [optional] [default to null] | +| **labels** | **List** | Labels must be map[string]string. For example vendor:nokia | [optional] [default to null] | | **operation** | **String** | Either `created`, `updated` or `deleted`. `created` and `updated` are identical and both apply the target. | [default to null] | diff --git a/docs/content/docs/examples/NetBox/webhook/_index.md b/docs/content/docs/examples/NetBox/webhook/_index.md index f53e8bb3..01e69eb3 100644 --- a/docs/content/docs/examples/NetBox/webhook/_index.md +++ b/docs/content/docs/examples/NetBox/webhook/_index.md @@ -1,6 +1,6 @@ --- -title: "Push mode with webhook" -linkTitle: "Push mode with webhook" +title: "Push Mode with Webhook" +linkTitle: "Push Mode with Webhook" weight: 2 description: > Configure a webhook in NetBox to update targets in the gNMIc Operator in real time. @@ -20,7 +20,7 @@ This example walks through configuring a webhook in NetBox to push real-time tar At the end, the logs will show the incoming POST requests and the targets are in status `READY`. -### Prerequisites +## Prerequisites - Kubernetes cluster with gNMIc Operator installed - `kubectl` access to your cluster @@ -71,8 +71,8 @@ Bearer authentication and signature verification both require Kubernetes secrets - Use `name` and `key` values that match the TargetSource spec. ```bash -kubectl create secret generic gnmic-api-auth --from-literal=bearer-token=YOUR_SECURE_TOKEN -n gnmic-system -kubectl create secret generic gnmic-signature --from-literal=signature=YOUR_SIGNATURE -n gnmic-system +kubectl create secret generic gnmic-api-auth --from-literal=bearer-token=YOUR_SECRET_TOKEN -n gnmic-system +kubectl create secret generic gnmic-signature --from-literal=signature=YOUR_SECRET_SIGNATURE -n gnmic-system ``` --- @@ -122,17 +122,18 @@ Next, configure a webhook in NetBox. The webhook is triggered by device events ( #### Configure Webhook - In NetBox, go to `Operations > Webhooks` and create a webhook with the following settings: - *Name*: GNMIc operator push - *URL*: `http://gnmic-controller-manager-api.gnmic-system.svc.cluster.local:8082/api/v1/gnmic-system/target-source/netbox/applyTargets` - - Depending on your environment, the cluster address may instead be `http://localhost:8082/` or `http://servername:8082/`. - - URL contains the namespace `default` and TargetSource name `netbox`. + - URL contains the namespace `gnmic-system` and TargetSource name `netbox`. + - `gnmic-controller-manager-api.gnmic-system.svc.cluster.local` is only reachable if Netbox is inside the cluster. + - The address may instead be `http://localhost:8082/` or `http://servername:8082/`. + - See section address in [Push Mode](/docs/user-guide/targetsource/push/) for more details on URL construction. - *HTTP method*: POST - *HTTP content type*: application/json - *SSL Verification*: true -- *Additional headers:* `Authorization: Bearer thisIsASecureToken` +- *Additional headers:* `Authorization: Bearer YOUR_SECRET_TOKEN` - *Body Template*: ```json @@ -150,7 +151,7 @@ In NetBox, go to `Operations > Webhooks` and create a webhook with the following ] ``` -- *Secret*: `SecretSignature` +- *Secret*: `YOUR_SECRET_SIGNATURE` #### Create Event Rule @@ -180,7 +181,7 @@ Every incoming POST request is logged, including rejected requests. If no POST r --- -### Example: Complete Setup +## Example: Complete Setup Here's a complete example combining all resources: diff --git a/docs/content/docs/user-guide/targetsource/providers/http.md b/docs/content/docs/user-guide/targetsource/providers/http.md index 99bc1b2d..beea1673 100644 --- a/docs/content/docs/user-guide/targetsource/providers/http.md +++ b/docs/content/docs/user-guide/targetsource/providers/http.md @@ -453,6 +453,8 @@ spec: When `push.enabled` is true, the operator accepts incoming webhook notifications and can update targets without polling a remote endpoint. The `url` field is optional when push mode is enabled, but can still be used for polling and fallback behavior. +See [Push mode](/docs/user-guide/targetsource/push/) for more details. + #### Recommended pattern (labels example) ```yaml diff --git a/docs/content/docs/user-guide/targetsource/push.md b/docs/content/docs/user-guide/targetsource/push.md index 9fa35fdf..d7d57cad 100644 --- a/docs/content/docs/user-guide/targetsource/push.md +++ b/docs/content/docs/user-guide/targetsource/push.md @@ -17,11 +17,13 @@ metadata: name: targetsource-1 spec: provider: - http: + http: # can be changed to a differnet TargetSourceProvider push: enabled: true ``` +> `http` is currently the only TargetSourceProvider implemented, once others are added they can be used instead. Push mode is not coupled to a specific TargetSourceProvider implementation. + ## Spec Fields | Field | Type | Required | Default | Description | @@ -44,13 +46,15 @@ See [Push mode with webhook](/docs/examples/netbox/webhook) for an example on ho ## REST API -Refer to the [REST API documentation](/docs/advanced/rest-api-documentation/) for the expected request schema and payload format. +Refer to the [REST API documentation](/docs/advanced/rest-api-documentation/) for the expected request schema and payload format. Any system or script capable of sending HTTP POST requests can integrate with this interface. ## Security -The API supports Bearer Token authentication and X-Hook-Signature, both are optional and turned off by default. They can be used in combination and are enabled by adding them to the specification. +The API supports Bearer Token authentication and X-Hook-Signature, both are optional and **turned off by default**. They are enabled by adding them to the specification. They can also be used in combination. + + An example configuration of both is documented in the [Netbox webhook](/docs/examples/netbox/webhook) example. @@ -80,14 +84,14 @@ This requires the [creation](https://kubernetes.ltd/docs/reference/kubectl/gener - Must be in the same namespace the gNMIc controller runs in. - `name`: refers to the secret name - `key`: key of the secret -- Example: `kubectl create secret generic gnmic-api-auth --from-literal=bearer-token=Secret...` +- Example: `kubectl create secret generic gnmic-api-auth --from-literal=bearer-token=YOUR_SECRET_TOKEN` #### Authorization Header HTTP request must contain the Bearer token in the header in the format: ```yaml -Authorization: Bearer Secret... +Authorization: Bearer YOUR_SECRET_TOKEN ``` ### Signature From a8dae87ccfd3012c86261ba4ffef13eaefba8c40 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Wed, 10 Jun 2026 13:04:10 +0200 Subject: [PATCH 71/76] restructure moustache templates --- .../advanced/openapi-generator-config.yaml | 2 +- .../README.mustache => _index.mustache} | 0 .../api.mustache => apis.mustache} | 0 .../model.mustache => models.mustache} | 0 .../rest-api-documentation/Apis/DefaultApi.md | 22 ++++++------------- .../rest-api-documentation/Models/Target.md | 16 ++++---------- .../advanced/rest-api-documentation/_index.md | 18 +++++---------- 7 files changed, 17 insertions(+), 41 deletions(-) rename docs/content/docs/advanced/openapi-templates/{markdown-documentation/README.mustache => _index.mustache} (100%) rename docs/content/docs/advanced/openapi-templates/{markdown-documentation/api.mustache => apis.mustache} (100%) rename docs/content/docs/advanced/openapi-templates/{markdown-documentation/model.mustache => models.mustache} (100%) diff --git a/docs/content/docs/advanced/openapi-generator-config.yaml b/docs/content/docs/advanced/openapi-generator-config.yaml index 5f041408..75e95512 100644 --- a/docs/content/docs/advanced/openapi-generator-config.yaml +++ b/docs/content/docs/advanced/openapi-generator-config.yaml @@ -3,7 +3,7 @@ generatorName: markdown inputSpec: /local/internal/apiserver/openapi.yaml outputDir: /local/docs/content/docs/advanced/rest-api-documentation -templateDir: /local/docs/content/docs/advanced/openapi-templates/markdown-documentation +templateDir: /local/docs/content/docs/advanced/openapi-templates files: README.mustache: destinationFilename: _index.md \ No newline at end of file diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache b/docs/content/docs/advanced/openapi-templates/_index.mustache similarity index 100% rename from docs/content/docs/advanced/openapi-templates/markdown-documentation/README.mustache rename to docs/content/docs/advanced/openapi-templates/_index.mustache diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache b/docs/content/docs/advanced/openapi-templates/apis.mustache similarity index 100% rename from docs/content/docs/advanced/openapi-templates/markdown-documentation/api.mustache rename to docs/content/docs/advanced/openapi-templates/apis.mustache diff --git a/docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache b/docs/content/docs/advanced/openapi-templates/models.mustache similarity index 100% rename from docs/content/docs/advanced/openapi-templates/markdown-documentation/model.mustache rename to docs/content/docs/advanced/openapi-templates/models.mustache diff --git a/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md index ec3b4c08..a5f0f3d4 100644 --- a/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md +++ b/docs/content/docs/advanced/rest-api-documentation/Apis/DefaultApi.md @@ -1,19 +1,11 @@ ---- -title: "Routes" -linkTitle: "Routes" -weight: 4 -description: > - Available HTTP routes on the gNMIc Operator API interface. ---- +# DefaultApi -# defaultapi - -All URIs are relative to *http://localhost:8082* +All URIs are relative to *http://localhost* | Method | HTTP request | Description | |------------- | ------------- | -------------| -| **applyTargets** | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | -| **getClusterPlan** | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | +| [**applyTargets**](DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | +| [**getClusterPlan**](DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | @@ -26,15 +18,15 @@ Interface for real-time target updates, usually using a webhook. Targets are app |Name | Type | Description | Notes | |------------- | ------------- | ------------- | -------------| -| **Target** | [**List**](/docs/advanced/rest-api-documentation/models/target/)| Target must be passed as a list, multiple targets possible. | | +| **Target** | [**List**](../Models/Target.md)| Target must be passed as a list, multiple targets possible. | | ### Return type -[**List**](/docs/advanced/rest-api-documentation/models/target/) +[**List**](../Models/Target.md) ### Authorization -[signature](/docs/advanced/rest-api-documentation/#signature), [bearerAuth](/docs/advanced/rest-api-documentation/#bearerAuth) +[signature](../README.md#signature), [bearerAuth](../README.md#bearerAuth) ### HTTP request headers diff --git a/docs/content/docs/advanced/rest-api-documentation/Models/Target.md b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md index 86e318a4..18f7031c 100644 --- a/docs/content/docs/advanced/rest-api-documentation/Models/Target.md +++ b/docs/content/docs/advanced/rest-api-documentation/Models/Target.md @@ -1,14 +1,4 @@ ---- -title: "Model" -linkTitle: "Model" -weight: 4 -description: > - Documentation for OpenAPI models and their schema-defined properties. ---- - # Target -Network device to be monitored. Properties not marked as optional must be in JSON body. - ## Properties | Name | Type | Description | Notes | @@ -17,6 +7,8 @@ Network device to be monitored. Properties not marked as optional must be in JSO | **address** | **String** | IPv4/IPv6 address or hostname. | [default to null] | | **port** | **Integer** | gNMIc port. | [optional] [default to null] | | **targetProfile** | **String** | TargetProfile applied to apply to this router. | [optional] [default to null] | -| **labels** | **List** | Labels must be map[string]string. For example vendor:nokia | [optional] [default to null] | -| **operation** | **String** | Either `created`, `updated` or `deleted`. `created` and `updated` are identical and both apply the target. | [default to null] | +| **labels** | [**List**](map.md) | Labels must be map[string]string. For example vendor:nokia. | [optional] [default to null] | +| **operation** | **String** | Either `created`, `updated` or `deleted`. `created` and `updated` are identical and both apply the target. | [default to null] | + +[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md) diff --git a/docs/content/docs/advanced/rest-api-documentation/_index.md b/docs/content/docs/advanced/rest-api-documentation/_index.md index 93b2d678..a552b56a 100644 --- a/docs/content/docs/advanced/rest-api-documentation/_index.md +++ b/docs/content/docs/advanced/rest-api-documentation/_index.md @@ -1,33 +1,25 @@ ---- -title: "REST API interface" -linkTitle: "REST API interface" -weight: 3 -description: > - This document describes the REST API exposed by the gNMIc Operator, including the available endpoints, request formats, and usage examples. ---- +# Documentation for gNMIc Operator REST API ## Documentation for API Endpoints -All URIs are relative to *http://localhost:8082* +All URIs are relative to *http://localhost* | Class | Method | HTTP request | Description | |------------ | ------------- | ------------- | -------------| -| *defaultapi* | [**applyTargets**](/docs/advanced/rest-api-documentation/apis/defaultapi) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | -*defaultapi* | [**getClusterPlan**](/docs/advanced/rest-api-documentation/apis/defaultapi) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | +| *DefaultApi* | [**applyTargets**](Apis/DefaultApi.md#applyTargets) | **POST** /api/v1/:namespace/target-source/:name/applyTargets | Interface for real-time target updates, usually using a webhook. Targets are applied in the gNMIc Operator. | +*DefaultApi* | [**getClusterPlan**](Apis/DefaultApi.md#getClusterPlan) | **GET** /clusters/:namespace/:name/plan | Get cluster plan. | ## Documentation for Models - - [target](/docs/advanced/rest-api-documentation/models/target/) + - [Target](./Models/Target.md) ## Documentation for Authorization -For a detailed explanation on how to configure the required secrets within the gNMIc Operator, refer to [TargetSource > Push mode](/docs/user-guide/targetsource/push/). - ### bearerAuth From ea113aebdaee2cfff354a156a2591be395aebb98 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Wed, 10 Jun 2026 13:21:25 +0200 Subject: [PATCH 72/76] upate cluster address section --- docs/content/docs/user-guide/targetsource/push.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource/push.md b/docs/content/docs/user-guide/targetsource/push.md index d7d57cad..df93839c 100644 --- a/docs/content/docs/user-guide/targetsource/push.md +++ b/docs/content/docs/user-guide/targetsource/push.md @@ -38,12 +38,21 @@ spec: The REST API endpoint runs on `http://cluster-address:8082/api/v1/:namespace/target-source/:name/applyTargets`. -- `cluster-address`: Adress of your cluster, localhost during development. +- `cluster-address`: Address of your cluster. - `:namespace`: Namespace the TargetSource is created in. - `:name`: Name of the TargetSource. See [Push mode with webhook](/docs/examples/netbox/webhook) for an example on how to configure the URL. +### Cluster Address + +The cluster address depends on where the API is accessed from. + +- Use `http://:8082/` when accessing the API from outside the cluster. +- Use `http://localhost:8082/` for local development (requires port-forwarding). +- Use `gnmic-controller-manager-api.gnmic-system.svc.cluster.local` when NetBox (or another source of truth) runs in the same cluster. +- If you use a reverse proxy, run `kubectl get service -n ` and use the returned service address and port in your proxy configuration. + ## REST API Refer to the [REST API documentation](/docs/advanced/rest-api-documentation/) for the expected request schema and payload format. @@ -54,8 +63,6 @@ Any system or script capable of sending HTTP POST requests can integrate with th The API supports Bearer Token authentication and X-Hook-Signature, both are optional and **turned off by default**. They are enabled by adding them to the specification. They can also be used in combination. - - An example configuration of both is documented in the [Netbox webhook](/docs/examples/netbox/webhook) example. ### Bearer Authentication From 0d4d01c1a5305e9ed758ceecef75cea4009d0dcb Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Wed, 10 Jun 2026 13:42:06 +0200 Subject: [PATCH 73/76] add lines, fix title --- .../docs/user-guide/targetsource/push.md | 20 +++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/docs/content/docs/user-guide/targetsource/push.md b/docs/content/docs/user-guide/targetsource/push.md index df93839c..7b2e3bf2 100644 --- a/docs/content/docs/user-guide/targetsource/push.md +++ b/docs/content/docs/user-guide/targetsource/push.md @@ -1,6 +1,6 @@ --- -title: "Push mode" -linkTitle: "Push mode" +title: "Push Mode" +linkTitle: "Push Mode" weight: 4 description: > Enables REST API interface that accepts real-time target updates. @@ -24,6 +24,8 @@ spec: > `http` is currently the only TargetSourceProvider implemented, once others are added they can be used instead. Push mode is not coupled to a specific TargetSourceProvider implementation. +--- + ## Spec Fields | Field | Type | Required | Default | Description | @@ -34,6 +36,8 @@ spec: | `signature` | object | No | - | HTTP body verification using HMAC | | `algorithm` | string | No | sha512 | Algorithm for signature verification(`sha256`or`sha512`) | +--- + ## Address The REST API endpoint runs on `http://cluster-address:8082/api/v1/:namespace/target-source/:name/applyTargets`. @@ -53,11 +57,13 @@ The cluster address depends on where the API is accessed from. - Use `gnmic-controller-manager-api.gnmic-system.svc.cluster.local` when NetBox (or another source of truth) runs in the same cluster. - If you use a reverse proxy, run `kubectl get service -n ` and use the returned service address and port in your proxy configuration. +--- + ## REST API -Refer to the [REST API documentation](/docs/advanced/rest-api-documentation/) for the expected request schema and payload format. +Refer to the [REST API documentation](/docs/advanced/rest-api-documentation/) for the expected request schema and payload format. Any system or script capable of sending HTTP POST requests can integrate with this interface. -Any system or script capable of sending HTTP POST requests can integrate with this interface. +--- ## Security @@ -65,6 +71,8 @@ The API supports Bearer Token authentication and X-Hook-Signature, both are opti An example configuration of both is documented in the [Netbox webhook](/docs/examples/netbox/webhook) example. +--- + ### Bearer Authentication Bearer authentication compares a token stored in Kubernetes with the one sent in the HTTP header. The Kubernetes secret is referenced as `tokenSecretRef`. @@ -101,6 +109,8 @@ HTTP request must contain the Bearer token in the header in the format: Authorization: Bearer YOUR_SECRET_TOKEN ``` +--- + ### Signature Signature verification requires an Opaque Kubernetes secret that stores the shared key (see Bearer Authentication). For each request, the HMAC generated from the request body and shared key must be provided in the `X-Hook-Signature` header. @@ -119,6 +129,8 @@ spec: key: signature ``` +--- + #### Reverse Proxy In order to have a secure setup, the HTTP post requests must be sent using TLS. The REST API interface does not support HTTPS, at least not directly. It is recommended to terminate the TLS connection at the reverse proxy and forward a plain HTTP request to the gNMIc Operator. From dde381ab7ed0d211f714080c01c26b587e5f0990 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Wed, 10 Jun 2026 17:30:58 +0200 Subject: [PATCH 74/76] updates after example playthrough --- .../docs/examples/NetBox/webhook/_index.md | 21 +++++++++---------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/docs/content/docs/examples/NetBox/webhook/_index.md b/docs/content/docs/examples/NetBox/webhook/_index.md index 01e69eb3..5cc0d316 100644 --- a/docs/content/docs/examples/NetBox/webhook/_index.md +++ b/docs/content/docs/examples/NetBox/webhook/_index.md @@ -18,7 +18,7 @@ This example walks through configuring a webhook in NetBox to push real-time tar b: Create Event Rule 5. Verification -At the end, the logs will show the incoming POST requests and the targets are in status `READY`. +At the end, the logs will show the incoming POST requests and the targets updates can be verified with `kubectl get targets`. ## Prerequisites @@ -79,10 +79,10 @@ kubectl create secret generic gnmic-signature --from-literal=signature=YOUR_SECR ### 3. Apply TargetSource -Apply the TargetSource manifest: `kubectl apply -f netbox.yaml -n gnmic-system` +The TargetSource has the following settings configured: -- `enabled` must be set to `true`, otherwise updates are rejected. -- Bearer authentication and signature verification are enabled. +- `spec.provider.http.push.enabled` must be set to `true`, otherwise updates are rejected. +- Bearer authentication and signature verification are enabled, referencing to the secrets created in step 2. ```yaml # netbox.yaml @@ -124,15 +124,13 @@ Next, configure a webhook in NetBox. The webhook is triggered by device events ( In NetBox, go to `Operations > Webhooks` and create a webhook with the following settings: -- *Name*: GNMIc operator push +- *Name*: gNMIc Operator push - *URL*: `http://gnmic-controller-manager-api.gnmic-system.svc.cluster.local:8082/api/v1/gnmic-system/target-source/netbox/applyTargets` - - URL contains the namespace `gnmic-system` and TargetSource name `netbox`. + - URL contains the namespace `gnmic-system` and TargetSource name `netbox`. See section address in [Push Mode](/docs/user-guide/targetsource/push/) for more details on URL construction. - `gnmic-controller-manager-api.gnmic-system.svc.cluster.local` is only reachable if Netbox is inside the cluster. - The address may instead be `http://localhost:8082/` or `http://servername:8082/`. - - See section address in [Push Mode](/docs/user-guide/targetsource/push/) for more details on URL construction. - *HTTP method*: POST - *HTTP content type*: application/json -- *SSL Verification*: true - *Additional headers:* `Authorization: Bearer YOUR_SECRET_TOKEN` - *Body Template*: @@ -145,21 +143,22 @@ In NetBox, go to `Operations > Webhooks` and create a webhook with the following "targetProfile": "{{ data.custom_fields.target_profile | default('', true) }}", "port": {{ data.custom_fields.gnmic_port | default(57400, true) }}, "labels": [ - "vendor":"{{ data.device_type.manufacturer.name }}" + {"vendor":"{{ data.device_type.manufacturer.name }}"} ] } ] ``` - *Secret*: `YOUR_SECRET_SIGNATURE` +- *SSL Verification*: true #### Create Event Rule The webhook requires a trigger, configured as an event rule under `Operations > Event Rules`. - *Name*: gNMIc Operator push target change -- *Object types*: DCIM > Device -- *Event types*: "Object Created", "Object Updated", "Object Deleted" +- *Object types*: `DCIM > Device` +- *Event types*: `Object Created`, `Object Updated` and `Object Deleted` - *Action type*: Webhook - *Webhook*: gNMIc Operator push From 979d7e6f75cc35e96eb2ae51d32718407d718197 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Wed, 10 Jun 2026 17:47:02 +0200 Subject: [PATCH 75/76] run pipeline again --- docs/content/docs/user-guide/targetsource/push.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/docs/user-guide/targetsource/push.md b/docs/content/docs/user-guide/targetsource/push.md index 7b2e3bf2..ef8d77d3 100644 --- a/docs/content/docs/user-guide/targetsource/push.md +++ b/docs/content/docs/user-guide/targetsource/push.md @@ -133,4 +133,4 @@ spec: #### Reverse Proxy -In order to have a secure setup, the HTTP post requests must be sent using TLS. The REST API interface does not support HTTPS, at least not directly. It is recommended to terminate the TLS connection at the reverse proxy and forward a plain HTTP request to the gNMIc Operator. +In order to have a secure setup, the HTTP post requests must be sent using TLS. The REST API interface does not support HTTPS, at least not directly. It is recommended to terminate the TLS connection at the reverse proxy and forward a HTTP request to the gNMIc Operator. From cb717ef8dbeb8593faa1e7f3c45c1e7b961ec075 Mon Sep 17 00:00:00 2001 From: Roman Weber Date: Wed, 10 Jun 2026 17:51:14 +0200 Subject: [PATCH 76/76] remove folders from .gitignore --- .gitignore | 2 -- 1 file changed, 2 deletions(-) diff --git a/.gitignore b/.gitignore index 743dab8a..9028be6b 100644 --- a/.gitignore +++ b/.gitignore @@ -34,5 +34,3 @@ docs/public docs/resources/_gen/ docs/.hugo_build.lock test/integration/clab-* -docs/content/docs/advanced/rest-api-documentation/.openapi-generator -docs/content/docs/advanced/rest-api-documentation/.openapi-generator-ignore