Skip to content

cleanup docs around glue local executors #710

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
HarshCasper wants to merge 1 commit into
base: main
Choose a base branch
from
Open

cleanup docs around glue local executors #710

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion src/content/docs/aws/capabilities/config/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,6 @@ Also see [OpenSearch configuration variables](#opensearch) which are used to man

| Variable | Example Values | Description |
| - | - | - |
| `GLUE_JOB_EXECUTOR_PROVIDER` (**deprecated**) | `v1` \| `v2` (default) | Feature flag to switch between the old and new, default Glue job executor providers. |
| `GLUE_JOB_EXECUTOR` | `docker` (default) \| `kubernetes` | Whether to run Glue jobs when LocalStack is deployed on Kubernetes. Jobs are run as pods in the Kubernetes cluster. |
| `DOCKER_GLOBAL_IMAGE_PREFIX` | | Specify custom images for Glue jobs by configuring their custom image repository. |

Expand Down
2 changes: 1 addition & 1 deletion src/content/docs/aws/enterprise/kubernetes/kubernetes-executor.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ You can run the following services on Kubernetes clusters using the LocalStack E

- [DocumentDB](/aws/services/docdb)
- [MWAA](/aws/services/docdb)
- [Glue](/aws/services/glue) (requires setting `GLUE_JOB_EXECUTOR_PROVIDER` to `v2`)
- [Glue](/aws/services/glue)
- [RDS](/aws/services/rds) ([MySQL](/aws/services/rds/#mysql-engine) & [MSSQL](/aws/services/rds/#microsoft-sql-server-engine))

To use Kubernetes as the runtime backend, set the `CONTAINER_RUNTIME` configuration variable to `kubernetes`.
Expand Down
4 changes: 1 addition & 3 deletions src/content/docs/aws/services/glue.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,9 @@ import FeatureCoverage from '../../../../components/feature-coverage/FeatureCove
The Glue API in LocalStack for AWS allows you to run ETL (Extract-Transform-Load) jobs locally, maintaining table metadata in the local Glue data catalog, and using the Spark ecosystem (PySpark/Scala) to run data processing workflows.

LocalStack allows you to use the Glue APIs in your local environment.
LocalStack uses a container-based Glue job executor, running Glue jobs within a Docker environment (or as pods when deployed on Kubernetes).
The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Glue's integration with LocalStack.

:::note
As of LocalStack 4.13.0, the default Glue job executor provider is `v2`, meaining LocalStack will use the container-based Glue Job executor, enabling Glue jobs to run within a Docker environment. The `GLUE_JOB_EXECUTOR_PROVIDER` configuration variable that enables the use of the legacy Glue job executor provider is deprecated and will be removed in the next major release.
:::

## Getting started

Expand Down