new env config - #8
Conversation
|
Most
|
| def __getEnvConfigsDict(self): | ||
| """Build a nested config dict from CONFIG__-prefixed environment variables. | ||
|
|
||
| ``CONFIG__db__host=localhost`` becomes ``{"db": {"host": "localhost"}}``. | ||
|
|
||
| Returns: | ||
| dict: nested dictionary of configuration options | ||
| """ | ||
| config = {} | ||
| keys = [] | ||
| for key, value in os.environ.items(): | ||
| if not key.startswith("CONFIG__"): | ||
| continue | ||
| # to be logged | ||
| keys.append(key) | ||
| try: | ||
| key1, key2 = key.removeprefix("CONFIG__").split("__") | ||
| except ValueError: | ||
| logger.warning("Skipping %s", key) | ||
| continue | ||
| config.setdefault(key1, {})[key2] = value | ||
| logger.info("Found environment configs: %s", keys) | ||
| return config |
There was a problem hiding this comment.
I'm not a huge fan of this approach... It seems very hackish to rely on the specific naming structure of environmental variables. This could lead to confusion down the road (e.g., if someone were to see a weird envvar like CONFIG__db__host and think, "This seems unnecessarily long—let me clean it up"), as well as leaves open the chance of collisions with some other pre-existing envvar that happens to take on that naming structure.
Maybe with something more specific like EXDBCONFIG__..., then I'd feel better, but still it seems quite hackish.
I'd like to take a moment to think about some other ways we could possibly approach this, though...
There was a problem hiding this comment.
I'm happy to go a different route. I'm having trouble finding a nice way to do it though. We could change the code so its passed by cli for every task but these values are generally passed down the chain individually, instead of a dict. etl_tree_node_lists passes over to TreeNodeListWorker passes to DocumentLoader passes to Connection() which then pulls it in from self.__cfgOb.
This is a fine course of action but its just a lot of code changes across a lot of repos.
|
@piehld CONFIG_SUPPORT_TOKEN is pulled in directly from the env, but its a good idea to just check the value to see if its an env var. Now in this PR when a value is requested it will check if it starts with |
2 participants
No description provided.