When developing an application, it is highly likely that it needs to be configured with all sorts of settings. A simple list of key/value pairs is usually all you need. Some of the values can be read by anyone (e.g., a public URL) while some values should be treated as secrets (e.g., a connection string).
Azure App Configuration is a service to centrally manage these settings in addition to feature flags. In this post, we will look at storing and retrieving application settings and keeping feature flags for another time. I will also say App Config instead of App Configuration to save some keystrokes. 😉
We will do the following:
- Retrieve key-value pairs for multiple applications and environments from one App Config instance
- Use Key Vault references in App Config and retrieve these from Key Vault directly
- Use the Python provider client to retrieve key-value pairs and store them in a Python dictionary
Why use App Configuration at all?
App Configuration helps by providing a fully managed service to store configuration settings for your applications separately from your code. Storing configuration separate from code is a best practice that most developers should follow.
Although you could store configuration values in files, using a service like App Config provides some standardization within or across developer teams.
Some developers store both configuration values and secrets in Key Vault. Although that works, App Config is way more flexible in organizing the settings and retrieving lists of settings with key and label filters. If you need to work with more than a few settings, I would recommend using a combination of App Config and Key Vault.
In what follows, I will show how we store settings for multiple applications and environments in the same App Config instance. Some of these settings will be Key Vault references.
Read https://learn.microsoft.com/en-us/azure/azure-app-configuration/overview before continuing to know more about App Config.
Provisioning App Config
Provisioning App Configuration is very easy from the portal or the Azure CLI. With the Azure CLI, use the following commands to create a resource group and an App Configuration instance in that group:
az group create -n RESOURCEGROUP -l LOCATION
az appconfig create -g RESOURCEGROUP -n APPCONFIGNAME -l LOCATIONAfter deployment, we can check the portal and navigate to Configuration Explorer.
In Configuration Explorer, you can add the configuration values for your apps. They are just key/value pairs but they can be further enriched with labels, content types, and tags.
Note that there is a Free and a Standard tier of App Config. See https://azure.microsoft.com/en-us/pricing/details/app-configuration/ for more information. In production, you should use the Standard tier.
Storing configuration and secrets for multiple apps and environments
To store configuration values for multiple applications, you will have to identify the application in the key. App Configuration, oddly, has no knowledge of applications. For example, a key could be app1:setting1. You decide on the separator between the app name (app1 here) and its setting (setting1). In your code, you can easily query all settings for your app with a key filter (e.g. “app1:”. I will show an example of using a key filter later with the Python provider.
If you want to have different values for a key per environment (like dev, prd, etc…), you can add a label for each environment. To retrieve all settings for an environment, you can use a label filter. I will show an example of using a label filter later.
Suppose you want to use app1:setting1 in two environments: dev and prd. How do you create the key-value pairs? One way is to use the Azure CLI. You can also create them with the portal or from Python, C#, etc… With the CLI:
az appconfig kv set --name APPCONFIGNAME --key app1:setting1 --value "value1" --label devAPPCONFIG name is the name of your App Config instance. Just the name, not the full URL. For the prd environment:
az appconfig kv set --name APPCONFIGNAME --key app1:setting1 --value "value2" --label prdIn Configuration Explorer, you will now see:
For more examples of using the Azure CLI, see https://learn.microsoft.com/en-us/azure/azure-app-configuration/scripts/cli-work-with-keys.
In addition to these plain key-value pairs, you can also create Key Vault references. Let’s create one from the portal. In Configuration Explorer, click + Create and select Key Vault reference. You will get the following UI that allows you to create the reference. Make sure you have a Key Vault with a secret called dev-mysecret if you want to follow along. Below, set the label to dev. I forgot that in the screenshot below:
Above, I am using the same naming convention for the key in App Config: app1:mysecret. Notice though that the secret I am referencing in Key Vault contains the environment and a dash (-) before the actual secret name. If you use one Key Vault per app instead of a Key Vault per app and environment, you will have to identify the environment in the secret name in some way.
After creating the reference, you will see the following in Configuration explorer:
Note that the Key Vault reference has a content type. The content type is application/vnd.microsoft.appconfig.keyvaultref+json;charset=utf-8. You can use the content type in your code to know if the key contains a reference to a Key Vault secret. That reference will be something like https://kv-app1-geba.vault.azure.net/secrets/dev-mysecret. You can then use the Python SDK for Azure Key Vault to retrieve the secret from your code. Azure App Config will not do that for you.
You can use content types in other ways as well. For example, you could store a link to a storage account blob and use a content type that informs your code it needs to retrieve the blob from the account. Of course, you will need to write code to retrieve the blob. App Config only contains the reference.
Reading settings
There are many ways to read settings from App Config. If you need them in an Azure Pipeline, for instance, you can use the Azure App Configuration task to pull keys and values from App Config and set them as Azure pipeline variables.
If you deploy your app to Kubernetes and you do not want to read the settings from your code, you can integrate App Configuration with Helm. See https://learn.microsoft.com/en-us/azure/azure-app-configuration/integrate-kubernetes-deployment-helm for more information.
In most cases though, you will want to read the settings directly from your code. There is an SDK for several languages, including Python. The SDK has all the functionality you need to read and write settings.
Next to the Python SDK, there is also a Python provider which is optimized to read settings from App Config and store them in a Python dictionary. The provider has several options to automatically trim app names from keys and to automatically retrieve a secret from Key Vault if the setting in App Config is a Key Vault reference.
To authenticate to App Config, the default is access keys with a connection string. You can find the connection string in the Portal:
You can also use Azure AD (it’s always enabled) and disable access keys. In this example, I will use a connection string to start with:
Before we connect and retrieve the values, ensure you install the provider first:
pip install azure-appconfiguration-provider
Above, use pip or pip3 depending on your installation of Python.
In your code, ensure the proper imports:
from azure.appconfiguration.provider import (
AzureAppConfigurationProvider,
SettingSelector,
AzureAppConfigurationKeyVaultOptions
)
from azure.keyvault.secrets import SecretClient
from azure.identity import DefaultAzureCredential
To authenticate to Azure Key Vault with Azure AD, we can use DefaultAzureCredential():
try:
CREDENTIAL = DefaultAzureCredential(exclude_visual_studio_code_credential=True)
except Exception as ex:
print(f"error setting credentials: {ex}")
Note: on my machine, I had an issue with the VS Code credential feature so I turned that off.
Next, use a SettingSelector from the provider to provide a key filter and label filter. I want to retrieve key-value pairs for an app called app1 and an environment called dev:
app = 'app1'
env = 'dev'
selects = {SettingSelector(key_filter=f"{app}:*", label_filter=env)}
Next, when I retrieve the key-value pairs, I want to strip app1: from the keys:
trimmed_key_prefixes = {f"{app}:"}
In addition, I want the provider to automatically go to Key Vault and retrieve the secret:
key_vault_options = AzureAppConfigurationKeyVaultOptions(secret_resolver=retrieve_secret)
retrieve_secret refers to a function you need to write to retrieve the secret and add custom logic. There are other options as well.
def retrieve_secret(uri):
try:
# uri is in format: https://<keyvaultname>.vault.azure.net/secrets/<secretname>
# retrieve key vault uri and secret name from uri
vault_uri = "https://" + uri.split('/')[2]
secret_name = uri.split('/')[-1]
print(f"Retrieving secret {secret_name} from {vault_uri}...")
# retrieve the secret from Key Vault; CREDENTIAL was set globally
secret_client = SecretClient(vault_url=vault_uri, credential=CREDENTIAL)
# get secret value from Key Vault
secret_value = secret_client.get_secret(secret_name).value
except Exception as ex:
print(f"retrieving secret: {ex}", 1)
return secret_value
Now that we have all the options, we can retrieve the key-value pairs.
connection_string = 'YOURCONNSTR'
app_config = AzureAppConfigurationProvider.load(
connection_string=connection_string, selects=selects, key_vault_options=key_vault_options,
trimmed_key_prefixes=trimmed_key_prefixes)
print(app_config)
Now we have a Python dictionary app_config with all key-value pairs for app1 and environment dev. The key-value pairs are a mix of plain values from App Config and Key Vault.
You can now use this dictionary in your app in whatever way you like.
If you would like to use the same CREDENTIAL to connect to App Config, you can also use:
endpoint = 'APPCONFIGNAME.azconfig.io' # no https://
app_config = AzureAppConfigurationProvider.load(
endpoint=endpoint, credential=CREDENTIAL, selects=selects, key_vault_options=key_vault_options,
trimmed_key_prefixes=trimmed_key_prefixes)
Ensure the credential you use has the App Configuration Data Reader role to read the key-value pairs.
Here’s all the code in a gist: https://gist.github.com/gbaeke/9b075a87a1198cdcbcc2b2028492085b. Ensure you have the key-value pairs as above and provide the connection string to the connection_string variable.
Conclusion
In this post, we showed how to retrieve key-value pairs with the Python provider from one App Config instance for multiple applications and environments.
The application is stored as a prefix in the key (app1:). The environment is a label (e.g., dev), allowing us to have the same setting with different values per environment.
Some keys can contain a reference to Key Vault to allow your application to retrieve secrets from Key Vault as well. I like this approach to have a list of all settings for an app and environment, where the value of the key can be an actual value or a reference to some other entity like a secret, a blob, or anything else.
baeke.info 