Configuring ASP.NET Core In AWS

Creating a Custom Configuration Provider for AWS Systems Manager Parameter Store

by JamesQMurphy | May 20, 2020

When I looked back at my earlier posts about building this site, I realized that I never talked about configuration. After all, this is a blog about DevOps, and configuration is central to deployments. So why wasn't I talking about configuration?

Part of the reason was that, at the time, there was very little configuration to speak of. The first few versions of this site were strictly content; the blog posts were flat files, there was no user login, etc. But the main reason was that I didn't fully understand how configuration worked in ASP.NET Core. I knew there was an appsettings.json file, but I treated it merely as a JSON-formatted version of classic ASP.NET's Web.config file. It wasn't until much later, when my configuration needs became more sophisticated, that I needed to dive in and see how it worked. The end result was a custom configuration provider for the AWS Systems Manager Parameter Store, and a deeper appreciation for ASP.NET Core configuration in general.

Most of what I learned came from Rick Anderson and Kirk Larkin's excellent article on ASP.NET configuration. It's definitely worth a read.

Hierarchical Configuration Data

When I first started adding configuration data to the website, I added it the only way I knew how -- as simple key-value pairs in the root node of the JSON document:

{
...
  "WebSiteTitle": "Cold-Brewed DevOps",
  "AppName": "JamesQMurphyWeb-Local",
  "ImageBasePath":  "/blogimages",
  "WarmUrl": "/warm",
  "AllowedHosts": "*",
  "DataProtection": "",
  "UseStaticFiles": "true",
...
}  

The "flat" approach worked well enough for the simple configuration items that I needed. But I knew that configuration could be set up hierarchically. Indeed, the default "Logging" configuration is actually hierarchical:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information"
    }
  },
...
}

As a classic .NET developer used to the simple key-value configuration model, this hierarchical configuration approach threw me off at first. But the answer was refreshingly simple: the configuration data is still stored as a dictionary of key-value pairs. The article I mentioned says it best:

The Configuration API reads hierarchical configuration data by flattening the hierarchical data with the use of a delimiter in the configuration keys.

So the hierarchy structure is stored in the keys themselves with the help of a delimiter, which by the way is a colon (:). Suppose you have the following in your appsettings.json:

{
    "TopLevelKey" : "value for TopLevelKey",
    "MySubnode" : {
        "DeepKey1" : "value for DeepKey1",
        "DeepKey2" : "value for DeepKey2"
    }
}

The underlying configuration dictionary would contain these keys and values:

Key Value
TopLevelKey value for TopLevelKey
MySubnode:DeepKey1 value for DeepKey1
MySubnode:DeepKey2 value for DeepKey2

Note that key lookup is not case-sensitive, even on Linux platforms. Specifying a key of either MySubnode:DeepKey1 or mysuBnODE:dEePkEy1 will return the same value ("value for DeepKey1" in this case).

It's also worth mentioning that arrays in appsettings.json are expanded into key-value pairs. If we add an array to the previous example:

{
    "TopLevelKey" : "value for TopLevelKey",
    "MySubnode" : {
        "DeepKey1" : "value for DeepKey1",
        "DeepKey2" : "value for DeepKey2",
        "MyArray" : [ "first", "second", "third" ]
    }
}

The resulting configuration dictionary would contain the following entries:

Key Value
TopLevelKey value for TopLevelKey
MySubnode:DeepKey1 value for DeepKey1
MySubnode:DeepKey2 value for DeepKey2
MySubnode:MyArray:0 first
MySubnode:MyArray:1 second
MySubnode:MyArray:2 third

Configuration Binding

Besides allowing better organization, the hierarchical structure of the config file allows ASP.NET Core to bind objects to individual sections of the configuration. Suppose we wanted to bind the MySubnode section of the previous configuration to an object. We can define a class with properties (not fields) like this:

class MyConfig
{
   public string DeepKey1 {get; set;}
   public string DeepKey2 {get; set;}
   public string[] MyArray {get; set;}
}

and then bind to it like this:

var myConfigInstance = Configuration
                        .GetSection("MySubnode")
                        .Get<MyConfig>();

The result is an object of type MyConfig that contains the values from the configuration data.

Overriding Configuration Values with Environment Variables

Another feature of ASP.NET Core configuration that was new to me was that environment variables could be used to override configuration values. However, it made sense, since environment variables are one of the primary means of configuring Docker containers and Lambda functions. By default, all environment variables are added to the configuration data dictionary as key-value pairs. If any environment variables have the same name as an existing configuration key, the value is replaced with the value from the environment.

Suppose the configuration file above is part of an application running inside a Docker container, and you run the container with an environment variable named TopLevelKey:

docker run --env TopLevelKey="overridden" mycontainer

Even with the same appsettings.json file, the resulting configuration data for the app in the container would look like this:

Key Value
TopLevelKey overridden
MySubnode:DeepKey1 value for DeepKey1
MySubnode:DeepKey2 value for DeepKey2
MySubnode:MyArray:0 first
MySubnode:MyArray:1 second
MySubnode:MyArray:2 third

But what if you wanted to override a value deep in the hierarchy? Environment variables cannot have colons in their name, but you can use a double underscore in place of a colon:

docker run --env TopLevelKey="overridden" --env MySubnode__DeepKey2="also overridden" mycontainer
Key Value
TopLevelKey overridden
MySubnode:DeepKey1 value for DeepKey1
MySubnode:DeepKey2 also overridden
MySubnode:MyArray:0 first
MySubnode:MyArray:1 second
MySubnode:MyArray:2 third

Default Configuration Providers

Earlier, I said that all environment variables are added to the configuration data dictionary by default. More precisely, it happens if your application calls Host.CreateDefaultBuilder(), which most apps do since the standard .NET Core templates generate the following code:

public class Program
{
    public static void Main(string[] args)
    {
        CreateHostBuilder(args).Build().Run();
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureWebHostDefaults(webBuilder =>
            {
                webBuilder.UseStartup<Startup>();
            });
}

In addition to the appsettings.json file and the environment variables, CreateDefaultBuilder() sets up several configuration providers. According to the documentation, it sets up the sources in the following order:

  1. appsettings.json
  2. appsettings.{environment}.json, where {environment} is the value of the environment variable ASPNETCORE_ENVIRONMENT, or the value "Production" if the variable is not present. If you run your application using Visual Studio, it will set this value to "Development", which causes appsettings.Development.json to be loaded.1
  3. The local development Secrets Manager, but only if ASPNETCORE_ENVIRONMENT is equal to "Development". This is used to keep passwords and API Keys outside of your source code repository.
  4. Environment variables, as described above
  5. The command line (details are available here)

Sources further down the list override the sources above them. So an environment variable will override a config value specified in appsettings.json, but a value supplied on the command line will override an environment variable.

Configuration and Secrets

There's a limitation with appsettings.json and environment variables -- neither one of them is secure. appsettings.json contains unencrypted values at rest, right next to your application. Environment variables are marginally better, since they don't exist at rest, but they're still unencrypted and not the best choice for things like the client secrets needed for Twitter and GitHub authentication. For this site, I use the AWS Systems Manager Parameter Store to securely store the secrets.

The AWS Parameter Store also stores its data as key-value pairs, and it, too, has a hierarchical delimiter (a forward slash). Consider this snapshot of my own Parameter Store:

Screenshot of AWS Parameter Store

The configuration parameters are organized hierarchically, using forward slashes in the key names. All of the keys are organized under the /AppSettings/JamesQMurphyWeb-xxx/ prefix, to keep them separated from other configuration data (there's only one Parameter Store per AWS Account). Translating the key names is simply a matter of chopping off the common prefix and replacing the remaining forward slashes with colons:

AWS Parameter Store Name ASP.NET Core Configuration Key
/AppSettings/JamesQMurphyWeb-dev/Authentication/Twitter/ConsumerAPIKey Authentication:Twitter:ConsumerAPIKey
/AppSettings/JamesQMurphyWeb-dev/Authentication/Twitter/ConsumerSecret Authentication:Twitter:ConsumerSecret
/AppSettings/JamesQMurphyWeb-dev/WebSiteOptions/WebSiteTitle WebSiteOptions:WebSiteTitle

Initially, this was how I was adding the values to configuration. (This code was inside my Startup.cs file.):

// Read all of the secrets from Parameter Store
var keys = new List<string> { /* list of keys */ }
var response = ssmClient.GetParametersAsync(
    new GetParametersRequest
    {
        Names = keys,
        WithDecryption = true
    }
).GetAwaiter().GetResult();

// Insert the secrets directly into configuration
foreach(var p in response.Parameters)
{
    Configuration[p.Name.Substring(basePathLength).Replace('/',':')] = p.Value;
}

It worked, but besides being a chore to maintain, it didn't integrate naturally with ASP.NET Core's configuration model. What I really needed was a custom configuration provider for the AWS Parameter Store.

Custom Configuration Providers

Writing a custom configuration provider isn't that difficult once you understand how ASP.NET Core handles configuration data. A custom configuration provider's job is to load configuration data from a particular source and translate it into the hierarchical key-value structure that ASP.NET Core expects. It involves implementing two classes: a configuration source and a configuration provider.

The configuration source class implements the IConfigurationSource interface, which contains a single method named Build(). Essentially, it's a factory class for the provider and lets you specify what parameters you need to construct the provider. I'll discuss the configuration source class in my next blog post.

The configuration provider is the class that does the actual work of retrieving the configuration data and translating it into ASP.NET Core's hierarchical key-value structure. It needs to implement the IConfigurationProvider interface:

public interface IConfigurationProvider
{
  IEnumerable<string> GetChildKeys(IEnumerable<string> earlierKeys, string ParentPath);
  IChangeToken GetReloadToken();
  void Load();
  void Set(string key, string value);
  bool TryGet(string key, out string value);
}

To retrieve a specific configuration value, ASP.NET will call the TryGet function of your configuration provider. If the provider can supply a value, it returns true and returns the value in the value out parameter. If the provider cannot return the value, it returns false. Simple enough. There is also a Set function for completeness, but for a custom configuration provider, it doesn't serve any real purpose and can be left with an empty implementation.

To bind to a class, ASP.NET will call the GetChildKeys to get a list of the keys that can be returned, then it calls TryGet for each key to get the corresponding value. It's up to the provider to return the key names using the proper delimiter (a colon, or the ConfigurationPath.KeyDelimiter static field if you prefer.) Note that that the provider must append its list of keys to the list of keys passed in through the earlierKeys parameter (which is, frankly, a little odd). Oh, and ParentPath can be null.

The Load function is called once, after the class is instantiated, to give the custom provider an opportunity to load all the data upfront. The GetReloadToken function provides an IChangeToken that the client can use to detect configuration changes. If the provider doesn't support it, it still needs to return a valid token; you can't return null.

My Initial Implementation

My first try looked something like this. I simply mapped GetChildKeys to the Parameter Store's GetParametersByPath command, and TryGet to GetParameter:

// First attempt -- do not use
public IEnumerable<string> GetChildKeys(IEnumerable<string> earlierKeys, string parentPath)
{
    var listEarlierKeys = new List<string>(earlierKeys);
    try
    {
        string nextToken = default;
        do {
            var response = ssmClient.GetParametersByPathAsync(
                new GetParametersByPathRequest
                {
                    Path = $"{configSrc.BasePath}{ConvertToSMSHeirarchy(parentPath)}",
                    WithDecryption = true,
                    NextToken = nextToken
                }).GetAwaiter().GetResult();
            listEarlierKeys.AddRange(response.Parameters.Select(p => p.Name));
            nextToken = response.NextToken;
        } while (!String.IsNullOrEmpty(nextToken));

        return listEarlierKeys;
    }
    catch (Amazon.Runtime.AmazonServiceException) {return earlierKeys;}
}

public bool TryGet(string key, out string value)
{
    try
    {
        var response = ssmClient.GetParameterAsync(
            new GetParameterRequest
            {
                Name = $"{configSrc.BasePath}{ConvertToSMSHeirarchy(key)}",
                WithDecryption = true
            }).GetAwaiter().GetResult();
        value = response.Parameter.Value;
        return true;
    }
    catch(Amazon.Runtime.AmazonServiceException)
    {
        value = null;
        return false;
    }
}

private static string ConvertToSMSHeirarchy(string parameterName) => parameterName?.Replace(':', '/') ?? "";

// Other methods were not needed
private IChangeToken _changeToken = new ConfigurationReloadToken();
public IChangeToken GetReloadToken() => _changeToken;
public void Load() {}
public void Set(string key, string value) { }

It actually worked, but there was a problem: the TryGet method gets called every time that the configuration value is requested (ASP.NET Core does not cache the value for you). According to the AWS Systems Management pricing page, AWS charges per "API Interation." To make matters worse, the call to GetParametersByPathAsync() gets both the keys and the values, but the GetChildKeys method above throws away those perfectly good values and only returns the keys.

Using The Helper Class

I briefly considered writing my own optimized version to address these problems, but I couldn't do any better than the ASP.NET Core's ConfigurationSource helper class2. By using the helper class, all I had to implement was the Load method; the class provides default implementations for all the other methods of IConfigurationProvider. The Load method was straightforward to implement; after calling GetParametersByPathAsync(), I simply iterated over the returned key-value pairs and added them to the internal Data dictionary.

The full class can be found here, but I've included the implementation of the Load method here:

public static readonly string KeyDelimiter = "/";

// These are set in the constructor
public AmazonSimpleSystemsManagementClient SmsClient { get; }
public string BasePath {get; }

public override void Load()
{
    try
    {
        string nextToken = default;
        do {
            // Query AWS Parameter Store
            var response = SmsClient.GetParametersByPathAsync(
                new GetParametersByPathRequest
                {
                    Path = BasePath,
                    WithDecryption = true,
                    Recursive = true,
                    NextToken = nextToken
                }).GetAwaiter().GetResult();

            // Store the keys/values that we got back into the protected Data dictionary
            foreach (var parameter in response.Parameters)
            {
                var dotNetKey = parameter.Name.Substring(BasePath.Length).Replace(KeyDelimiter, ConfigurationPath.KeyDelimiter);
                Data[dotNetKey] = parameter.Value;
            }

            // Possibly get more
            nextToken = response.NextToken;
        } while (!String.IsNullOrEmpty(nextToken));
    }
    catch (Amazon.Runtime.AmazonServiceException)
    {
        // Typically an IAM permissions issue, but could also be that there are no values to retrieve.
        return;
    }
}

Next Steps

I've covered the basics of configuration in ASP.NET Core, and how to write a custom configuration provider. But in order to use the provider, we need to incorporate it into the host build step. There's a few more decisions to be made: Where in the provider list should the new provider be? What can it override? How do we configure the provider itself? I'll cover all of that in my next post.


  1. I'll have tons more to say about this in an upcoming blog post.

  2. The source code can be found here: https://github.com/aspnet/Configuration/blob/master/src/Config/ConfigurationProvider.cs

 

Comments


 

Want to leave a comment? Sign up!
Already signed up? Sign in!