Intent

Configuration Management is a pattern for centralizing configuration and secret values in order to increase security, speed up onboarding or bootstrapping new environments and minimize misconfigured deployments.

Problem

In modern distributed architectures (service-oriented, microservice, serverless, etc) your configuration needs to live in several locations, including local development. With the advent of continuous delivery, this problem is exacerbated through an ever-changing application and added stress of an increased number of environments.

Using hardcoded, or bundled, configuration leaves your secrets vulnerable on disk. The sets of values are challenging to audit to ensure that the correct values are delivered for the specified environment. Correcting any abnormality requires you to re-deploy the application.

Keeping a configuration bundle secure often consists of complex encryption procedures, reduced number of individuals who have access, if it’s secure at all. Hardcoded, or bundled, configuration typically offers zero access controls, making the process of identifying the source of a change or controlling who is authorized to introduce new configuration impossible.

Story Content

Solution

How to stay secure

Secrets should never be checked into source control, and should not live on disk unencrypted.

Configuration management frameworks typically give you a variety of access controls, which enable you to adhere to the principle of least privilege. This means your operations team could have full write access, but nodes will only have read access to specifically what they need.

Platforms may also provide auditing features which allow you to see which values were accessed, or modified, and by who.

Agent vs application-level

When deciding on a platform for configuration management you will need to understand the differences between a service which operates with an agent, and one that does not.

Tools which rely on an agent increase overhead necessary to deploy/maintain the system. Each node which accesses configuration requires an additional process (agent) which communicates with the master at a specific interval to keep its configuration up to date. Deploying an agent to every node may increase your attack surface area.

For application-level configuration management, the application is expected to request its configuration during startup. While this does not incur any additional overhead on the machine, it does require external access to the config management platform.

Multiple environments

Each environment your applications lives in should have its own isolated configuration and secrets. This often means maintaining a host of tiny files in several locations.

Configuration management removes this headache by centralizing the shape of your deployments’ configuration, enabling you to easily see which values are required for each environment and specific unique values for each one.

Ease of rotation

In the event of a security breach, either to your platform or a service you rely on, you may need to change the values your application depends on.

Config management enables you to quickly change values and deliver those changes either automatically or selectively, compared to the traditional method of performing a complete deploy of your application.

The single view of your entire configuration dependency makes it easier to comprehend the impact changing a particular value will have your system.

Structure

Story Content

Pros and Cons

Pros

  • Access controls
  • Single view for all configuration and secrets
  • Synchronized updates, across environments
  • Streamlined onboarding of developers and systems

Cons

  • Potential single point of failure
  • Latency during builds and runtime
  • Provider lock-in
  • Overhead from maintenance/operation

Relevant Tools and Services

Hashicorp Vault

With its robust access controls and a complete set of interfaces (UI, CLI, or HTTP API), Vault is a popular tool to protect tokens, passwords, certificates, and encryption keys. Vault is notable as one of the first systems to support automated credential rotation.

Manifold

In addition to storing credentials for purchased services, Manifold supports secure storage of arbitrary credentials. As a hosted service, Manifold does not require you to run and maintain additional infrastructure.

AWS Secrets Manager

Built on top of AWS Key Management Service, AWS Secrets Manager provides secure credential storage, and advanced features like credential rotation through Lambda functions. Like Manifold, AWS Secrets Manager does not require you to run and maintain additional infrastructure.

ZooKeeper

Originally a subproject of Hadoop, the venerable Apache ZooKeeper is one of the first hierarchical key-value stores. Many distributed systems rely on ZooKeeper for coordination and configuration; you may already use it for another service, making it a good choice for application configuration.

etcd

etc fills a similar role to ZooKeeper, but provides a more familiar HTTP REST API, in addition to faster binary protocols. Newer distributed systems tend to rely on etcd over ZooKeeper.

Manifold Background

Config Management

Stay secure, streamline deployments, avoid misconfiguration
twitterShare Pattern