Dave HarriganEngineering @ Manifold

New in version 2.8, Ansible has added a built-in Manifold lookup plugin that allows you to get credentials from Manifold. This is really powerful if you’re looking for a solution to securely manage your secrets and resources provisioned from Manifold in one place. In this post, I’ll create a very simple Python application to show how easy it is to inject secrets from Manifold into your existing application via Ansible.

Embedded content: https://manifold.wistia.com/medias/xawimlt9d1

What is Ansible anyway?

Great question! Ansible is a server automation framework that allows you to provision an entire infrastructure, manage configuration, and deploy applications to your systems. It aims to be easy to use—the instructions (called playbooks) are written in YAML, and it relies on modules written in Python that implement the directives used in playbooks.

Before we start

If you’d like to run through the example we show here, you’ll need:

  1. A Manifold account
  2. Ansible 2.8 or higher
  3. Vagrant
  4. VirtualBox

Let’s get Manifoldin’

Once you’ve logged in to Manifold.co, the first step is to create a new project and provision a new LogDNA resource. This will populate new credentials for accessing LogDNA under the selected project.

Here’s what to do, step-by-step. First, create a new project.

Create a new project

From your new project, click Add a new resource.

Add a new resource

Scroll down and select LogDNA.

Select LogDNA

Keep the default free plan selection. Default Free Plan

Give your resource a unique name, and click Create LogDNA Resource.

Create LogDNA Resource

That’s it! Now you have a cloud-based log aggregation and monitoring platform! If you’re feeling spicy, click Open LogDNA Dashboard.

Open LogDNA Dashboard

Be patient while your Manifold credentials are used for SSO access to LogDNA’s dashboard. Spoiler: there won’t be anything on the LogDNA dashboard for you to look at just yet, but we’ll get to that soon.

A look at the sample files

Clone the sample ansible-demo repo. The repo contains a very simple Python app along with an Ansible playbook that provisions a VirtualBox VM to run the app. The app adds a LogDNA logging handler for our app to use, and the health endpoint simply logs a “health check” message every time you call GET on the /health endpoint.

The key required to push our logs to LogDNA will come in as an environment variable, and in our case, we’ll let Ansible get this from Manifold, so that it’s automatically injected during service start-up.

1'logdna': {
2 'level': 'DEBUG',
3 'class': 'logging.handlers.LogDNAHandler',
4 'key': os.environ.get('KEY'),
5 'options': {
6 'app': 'ansible-manifold-demo',
7 },
8},

In order to daemonize the very simple Python app, the repo sets up a systemd script to run the app as a systemd service. This means we need to inject our environment variable into the systemd service. We can use EnvironmentFile to let Ansible generate a new environment file for the systemd service to use.

1- name: fetch credentials
2 set_fact:
3 manifold_secrets: "{{ lookup('manifold',
4 'ansible-demo-logging', project='ansible-demo') }}"
5
6- name: configure systemd env
7 template:
8 src: app.env.j2
9 dest: /etc/systemd/system/app.env
10 with_dict:
11- "{{ manifold_secrets }}"

We use set_fact to save the result of lookup to the manifold_secrets variable. We then use it to create an app.env file using the app.env.j2 template, which loops over the “manifold_secrets” dictionary.

1{% for key, value in manifold_secrets.items() %}
2{{ key }}={{ value }}
3{% endfor %}

Depending on how your app is set up, it might make more sense to generate a Python file with the secrets embedded, directly expose the secrets as environment variables in your task, or store the secrets file in another directory with stricter permissions.

The rest of the playbook is straightforward — it installs some dependencies, copies the systemd config, and starts up the service. Here’s what our entire playbook looks like:

1- name: update apt cache if needed
2 apt: update_cache=yes cache_valid_time=3600
3
4- name: install pip
5 apt:
6 name: "python3-pip"
7 state: latest
8
9- name: install pip deps
10 pip:
11 requirements: /usr/src/app/requirements.txt
12
13- name: install app systemd
14 template:
15 src: app.service.j2
16 dest: /etc/systemd/system/app.service
17
18- name: fetch credentials
19 set_fact:
20 manifold_secrets: "{{ lookup('manifold', 'ansible-demo-logging',
21 project='ansible-demo') }}"
22
23- name: configure systemd env
24 template:
25 src: app.env.j2
26 dest: /etc/systemd/system/app.env
27 with_dict:
28 "{{ manifold_secrets }}"

Grab the keys to the kingdom

Before we can run the playbook and start provisioning, we need to obtain an API token from Manifold and expose it as an environment variable. We need this token to securely access credentials from Manifold.

Navigate to Manifold’s API Tokens page and generate a token. Call it “ansible” and give it read-credentials permissions. Manifold API Tokens

Copy the generated token, and export it as MANIFOLD_API_TOKEN in your shell.

1export MANIFOLD_API_TOKEN=<your API token>

Let’s run that playbook

Now we’re ready to run our playbook against a real machine! For simplicity, the repo includes a Vagrantfile which sets up an Ubuntu box using the VirtualBox provider and runs the playbook.

Run vagrant up. After a little bit, your VM should be running under the IP configured in the Vagrantfile.

Go ahead and curl the health endpoint from your host.

1 curl -v 192.168.33.27:8000/health
2 * Trying 192.168.33.27...
3 * TCP_NODELAY set
4 * Connected to 192.168.33.27 (192.168.33.27) port 8000 (#0)
5 > GET /health HTTP/1.1
6 > Host: 192.168.33.27:8000
7 > User-Agent: curl/7.54.0
8 > Accept: */*
9 >
10 * HTTP 1.0 assume close after body
11 < HTTP/1.0 204 No Content
12 < Date: Mon, 17 2019 21:24:51 GMT
13 < Server: WSGIServer/0.2 CPython/3.7.3
14 < Content-Length: 0
15 <
16 * Closing connection 0

You should soon start seeing your logs in the LogDNA dashboard, which you can access from your LogDNA resource page in the Manifold dashboard. LogDNA Dashboard

Wrapping it up

I hope you’re as excited as we are about the new Ansible Manifold integration. You can provision managed cloud services as well as set up configuration and secrets independent of the tools you’re using to manage your infrastructure — all you need is love and Manifold :D.

Stratus Background
StratusUpdate

Sign up for the Stratus Update newsletter

With our monthly newsletter, we’ll keep you up to date with a curated selection of the latest cloud services, projects and best practices.
Click here to read the latest issue.