Introduction to automating your VM configuration

Create a new VM, set the cores, set the memory… disk sizes… image. Great, it’s deploying. Now let’s go ahead and manually install our application… RDP to the machine, download the file(s), double click… you get the picture. Does this sound like your workflow when deploying a virtual machine to the public cloud?

While this works, it is possible to remove all of that complexity and create a reusable, declarative template that will automatically configure your VM. It can install software, connect to ancillary resources, do some post-deployment configuration, and all in an unattended fashion. In Azure, this functionality is called “Custom Data” and in AWS it is referred to as “User Data”. In Tuono, we use the catch-all term “userdata” and support this functionality in either AWS or Azure.

Why would you use userdata?

Sometimes you just need infrastructure, but other times you need something that can do real work immediately. Userdata allows you to define specific commands that will be reliably executed on every deploy. What do I mean by that? Well, it might be something as simple as updating the system, joining a domain, setting a compliant hostname, but it could be something much more complex. We’ll talk through some specific examples and provide some example code in later articles, but for now you might want to:

  • Define OS-level properties post-deploy
  • Install required applications post-deploy
  • Bootstrap a VM that requires access to specific resources
  • Pull down existing scripts from a repository to bootstrap a VM

What tools are available in userdata?

There are three main types of userdata in Tuono:

  • BASH (Borne Again Shell)
  • Powershell
  • cloud-init

BASH

Bash is commonly used for Linux administration tasks. Anyone who has used Linux has almost certainly used some bash and the key to its’ ubiquity is its’ utility. It is the de facto language of system administration on Linux and it can be used for nearly everything within a Linux system. Every system configuration tool is exposed through bash, so you can leverage these to configure a Linux virtual machine using bash.

Significantly, through bash you can also use other languages as needed. For example, you may have some Python code that you need to run. No problem! In this short example I am writing out and executing a Python script directly, but you could just as easily pull down an existing Python script. Note that I also use a Tuono variables for the “admin_username” within this script.

userdata:
          type: shell
          content: |
            #!/bin/sh
            
            ...
                        
            echo "#!/usr/bin/python3
            
            with open('/home/((admin_username))/output.txt', 'w') as output:
              output.write('I am a Python output!')" >> my_script.py
            
            chmod +x my_script.py
            
            ./my_script.py

Powershell

Powershell offers exactly the same benefits for Windows users, as bash does for Linux users. On top of being a fairly robust general purpose scripting language, there are many modules available to allow you to integrate with the wider Windows ecosystem – joining a domain for example. With that said, there are also many third-party modules to dramatically increase utility. Of particular interest might be the Azure Az cmdlet to integrate directly with the Azure platform.

What if you can’t do what you want in Powershell? No problem, we can also use .NET in userdata directly. In this example I am using .NET to pull down a series of bootstrapping Powershell scripts from an internal repository:

userdata:
      type: powershell
      content: |
            function Get-File ($url, $uri) {
              $WebRequest = New-Object System.Net.WebClient
              [System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
              $webRequest.Headers.Add("AUTHORIZATION", "Basic $encodedCreds")
              $WebRequest.DownloadFile($url, $uri)
            }

            $script:encodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes("<byte_string>"))
            $base_url = '<https://url.for.resource>'
            Get-File -url ($base_url + 'file_1.ps1') -uri .\file_1.ps1
            Get-File -url ($base_url + 'file_2.ps2') -uri .\file_2.ps1
            
            .\file_1.ps1
            .\file_2.ps1

cloud-init

As you may be aware, every Linux base uses a slightly different method for configuring a system. Adding a user may be slightly different, manipulating the network might be slightly different, etc. and this means you’re effectively writing bash that is specific to your Linux distribution, so it is not reusable. cloud-init aims to provide a cross-platform method to abstract away a lot of this complexity. cloud-init uses a standard syntax for many tasks and the exact implementation is handled by cloud-init itself. The benefits here are a single userdata stanza can be used across your Linux systems, whether they be DEB or RPM based. In this example, you can see how easy it is to install applications, configure users, and make configuration changes. Note that this uses several Tuono variables throughout.

userdata:
          type: cloud-init
          content: |
            #cloud-config
            package_upgrade: false
            packages:
              - nginx
            users:
              - name: ((admin_username))
                groups:
                  - sudo
                sudo: ALL=(ALL) NOPASSWD:ALL
                ssh_authorized_keys:
                  - ((admin_public_key))
            runcmd:
              - sudo su 
              - echo '((your_caption))' > /var/www/html/index.nginx-debian.html
              - sed -i 's/listen 80 default_server;/listen 8080 default_server;/' /etc/nginx/sites-enabled/default
              - sed -i 's/listen \[\:\:\]\:80 default_server;/listen \[\:\:\]\:8080 default_server;/' /etc/nginx/sites-enabled/default
              - systemctl restart nginx

Using secrets in userdata

What about “sensitive” data, such as passwords? These are obviously required for many use cases where security is important. Can you use them in userdata? Yes, you can, but there is an extra step involved. Directly in userdata, the simple answer is no. AWS and Azure store the user data content itself in plain text (AWS), or simple base64 encoded (Azure), but all is not lost.

With Tuono, you can add a value from your Tuono Secret Vault to the native AWS/Azure secret store. Additionally, the Blueprint can contain a stanza that will allow the virtual machine to gain access to the stored secret programmatically. Using well-documented AWS/Azure recipes, we can write userdata that can access these secrets programmatically and securely.

You can think of this of this as an ephemeral, virtual machine scoped secret, which is only accessible to this machine. Pretty neat, eh?

Azure

In the case of Azure, we take a secret from the Tuono Secret Vault and push it to the Azure Key Vault. We then add a Managed Identity to the virtual machine which grants it the ability to access the stored secret programmatically. It is this secret that can be securely accessed via the userdata.

AWS

In the case of AWS, we take a secret from the Tuono Secret Vault and push this secret into the AWS Parameter Store. We then apply an appropriate IAM policy to VM that allows it to access the stored secret programmatically. This secret can then be accessed directly within userdata. (This feature is in development and coming soon.)

The power of userdata

Hopefully this article has demonstrated how powerful userdata can be. While you may simply want to deploy some infrastructure, you can see how powerful it can be to make sure that all of your VMs meet internal compliance requirements, have access to the same resources, or have specific configurations applied at deployment time.

In the next article, we’ll go into some specific examples of how to use userdata (with code). We’ll cover some basic initial configuration, accessing resources from a blobstore and using secure values in userdata. Stay tuned for Part 2 of our series on userdata. In the meantime, you can sign up for a free Community Edition account and try this functionality for yourself.

Deploy your first environment