How to deploy Django application using Chef

Django is a popular Python web framework. I’ve been asked a couple of times how Python based applications can be automated so here I’ll give an example.

We’ll be using Chef. Before we get started let’s discuss most of the items that make Chef’s ecosystem.

  • Recipe - a set of steps to make something happen. Just like a recipe for cake. In our case think of it as the recipe of how to build our application or install some dependencies.
  • Cookbook - well.. a collection of recipes. Plus few more things we’ll be playing with shortly.

There are different commands like knife but we won’t discuss them in much detail. Now we’ll go over one cookbook used to deploy the following Django app:

https://github.com/gothinkster/django-realworld-example-app

It’s a real world example of a web application that I found not long ago. It has most of the things you would see in such app - database, frontend, some backend.

All Chef code we are going to use to deploy this application is here:

https://github.com/zinderic/django-realworld

Prerequisites

  • Chef DK or Chef Workstation
  • VirtualBox
  • Vagrant

The Chef Cookbook Structure - recipes

The most important files we’ll be looking at are located at the recipes folder.

Here’s what recipes we have in their order of execution:

  • install_packages - this one installs some package dependencies using apt-get

We’ll be deploying to Ubuntu. It’s of course an option to support multiple Linux families like Arch, RedHat and Debian in the same cookbook but that’s a more advanced setup we won’t cover here.

  • create_user - creates django user and group
  • pyenv - setups python virtual environment
  • app - deploy the application

Here’s one example of a recipe:

#
# Cookbook:: django-realworld
# Recipe:: install_packages
#
# Copyright:: 2018, The Authors, All Rights Reserved.

execute "update-package-cache" do
    command "sudo apt-get update"
    action :run
end
execute "install-build-essential" do
    command "sudo apt-get install -y build-essential checkinstall"
    action :run
end
 execute "install-prereq" do
    command "sudo apt-get install -y libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev"
    action :run
end

Here we execute commands but you would often employ Chef’s functions. Like in here:

git '/home/django/.pyenv' do
    user 'django'
    group 'django'
    repository 'https://github.com/pyenv/pyenv.git'
    revision 'master'
    action :sync
end

Notice that we’re using a resource called git which comes with Chef’s namespace and handles git related work. If you want to learn more about the git resource go read the docs here:

https://docs.chef.io/resource_git.html

Every time you need to write something you can check for pre-defined resource you can use and build things up from there. Of course you can also use Ruby directly. I would prefer to use pre-defined resources because this often saves me a lot of time and I know that those implementations are safe and tested. They are probably much better than what I would come up with now because I’ll be using it in just my cookbook while the pre-defined resources are used by the whole community.

The Chef Cookbook Structure - tests

Just like with software development it’s a good idea to write tests. Tests that cover the functionality of our cookbook are critical for the long-term success of our CI/CD pipelines. It will help us make future changes easier and feel more secure that we won’t break the deployment chain of events and impair our production releases.

Tests are located in their test directory and in our case they cover one to one each recipe.

Here’s how a test looks like:

%w(
    'build-essential'
    'checkinstall'
    'libreadline-gplv2-dev'
    'libncursesw5-dev'
    'libssl-dev'
    'libsqlite3-dev'
    'tk-dev'
    'libgdbm-dev'
    'libc6-dev'
    'libbz2-dev'
).each do |pkg|
    describe package(pkg) do
        it { should be_installed }
    end
end

Notice that the whole syntax is pure Ruby. The interesting part is the describe method. For the testing part you can start with ChefSpec which is the main facility for writing Chef tests. You can find it here:

https://docs.chef.io/chefspec.html

There is more to testing. You can mock all kinds of objects that you don’t normally have. Imagine for example that in production you need a mysql database. Or maybe Kafka or Zookeeper. They will provide some service to your application that during testing you don’t need to have. You can safely mock it and make your tests run independently and quickly which is one of the most important thing about tests.

Don’t go and engineer your whole recipes just so you can test them. If you put data in your database during a recipe mock that data up in your test and expect it to be there while you are smoke testing your recipe. That way you’ll know that when you call the real database your smoke will indeed check this data and validate it the way your described.

OK, we have some recipes and tests. How can we really see them run? That’s the fun part. Let’s go and checkout Kitchen.

Testing Cookbooks with Kitchen - configuration

The first thing to do when you want to develop Chef cookbook is install Chef DK or nowadays you can also take advantage of the great Chef Workstation. It has some advantages but it’s still in its early days. Both tools will have the kitchen command.

Here’s our Kitchen definition file called .kitchen.yml:

---
driver:
  name: vagrant
  customize:
    memory: 4096
    cpuexecutioncap: 100

provisioner:
  name: chef_zero
  # You may wish to disable always updating cookbooks in CI or other testing environments.
  # For example:
  #   always_update_cookbooks: <%= !ENV['CI'] %>
  always_update_cookbooks: true

verifier:
  name: inspec

platforms:
  - name: ubuntu/xenial64

suites:
  - name: default
    run_list:
      - recipe[django-realworld::install_packages]
      - recipe[django-realworld::create_user]
      - recipe[django-realworld::pyenv]
      - recipe[django-realworld::app]
    # verifier:
    #   inspec_tests:
    #     - test/integration/default
    attributes:

It’s your door to quick and easy development so let’s see what each section does.

  • Driver - that’s the hypervisor you’ll be using when Kitchen runs. In our case we’ll use Virtualbox via a tool called Vagrant.
  • Provisioner - that’s our automation tool and while we write Chef Cookbooks we’ll be using chef_zero.
  • Verifier - testing facility.
  • Platform - what image we will use. As I said we’ll be using Ubuntu image.
  • Suites - how many different systems (or VMs in our case) we will build to test with. Here we also put every recipe we want to run.

Testing Cookbooks with Kitchen - execution

  • kitchen create - creates the VM.
  • kitchen converge - runs the Chef recipes as defined in the Kitchen configuration file and created in our Cookbook.
  • kitchen verify - run tests.
  • kitchen destroy - destroys the VM.

This is one nice order of execution while you’re building things up. If you want to quickly test you can also do:

  • kitchen test - creates VM and run tests.

If you have a VM created via kitchen create you can also login to the VM.

  • kitchen login <name> - logins to a suite. In our case since we have only one we can just say kitchen login or if we feel more expressive we can still say kitchen login default.

Summary

In this article you saw how to:

  • Create Chef Cookbook recipes
  • Create Chef Cookbook tests
  • Use Kitchen to aid you while creating or testing your cookbooks

Categories:

Updated: