Thomas Martin I/O

Programming, Sysadmin, Open Source

Salt, improving Jinja usage

2014-02-27 SYSADMIN SALT


Salt and Jinja

Salt is a configuration management and remote execution software. One of its main usage is to define intents about system configuration, which are described in states files. In these states files, the Jinja rendering language can be used to manipulate data coming from external sources (like Pillar). The following assumes you've a basic understanding of how states and pillars work.

Last week, I pushed this commit on a repository containing a collection of Salt's states. It substantially simplify the way I access data using Jinja. Furthermore, it fixes what I consider an important bug : previously, my states were not working if some pillar data were not defined. This is something I care, because I want my states to be reusable by other people, and I don't want to break their Salt setup if they don't have the right Pillar data defined.

So, I made the following changes.

Using pillar.get

First, I switched to use systematically the pillar.get function to access Pillar data :

{{ salt['pillar.get']('some:nested:dict', 'my_default') }}

This Jinja variable interpolation calls the get function of the salt's pillar module in order to :

  • Access a nested pillar data (value of the dict key in the nested dictionary in the some dictionary)
  • Provide a defaut value (my_default), avoiding to break the state execution if the pillar data is not defined

Iterating over Pillar

Next, I improved the way I perform iterations over lists of pillar. I will use this Pillar tree as an example :

users:
    bob:
        uid: 8001
        fullname: "Bob"
        password: '...'
    audrey:
        uid: 8002
        fullname: "Audrey"
        password: '...'

Previously, I used to iterate over the username and associated user information as this :

{% if pillar['users'] is defined %}
{% for user, userinfo in pillar['users'].iteritems() %}
[...]
{% endfor %}
{% endif %}

The {% if test was here to avoid iteritems() to fail if pillar['users'] is not defined.

Instead, it is much better and simpler to use pillar.get() and to define an empty dictionary as default value :

{% for user, userinfo in salt['pillar.get']('users', {}).iteritems() %}
[...]
{% endfor %}

In this case, iteritems() cannot fails.

Using .get()

Finally, I need sometimes to test if an item is present in a list stored in a dictionary coming from Pillar (root_pathinfo['config_tags'] in the example below). This list can be undefined when no entries are relevant in a particular case.

I was used to surround these tests with a {% if to check if the list is defined :

{% if root_pathinfo['config_tags'] is defined %}
{% if 'php5' in root_pathinfo['config_tags'] %}
[...]
{% endif %}
{% endif %}

Instead, I now use the .get() Python dictionary method with a empty list as default value :

{% if 'php5' in root_pathinfo.get('config_tags', []) %}
[...]
{% endif %}

Two lines removed.

Conclusion

In conclusion, one should try to keep Jinja usage the simplest, to allow better maintainability, readability and debugging.

If the code aims to be reusable, it is also important to keep it functional even is some Pillar data are not defined. For that, one can consider to add automatic testing to the Salt code. In this regard, I heard during cfgmgmtcamp about the kitchen-salt project, a Salt provisioner for Test Kitchen.

In a second part, I will explain how to define all your configuration variables in a single map.jinja file. This allows to not access directly pillar data into state files, and to provide default and OS-specific values. This is a requirement if you plan to write new Salt Formulas.