Thomas Martin I/O

Programming, Sysadmin, Open Source

Nodes inventory using reclass

2014-06-24 SYSADMIN SALT RECLASS


Introduction

When managing a computer infrastructure, it's convenient to keep a central inventory of all the nodes composing it, along with their associated software lists, hardware specifications, specific parameters and documentations.

The first version of this article referred to the term hosts instead of nodes. But I finally decided to modify that, because the node term is more generic (a node can refer to a network equipment, storage bay, or whatever takes a role in a computer infrastructure), and it is also what is used by the reclass software.

This central inventory then allows to generate and configure various parts of the infrastructure, like :

  • Configuration management system
  • Firewall configuration
  • Parts of documentation
  • Monitoring configuration
  • Containers and virtual machines configuration

Nodes inventory software

LDAP, or other hierarchical databases, are traditionally a good fit for this job. Each entry in the directory can correspond to a node of the infrastructure, and contains a list of mandatory and optional properties. LDAP provides schema inheritance, useful for enforcing the existence of particular properties when a given class is associated to a node. Finally, LDAP can be easily requested by numerous third-party applications. Unfortunately, LDAP does not provide data inheritance, so you cannot define a property's value in a upper-level class, and retrieve it automatically when you request a node's entry which inherits from this class.

CMDB software are heavier solutions doing usually more than just nodes inventory. They can feature for example a ticketing system for end-user requests, a knowledge base, and various dashboards. The drawback is that their data is generally stored in complex databases, and cannot be source controlled. The webpage 6 Open Source CMDB lists some Open Source CMDB software.

Finally, there is reclass, a minimalist free software describing itself as an External Node Classifier. It stores all its data in YAML, and is handled using command line. The next sections explain some basic concepts around it.

reclass

Concepts

reclass offers the possibility to associate different kind of informations to your nodes :

  • Applications : software and feature list (e.g. nginx, ssh_client)
  • Properties : tree of key/value parameters (e.g. ipaddr = 192.0.2.1, nginx_port = 8080)
  • Classes : list of inherited classes (e.g. web_server, debian_server)
  • Environment : environment belonged to (e.g. production, staging, devel)

Classes contain the same set of informations as nodes. Each node or class can inherit from one or several classes, and can overwrite its inherited applications and properties.

Note that example values above are all arbitrary. reclass does not predefine any application, parameter or class names.

Then, reclass allows to perform two main types of operations :

  • Retrieving informations for a specific node (applications, parameters)
  • Listing the full inventory of nodes and associated informations

For a full understanding of reclass concepts, please read the following pages in the documentation : reclass concepts and reclass operations.

Installation

The installation page on the reclass website lists all available installation methods.

There are packages for a few Linux distributions (Debian Jessie, Arch).

To perform a source installation, execute the following commands :

git clone https://github.com/madduck/reclass.git
cd reclass
python setup.py install --prefix=/usr/local

My personal setup defined in the configuration file /etc/reclass/reclass-config.yml is the following :

pretty_print: True
output: json
inventory_base_uri: /srv/inventory/

It configures a useful output when reclass is invoked manually, and defines the location of inventory data in /srv/inventory.

Some options allow to change the location of nodes and classes definitions but I prefer to keep the defaults. So classes will be found in /srv/inventory/classes/ and nodes in /srv/inventory/nodes/.

Adding classes

Given the previous configuration, class files should be created in /srv/inventory/classes, using the YAML format and a .yml extension.

Example of a class called infra1 defined in file /srv/inventory/classes/infra1.yml :

applications:
    - sshd
    - postfix

parameters:
    description: "Base class for infra1"
    admin_mail_address: root@example.com

Each node which inherits from this class :

  • Should have the sshd and postfix applications installed
  • Should send administrator e-mail to root@example.com (unless redefined)

Adding nodes

So, let's add a first node called web1 in /srv/inventory/nodes/web1.yml :

classes:
    - infra1

environment: production

applications:
    - nginx
    - php

parameters:
    admin_mail_address: bob@example.com
    hypervisor: kvm1
    nginx:
        virtual_servers:
            example:
                domain_name: www.example.com
                root: /srv/www/example

Here we define that web1 inherits from class infra1, and belongs to environment production.

The applications list has been extended with two new entries : nginx and php, which should be installed in addition to sshd and postfix defined in class infra1.

Finally, the parameters list has been extended with two new parameters hypervisor and nginx, and the field admin_mail_address inherited from infra1 has been redefined.

Retrieving informations

Execute the following command to retrieve informations for web1 :

# reclass --nodeinfo web1
{
  "environment": "base",
  "applications": [
    "base",
    "postfix",
    "nginx",
    "php"
  ],
  "classes": [
    "infra1"
  ],
  "parameters": {
    "hypervisor": "kvm1",
    "admin_mail_address": "bob@example.com",
    "description": "Base class for infra1",
    "nginx": {
      "virtual_servers": {
        "example": {
          "root": "/srv/www/example",
          "domain_name": "www.example.com"
        }
      }
    }
  }
}

We can see that the JSON-formatted output contains all informations for web1, according to the rules explained in the previous sections.

Finally, if you want to retrieve informations for all defined nodes, use the following command which produces a similar kind of output :

# reclass --inventory

Conclusion

Reclass is a nice tool for people who like the KISS principle and minimalist tools. Furthermore, it supports more advanced features than what we see above (wildcard/regexp mappings, parameter interpolation), so be sure to check the full documentation.

Currently, reclass only supports the YAML FS storage, but could be extended to extract its data from other backends (MongoDB would be cool !).

Finally, the real power of reclass comes from the interaction with other tools. In a forthcoming article, I will write about interactions with the configuration management tool Salt.