SaltStack: Install RPMs from a list in text file

Introduction
To describe a rpm package installation in SaltStack, normally a salt formula with contain both the name of RPM package and a instruction to install it. If list of RPMS is large and requires changes quite regularly then maintaining states file would be very cumbersome. As state files are ususally yaml files, a human typo/error would be quite frequent then. I had a similar kind of issue.

Therefore I created a Salt formula that would use just a plain list of RPMS files in a text file. So I would just update that text file with the list of RPMS.

Strategy
We will create pillar data from a plain text file having list of RPM names on Salt Master. This pillar data will be exposed to agents with "pillar_rpm_packages" pillar key. Following will be the structure of pillar data that will be created.

'pillar_rpm_packages': [ 'package1==ver1.rel1.x86_64', 
                         'package2==ver2.rel2.x86_64',
                         'package3==ver3.rel2.x86_64', ]

Then we will create a salt module custmod_rpm_packages_from_list (on SaltMaster). This module will have a function called "create_packages_states". This salt module will be pushed to salt agents. Agents will call this module.function as custmod_rpm_packages_from_list.create_packages_states. This function accepts "pillar_rpm_packages" as its argument and then generates "json" formatted states of package installation.
create_packages_states function will create similar to following json structure.

{'pkg': ['installed', {'version': pack_version }]}

Please note that we wanted to have package installation states generated with version number included, so that only a specific version of RPM package can be installed not latest one.

Structure
This setup assumes that Salt is configured in Master/Agent mode.
Pillar is in following directory.

/srv/salt/baseenv/pillar/

States are in following directory.

/srv/salt/baseenv/states/

PILLAR DATA GENERATION
Our rpm list file , a text file, will be in following location.

# mkdir /srv/salt/baseenv/pillar/data
# touch /srv/salt/baseenv/pillar/data/rpm_packages.list

Following are the contents of rpm_packages.list file. Ofcourse you can add your own.

# cat /srv/salt/baseenv/pillar/data/rpm_packages.list
lsof-4.82-4.el6.x86_64
dash-0.5.5.1-4.el6.x86_64
ed-1.1-3.3.el6.x86_64

Create Pillar function.

# mkdir /srv/salt/baseenv/pillar/allrpmpackages
# touch /srv/salt/baseenv/pillar/allrpmpackages/init.sls

Following are the contents of pillar/allrpmpackages/init.sls file.

#!py
#
# This state file will just extract packages name versions and release info and create a salt compatibale data structure 
# created data structure will be : 
# config = { 'pillar_rpm_packages': [ 'package1==ver1.rel1.x86_64', 'package2==ver2.rel2.x86_64' ] }
#

import os, subprocess

def run():
  config = { 'pillar_rpm_packages': []}
  for package in open('/srv/salt/baseenv/pillar/data/rpm_packages.list'):
    package=package.rstrip()
    repo_query_command="repoquery " + package + " --qf '%{NAME}==%{VERSION}-%{RELEASE}'"
    rep_query_process=subprocess.Popen(repo_query_command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    rep_query_process.wait()
    if rep_query_process.returncode == 0:
      formatted_package_name=rep_query_process.stdout.readline().rstrip()
      config['pillar_rpm_packages'].append(formatted_package_name)
  return config

/srv/salt/baseenv/pillar/top.sls will have something similar to following.

baseenv:
  '*':
    - allrpmpackages

STATES and MODULE GENERATION
Create module.

# mkdir -p /srv/salt/baseenv/states/_modules
# touch /srv/salt/baseenv/states/_modules/custmod_rpm_packages_from_list.py

Following are the contents of states/_modules/custmod_rpm_packages_from_list.py

#! /usr/bin/env python

import os, subprocess, re, os.path

packages_states = { }
def create_packages_states(pillar_key):
 for pillar_pack in __pillar__[pillar_key]:
   match_obj=re.search(r'(?P.+)==(?P.+)',pillar_pack)
   pack=match_obj.group('rpm_pack_name')
   pack_version=match_obj.group('rpm_version')
   pack_hash = { pack: pack_version } 
   packages_states[pack] = {'pkg': ['installed', {'version': pack_version }]}
 return packages_states

Now if we want to use above module to Install RPMS, then create a state file with following contents.

# touch /srv/salt/baseenv/states/install_rpm_packages.sls

states/install_rpm_packages.sls file will have following contents.

#!py
config = {  }
def run():
  config = __salt__['custmod_rpm_packages_from_list.create_packages_states']('pillar_rpm_packages')
  return config

baseenv/states/top.sls may have following contents.

baseenv:
  '*':
   - install_rpm_packages

Above code can also be found at https://github.com/spareslant/SaltStackRPMInstallFromListFormula