Puppet Portage module version 2.0

After a few months of a lot of hard work, I’m thrilled to announce the availability of the Portage Puppet module version 2.0!

Background

This module was initally developed by Lance Albertson and some other guys from OSUOSL. Adrien Thebo, who by the way works for PuppetLabs, stepped in after some time and did some cleanup in the native types/providers that were included in there, and later he took over completely. Judging from the commit log though, the module was not getting the love it deserved :-)

Meanwhile, we, the Gentoo Infrastructure team, decided to migrate from Cfengine 2 to Puppet. So I started digging around to check about Gentoo support in Puppet. Given the complexity of the package management in Gentoo, I didn’t have high expectations, and I wasn’t wrong. The internal Puppet Portage provider has pretty basic support, so I had to get my hands dirty and write something decent. Then I stumbled upon Adrien’s puppet-portage repo, which made me felt like I hit the jackpot. I started testing it to understand the functionality it covers, and meanwhile I spent some time reading books and experimenting with Puppet and Ruby, as both of them were pretty new to me.

Right after the Gentoo Miniconf I finally contacted Adrien and expressed my interest in developing more functionality to the module. In 3 – 4 months of hard work we managed to have all the functionality I had in mind, which is impressive. It was a great journey, as I had the chance to do quite some stuff in a technology that was completely unknown to me. Adrien was very helpful in giving me correct directions, and I even managed to write a native type and provider from scratch! Thanks a lot Adrien!

Special thanks also to Zac Medico, main Portage developer, for all his valuable input.

Features

Let’s see all the functionality this module provides. Most of the info below is stolen from the README file.

1) /etc/portage/package.*/*

There is a set of native providers that add support of handling entries for
/etc/portage/package.keywords, /etc/portage/package.use, /etc/portage/package.mask, /etc/portage/package.unmask. Examples:

package_use { 'app-admin/puppet':
  use     => ['flag1', 'flag2'],
  target  => 'puppet-flags',
  version => '>=3.0.1',
  ensure  => present,
}

“$use” can be either a string or an array of strings.

package_keywords { 'app-admin/puppet':
  keywords => ['~x86', '-hppa'],
  target   => 'puppet',
  version  => '>=3.0.1',
  ensure   => present,
}

“$keywords” can be either a string or an array of strings.

package_unmask { 'app-admin/puppet':
  target  => 'puppet',
  version => '>=3.0.1',
  ensure  => present,
}
package_mask { 'app-admin/puppet':
  target  => 'tree',
  version => '>=3.0.1',
  ensure  => present,
}

A few issues for the above providers (all the issues are reported in the repo’s Github tracker, but we decided to leave them for the next milestone)

for target in keywords use mask unmask; do
    if [[ -f /etc/portage/package.$target ]]; then
        mv /etc/portage/package.$target /etc/portage/package.${target}_bak
        mkdir /etc/portage/package.$target
        mv /etc/portage/package.${target}_bak /etc/portage/package.$target/old
    fi
done

2) make.conf

NOTE: Be aware that make.conf has changed location, from /etc/make.conf to /etc/portage/make.conf. Make sure to update your systems as well. Its new location makes much more sense. The Portage module uses the new location by default.

The Portage module provides a custom class to handle your entries in make.conf. Example:

portage::makeconf { 'use':
  content => 'flag1 flag2 flag3'
}

This entry will also trigger rebuild of the affected packages.

portage::makeconf { 'gentoo_mirrors':
  content => 'url1 url2'
}

As stated in issue #56, in a later milestone we will convert this class to a native type/provider, in order to make it more powerful.

3) portage::package

This is another custom class, which acts as a wrapper to the native package resource of Puppet. The following example sums up pretty much all of its functionality:

portage::package { 'app-admin/puppet':
  use              => ['-minimal', 'augeas'],
  use_version      => '>=3.0.1',
  keywords         => ['~amd64', '~x86'],
  keywords_version => '>=3.0.1',
  mask             => '<=2.3.17',
  unmask           => '>=3.0.1',
  target           => 'puppet',
  target_keywords  => 'puppet-keywords',
  ensure           => '3.0.1',
}
  • If no $target_{keywords,use,mask,unmask} is specified, then the value of $target is being used.
  • The variables keywords, mask and unmask also accept the special value ‘all’, that will create versionless entries. (This applies only to portage::package, if you want versionless entries in any of the above package_* types, you can just omit the version attribute.)
  • Any change in portage::package will also trigger the appropriate re-emerge to the affected package.

This class was my ultimate functionality request for Portage support in Puppet generally, thus it’s the part I’ve spent most of my time on.

4) Facts

All make.conf variables and most of the eselect modules are shown by facter:

...
eselect_profile => hardened/linux/amd64
eselect_python => python3.2
eselect_ruby => ruby19
...
portage_portage_tmpdir => /var/tmp
portage_portdir => /usr/portage
portage_python_single_target => python2_7
portage_python_targets => python2_7 python3_2
portage_ruby_targets => ruby19
portage_sync => rsync://rsync.gentoo.org/gentoo-portage
...

Keep in mind though that some of the eselect modules are not being shown as facts on purpose. The reason is that either they are not useful, or they produce too complex output that needs further investigation on how to implement. The blacklisted eselect modules are ‘help’, ‘usage’, ‘version’, ‘bashcomp’, ‘env’, ‘fontconfig’, ‘modules’, ‘news’ and ‘rc’.

5) eselect

The eselect type/provider checks for the current state of an eselect module by reading the variable of the equivalent fact. Examples:

eselect { 'ruby':
  set => 'ruby19',
}

For eselect modules that have submodules (eg php):

eselect { 'php_apache2':
  set => 'php5.3',
}

This pretty much covers everything. I hope it will be useful for the community. Feel free to submit bugs, patches or ideas at the repo’s Github issue tracker.

6 Responses to Puppet Portage module version 2.0

  1. Flo says:

    Why did you decide to move from Cfengine 2 to Puppet?

    Reply
  2. tampakrap says:

    we were looking for upgrade from cfengine 2 to 3, but since the syntax turned out to be pretty different, and since two people were already familiar with puppet, we chose that instead (and I’m really glad about that decision)

    Reply
  3. Pingback: Catchup blogging 201305 - J's blog

  4. JMS says:

    This looks really cool. I’ve been trying it out, however, I consistently run into the following issue:

    Error: /Stage[main]/Phonon/Portage::Package[media-libs/vlc]/Package[media-libs/vlc]: Could not evaluate: Execution of ‘/usr/bin/eix –nocolor –pure-packages –format ‘ [] [] ‘ –exact –category-name media-libs/vlc’ returned 1: No $HOME found in environment.

    $HOME is set in the calling environment, but apparently not within the puppet provider, causing eix to always fail to run. Any ideas on how to get around this? I suppose we could hardcode $HOME in the ruby code, but that doesn’t seem ideal.

    Reply
  5. tampakrap says:

    File a new issue in the issue tracker in github please, you probably hit a bug

    Reply
  6. alexey says:

    The real problem is that vlc package has changed the category name, now it’s media-video/vlc and not media-libs/vlc. The error message is misleading and needs to be fixed though

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>