Mangling shell options in ebuilds

A long time ago eutils.eclass was gifted with a set of terribly ugly functions to push/pop various variables and shell options. Those functions were written very badly, and committed without any review. As a result, a number of eclasses and ebuilds are now using that code without even understanding how bad it is.

In this post, I would like to shortly summarize how to properly and reliably save states of shell options. While the resulting code is a little bit longer than use of e*_push and e*_pop functions, it is much more readable, does not abuse eval, does not abuse global variables and is more reliable.

Preferable solution: subshell scope

Of course, the preferable way of altering shell options is to do that in a subshell. This is the only way that reliably isolates the alterations from parent ebuild environment. However, subshells are rarely desired — so this is something you’d rather reuse if it’s already there, rather than introducing just for the sake of shell option mangling.

Mangling shopt options

Most of the ‘new’ bash options are mangled using shopt builtin. In this case, the -s and -u switches are used to change the option state, while the -p option can be used to get the current value. The current value is output in the form of shopt command syntax that can be called directly to restore the previous value.

my_function() {
	local prev_shopt=$(shopt -p nullglob)
	# prev_shopt='shopt -u nullglob' now
	shopt -s nullglob
	# ...
	${prev_shopt}
}

Mangling set options

The options set using the set builtin can be manipulated in a similar way. While the builtin support both short and long options, I strongly recommend using long options for readability. In fact, the long option names can be used through shopt with the additional -o parameter.

my_function() {
	local prev_shopt=$(shopt -p -o noglob)
	# prev_shopt='set +o noglob' now
	set -o noglob  # or shopt -s -o noglob
	# ...
	${prev_shopt}
}

Mangling umask

The umask builtin returns the current octal umask when called with no parameters. Furthermore, the -p parameter can be used to get full command for use alike shopt -p output.

my_function() {
	local prev_umask=$(umask)
	# prev_umask=0022 now
	umask 077
	# ...
	umask "${prev_umask}"
}

alternative_function() {
	local prev_umask=$(umask -p)
	# prev_umask='umask 0022' now
	umask 077
	# ...
	${prev_umask}
}

Mangling environment variables

The eutils hackery went as far as to reinvent local variables using… global stacks. Not that it makes any sense. Whenever you want to change variable’s value, attributes or just unset it temporarily, just use local variables. If the change needs to apply to part of a function, create a sub-function and put the local variable inside it.

While at it, please remember that bash does not support local functions. Therefore, you need to namespace your functions to avoid collisions and unset them after use.

my_function() {
	# unset FOO in local scope (this also prevents it from being exported)
	local FOO
	# 'localize' bar for modifications, preserving value
	local bar="${bar}"

	#...

	my_sub_func() {
		# export LC_ALL=POSIX in function scope
		local -x LC_ALL=POSIX
		#...
	}
	my_sub_func
	# unset the function after use
	unset -f my_sub_func
}

Update: mangling shell options without a single subshell

(added on 2016-01-28)

izabera has brought it to my attention that the shopt builtin supports -q option to suppress output and uses exit statuses to return the original flag state. This makes it possible to set and unset the flags without using a single subshell or executing returned commands.

Since I do not expect most shell script writers to use such a long replacement, I present it merely as a curiosity.

my_setting_function() {
	shopt -q nullglob
	local prev_shopt=${?}
	shopt -s nullglob

	#...

	[[ ${prev_shopt} -eq 0 ]] || shopt -u nullglob
}

my_unsetting_function() {
	shopt -q extquote
	local prev_shopt=${?}
	shopt -u extquote

	#...

	[[ ${prev_shopt} -eq 0 ]] && shopt -s extquote
}

GLEP67, or how packages are going to be maintained

The way packages are maintained in Gentoo have been evolving for quite some time already. So far all of that has been happening on top of old file formats which slowly started to diverge from the needs of Gentoo developers, and become partially broken. The concept of herds has become blurry, with confusion in definition between different developers and partial assumption about their deprecation. Maintenance of herd by project has been broken by moving projects to the Wiki. Some projects have stopped using herds, others have been declaring them in metadata.xml in different ways.

The problem has finally reached the Gentoo Council and has been discussed on 2015-10-25 meeting (note: no summary still…). The Council attempted to address different problems by votes, and create a new solution by combining the results of votes. However, finally it decided that it is not possible to create a good specification this way. Instead, the meeting has brought two major points. Firstly, herds are definitely deprecated. Secondly, someone needs to provide a complete, consistent replacement in GLEP form.

This is how GLEP 67 came to be. It was based on results of previous discussion, Council votes and thorough analysis of different problems. It provides a complete, consistent system for maintaining packages and expressing the maintenance information. It has been approved by the Council on 2016-01-10, with two week deadline on preparing to the switch.

Therefore, on 2016-01-24 Gentoo is going to switch to the new maintenance structure described in GLEP 67 completely. The announcement with transition details has been sent already. Instead, I’d like to focus on describing how things are going to work starting from the day GLEP 67 becomes implemented.

Continue reading GLEP67, or how packages are going to be maintained

The Ultimate Guide to EAPI 6

Now that EAPI 6 is Council-approved and pretty close to being deployed, I think it’s about time to write up a not-so-short guide to it. It’s especially important that EAPI 6 is a bit different from the previous EAPIs. It was not only seen as an opportunity to add new features but also to tidy things up a bit and improve strictness.

If you look into the PMS, you’d see that we’d not only added completely new stuff but also ported a few common eclass functions, in a little cleaner form, and new useful phase functions. Furthermore, we used EAPI 6 as an opportunity to finally increase strictness of Portage in regard to PMS, and ban some long-unwanted eclasses.

Therefore, I’d like to ask you — please don’t go hooray-enabling EAPI 6 support in your ebuilds and eclasses. Take a while to think, and do things right. Please think of EAPI 6 as a lifetime opportunity of improving your eclass APIs, as we improved the API provided by Package Managers.

Now, time for a little summary of changes, their implications and a bit of rationale.

Continue reading The Ultimate Guide to EAPI 6

Surround sound over network with Windows 8

I’ve got a notebook with some fancy HD Audio sound card (stereo!), and a single output jack — not a sane way to get surround sound (sure, cool kids use HDMI these days). Even worse, connecting an external amplifier to the jack results in catching a lot of electrical interference. Since I also have a PC which has surround speakers connected, I figured it would be a good idea to stream the audio over the network.

On non-Windows, the streaming would be trivial to setup. Likely PulseAudio on both machines, few setup bits and done. If you are looking for a guide on how to do such a thing in Windows, you’re likely end up setting up an icecast server listening to the stereo mix. Bad twice. Firstly, stereo-only. Secondly, poor latency. Now imagine playing a game or watching a movie with sound noticeably delayed after picture (well, in the movie player you could at least play with A/V delay to work-around that). But there must be another way…

Continue reading Surround sound over network with Windows 8

pshs — the awesome file sharing tool

For a long time I lacked a proper tool to quickly share a few files for a short time. The tools I was able to find either required some setup, installing client counterparts or sending my files to a third-party host. So I felt the need to write something new.

The HTTP protocol seemed an obvious choice. Relatively simple, efficient, with some client software installed almost everywhere. So I took HTTP::Server::Simple (I think) and wrote the first version of publish.pl script. I added a few features to that script but it never felt good enough…

So back in 2011 I decided to reboot the project. This time I decided to use C and libevent, and that’s how pshs came into being. With some development occuring in the last three years, lately I started adding new features aiming to turn it into something really awesome.

Continue reading pshs — the awesome file sharing tool