Bash has many subtle pitfalls, some of\u00a0them being able to live unnoticed for a\u00a0very long time. A\u00a0common example of\u00a0that kind of\u00a0pitfall is ubiquitous filename expansion, or\u00a0globbing. What many script writers forget about to notice is that practically anything that looks like a\u00a0pattern and\u00a0is not\u00a0quoted is\u00a0subject to globbing, including unquoted variables.<\/p>\n
There are two extra snags that add up to this. Firstly, many people forget that not only asterisks ( It’s all just a\u00a0matter of\u00a0running in\u00a0the\u00a0proper directory for the\u00a0result to\u00a0change. Of\u00a0course, it’s often unlikely \u2014 maybe even close to impossible. You can work towards preventing that by\u00a0running in\u00a0a\u00a0safe directory. But in\u00a0the\u00a0end, writing predictable software is\u00a0a\u00a0fine quality.<\/p>\n <\/p>\n Bash provides a\u00a0two major facilities that could help you stop mistakes \u2014 shopts nullglob<\/em> and\u00a0failglob<\/em>.<\/p>\n The\u00a0nullglob option is\u00a0a\u00a0good choice for a\u00a0default for your script. After enabling it, failing filename expansions result in\u00a0no\u00a0parameters rather than verbatim pattern itself. This has two important implications.<\/p>\n Firstly, it makes iterating over optional files easy:<\/p>\n Without nullglob, the above may actually return Secondly, it turns every accidental glob into null. While this isn’t the\u00a0most friendly warning and\u00a0in\u00a0fact it may have very undesired results, you’re more likely to notice that something is going wrong.<\/p>\n The\u00a0failglob option is\u00a0better if you can assume you don’t need to match files in\u00a0its scope. In\u00a0this case, bash treats every failing filename expansion as a\u00a0fatal error and\u00a0terminates execution with an\u00a0appropriate message.<\/p>\n The\u00a0main advantage of\u00a0failglob is that it makes you aware of\u00a0any mistake before someone hits it the\u00a0hard way. Of course, assuming that it doesn’t accidentally expand into something already.<\/p>\n There is also\u00a0a\u00a0choice of\u00a0noglob. However, I wouldn’t recommend it since it works around mistakes rather than fixing them, and\u00a0makes the\u00a0code rely on a\u00a0non-standard environment.<\/p>\n One of\u00a0the\u00a0pitfalls I myself noticed lately is the\u00a0attempt of\u00a0using unquoted variable substitution to do word splitting. For example:<\/p>\n At\u00a0a\u00a0first glance, everything looks fine. I am aware of\u00a0three solutions that can be used to accomplish word splitting without implicit globbing:<\/p>\n Personally, I dislike the\u00a0first two since they require set-and-restore magic, and\u00a0the\u00a0latter also has the\u00a0penalty of\u00a0doing the\u00a0globbing then discarding the\u00a0result. Therefore, I will expand on using While normally Bash has many subtle pitfalls, some of\u00a0them being able to live unnoticed for a\u00a0very long time. A\u00a0common example of\u00a0that kind of\u00a0pitfall is ubiquitous filename expansion, or\u00a0globbing. What many script writers forget about to notice is that practically anything that looks like a\u00a0pattern and\u00a0is not\u00a0quoted is\u00a0subject to globbing, including unquoted variables. There are two extra snags … Continue reading “Bash pitfalls: globbing everywhere!”<\/span><\/a><\/p>\n","protected":false},"author":137,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[3],"tags":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts\/307"}],"collection":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/users\/137"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/comments?post=307"}],"version-history":[{"count":7,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts\/307\/revisions"}],"predecessor-version":[{"id":315,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/posts\/307\/revisions\/315"}],"wp:attachment":[{"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/media?parent=307"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/categories?post=307"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/mgorny\/wp-json\/wp\/v2\/tags?post=307"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}*<\/code>) and\u00a0question marks (?<\/code>) make up patterns \u2014 square brackets ([<\/code>) do that as\u00a0well. Secondly, by\u00a0default bash (and\u00a0POSIX shell) take failed expansions literally. That is, if\u00a0your glob does not match any\u00a0file, you may not even know that you are globbing.<\/p>\nHow to notice mistakes?<\/h2>\n
for f in a\/* b\/* c\/*; do\r\n some_magic \"${f}\"\r\ndone<\/code><\/pre>\na\/*<\/code> if no\u00a0file matches the\u00a0pattern. For this reason, you would need to add an\u00a0additional check for existence of\u00a0file inside the\u00a0loop. With nullglob, it will just \u2018omit\u2019 the\u00a0unmatched arguments. In\u00a0fact, if none of\u00a0the\u00a0patterns match the\u00a0loop won’t be run even once.<\/p>\nWord splitting without globbing<\/h2>\n
for i in ${v}; do\r\n echo \"${i}\"\r\ndone<\/code><\/pre>\n${v}<\/code> contains a\u00a0whitespace-separated list of\u00a0words and\u00a0we iterate over each word. The\u00a0pitfall here is that words in\u00a0${v}<\/code> are subject to filename expansion. For\u00a0example, if\u00a0a\u00a0lone asterisk would happen to be\u00a0there (like v='10 * 4'<\/code>), you’d actually get all files in\u00a0the\u00a0current directory. Unexpected, isn’t it?<\/p>\n\n
shopt -s noglob<\/code> locally,<\/li>\nGLOBIGNORE='*'<\/code> locally,<\/li>\nread<\/code> to perform word splitting.<\/li>\n<\/ol>\nread<\/code>:<\/p>\nread -r -d '' -a words <<<\"${v}\"\r\nfor i in \"${words[@]}\"; do\r\n echo \"${i}\"\r\ndone<\/code><\/pre>\nread<\/code> is used to read from\u00a0files, we can use the\u00a0here string<\/em> syntax of\u00a0bash to feed the\u00a0variable into it. The\u00a0-r<\/code> option disables backslash escape processing that is undesired here. -d ''<\/code> causes read to\u00a0process the\u00a0whole input and\u00a0not\u00a0stop at any\u00a0delimiter (like newline). -a words<\/code> causes it to put the\u00a0split words into array ${words[@]}<\/code> \u2014 and\u00a0since we know how to safely iterate over an\u00a0array, the\u00a0underlying issue is solved.<\/p>\n","protected":false},"excerpt":{"rendered":"