Executing variables that contain shell operators

I touched a little on pipes in a previous post. Here’s a quick example of an echo utility which outputs two lines, and a pipe operator which redirects that output to a grep utility which performs a simple filter to only include lines that contain the word “cat”:

shell$ echo -e 'the cat \n sat on the mat' | grep cat
the cat

Cool - since that worked, what do you think will happen if you do the following?

shell$ cmd="echo -e 'the cat \n sat on the mat' | grep cat"
shell$ ${cmd}

In the above example we’re simply assigning the original utility to a shell variable, and then executing it. So why, then, would the output be this?

shell$ ${cmd}
'the cat
 sat on the mat' | grep cat

This is something that has bitten me in the past when I write shell scripts. What’s happening here is that the shell executes the contents of variable cmd as a single command, which means that everything after echo are treated as arguments to the echo utility, including the pipe.


What we actually need to happen is to have the entire contents of cmd evaluated by the shell so that the shell can create the pipeline between the two utilities. This is where the utility eval comes into play - eval tells the shell to concatenate the arguments and have them executed by the shell.

shell$ eval ${cmd}
the cat

The morale of this story is that if you want to execute a variable that includes any shell constructs (such as the pipe in our example) - then make sure you eval. Examples of shell constructs include redirections (i.e. echo "the cat" > file1.txt), shell conditionals, loops and functions.

About the author

Hadoop in Practice, Second Edition

Alex Holmes works on tough big-data problems. He is a software engineer, author, speaker, and blogger specializing in large-scale Hadoop projects. He is the author of Hadoop in Practice, a book published by Manning Publications. He has presented multiple times at JavaOne, and is a JavaOne Rock Star.

If you want to see what Alex is up to you can check out his work on GitHub, or follow him on Twitter or Google+.

comments powered by Disqus


Full post archive