linux - with - which command use the output of a program as the input of another one?




Shell scripting input redirection oddities (6)

A recent addition to bash is the lastpipe option, which allows the last command in a pipeline to run in the current shell, not a subshell, when job control is deactivated.

#!/bin/bash
set +m      # Deactiveate job control
shopt -s lastpipe
echo "hello world" | read var1 var2
echo $var1
echo $var2

will indeed output

hello
world

Can anyone explain this behavior? Running:

#!/bin/sh
echo "hello world" | read var1 var2
echo $var1
echo $var2

results in nothing being ouput, while:

#!/bin/sh
echo "hello world" > test.file
read var1 var2 < test.file
echo $var1
echo $var2

produces the expected output:

hello
world

Shouldn't the pipe do in one step what the redirection to test.file did in the second example? I tried the same code with both the dash and bash shells and got the same behavior from both of them.


For interactivity, use Zsh. For a while I was the maintainer of the FreeBSD port of the Bash tab-completion scripts, but abandoned it as soon as I tried Zsh for the first time. It can do everything Bash can do but more easily and more elegantly. It also has the nice property of having extremely Bash-like keystrokes, so if you're on a system without Zsh, you'll be able to make do (even if it wouldn't "feel" as nice).

For scripting, use Bourne Shell (sh). It's the POSIX standard scripting language and your scripts are pretty much guaranteed to work everywhere. Bash and Zsh and other shells have nice extensions that you'll miss but those tie you to a specific setup. Ignore that advice for personal-use-only scripts that you're certain you'll never run elsewhere, but remember that it's a real tradeoff that you need to consider.

But in summary, Zsh. I don't know of anyone who's tried it who didn't immediately and permanently switch. It really is that good.


I'm a korn-shell veteran, so know that I speak from that perspective.

However, I am comfortable with Bourne shell, ksh88, and ksh93, and I know which features are supported in which.

For interactive use, take whatever fits your need. Experiment. I like being able to use the same shell for interactive use and for programming.

I went from ksh88 on SVR2 to tcsh, to ksh88sun and ksh93. I tried bash, and hated it because it flattened my history. Then I discovered

shopt -s lithist

and all was well. (The lithist option assures that newlines are preserved in your command history.)

For shell programming, I'd seriously recommend ksh93 if you want a consistent programming language, good POSIX conformance, and good performance, as many common unix commands can be available as builtin functions.

If you want portability use at least both. And make sure you have a good test suite.

There are many subtle differences between shells. Consider for example reading from a pipe:

b=42 && echo one two three four |
    read a b junk && echo $b

This will produce different results in different shells. The korn-shell runs pipelines from back to front; the last element in the pipeline runs in the current process.

Another example illustrating consistency: The echo command itself, which was made obsolete by the split between BSD and SYSV unix, and each introduced their own convention for not printing newlines (and other behaviour). The result of this can still be seen in many 'configure' scripts.

Ksh took a radical approach to that - and introduced the print command, which actually supports both methods (the -n option from BSD, and the trailing \c special character from SYSV)

However, for serious systems programming I'd recommend something other than a shell, like python, perl. Or take it a step further, and use a platform like puppet - which allows you to watch and correct the state of whole clusters of systems, with good auditing.

Shell programming is like swimming in uncharted waters, with bad maps to boot.

Programming in any language requires familiarity with its syntax, its interfaces and behaviour. Shell programming isn't any different.


The difference between Kornshell and Bash are minimal. There are certain advantages one has over the other, but the differences are tiny:

  • BASH is much easier to set a prompt that displays the current directory. To do the same in Kornshell is hackish.
  • Kornshell has associative arrays and BASH doesn't. Now, the last time I used Associative arrays was... Let me think... Never.
  • Kornshell handles loop syntax a bit better. You can usually set a value in a Kornshell loop and have it available after the loop.
  • Bash handles getting exit codes from pipes in a cleaner way.
  • Kornshell has the print command which is way better than the echo command.
  • Bash has tab completions. In older versions
  • Kornshell has the r history command that allows me to quickly rerun older commands.
  • Kornshell has the syntax cd old new which replaces old with new in your directory and CDs over there. It's convenient when you have are in a directory called /foo/bar/barfoo/one/bar/bar/foo/bar and you need to cd to /foo/bar/barfoo/two/bar/bar/foo/bar In Kornshell, you can simply do cd one two and be done with it. In BASH, you'd have to cd ../../../../../two/bar/bar/foo/bar.

I'm an old Kornshell guy because I learned Unix in the 1990s, and that was the shell of choice back then. I can use Bash, but I get frustrated by it at times because in habit I use some minor feature that Kornshell has that BASH doesn't and it doesn't work. So, whenever possible, I set Kornshell as my default.

However, I am going to tell you to learn BASH. Bash is now implemented on most Unix systems as well as on Linux, and there are simply more resources available for learning BASH and getting help than Kornshell. If you need to do something exotic in BASH, you can go on , post your question, and you'll get a dozen answers in a few minutes -- and some of them will even be correct!.

If you have a Kornshell question and post it on , you'll have to wait for some old past their prime hacker like me wake up from his nap before you get an answer. And, forget getting any response if they're serving pudding up in the old age home that day.

BASH is simply the shell of choice now, so if you've got to learn something, might as well go with what is popular.


#!/bin/sh
echo "hello world" | read var1 var2
echo $var1
echo $var2

produces no output because pipelines run each of their components inside a subshell. Subshells inherit copies of the parent shell's variables, rather than sharing them. Try this:

#!/bin/sh
foo="contents of shell variable foo"
echo $foo
(
    echo $foo
    foo="foo contents modified"
    echo $foo
)
echo $foo

The parentheses define a region of code that gets run in a subshell, and $foo retains its original value after being modified inside them.

Now try this:

#!/bin/sh
foo="contents of shell variable foo"
echo $foo
{
    echo $foo
    foo="foo contents modified"
    echo $foo
}
echo $foo

The braces are purely for grouping, no subshell is created, and the $foo modified inside the braces is the same $foo modified outside them.

Now try this:

#!/bin/sh
echo "hello world" | {
    read var1 var2
    echo $var1
    echo $var2
}
echo $var1
echo $var2

Inside the braces, the read builtin creates $var1 and $var2 properly and you can see that they get echoed. Outside the braces, they don't exist any more. All the code within the braces has been run in a subshell because it's one component of a pipeline.

You can put arbitrary amounts of code between braces, so you can use this piping-into-a-block construction whenever you need to run a block of shell script that parses the output of something else.


Bash or KornShell (ksh)?

Bash.

The various UNIX and Linux implementations have various different source level implementations of ksh, some of which are real ksh, some of which are pdksh implementations and some of which are just symlinks to some other shell that has a "ksh" personality. This can lead to weird differences in execution behavior.

At least with bash you can be sure that it's a single code base, and all you need worry about is what (usually minimum) version of bash is installed. Having done a lot of scripting on pretty much every modern (and not-so-modern) UNIX, programming to bash is more reliably consistent in my experience.





dash