Weight: 4

Description: Candidates should be able to redirect streams and connect them in order to efficiently process textual data. Tasks include redirecting standard input, standard output and standard error, piping the output of one command to the input of another command, using the output of one command as arguments to another command and sending output to both stdout and a file.

Key Knowledge Areas:

• Redirecting standard input, standard output and standard error
• Pipe the output of one command to the input of another command
• Use the output of one command as arguments to another command
• Send output to both stdout and a file

Terms and Utilities:

tee
xargs

standard I/O stream

stdin 0
stdout 1
stderr 2

> create new file if not exist; o/w overwrite
>> append exist file; o/w create new file
2> write stderr if exist; o/w overwrite it
2>> append stderr to exist file; o/w create new file
&>  stdout+stderr to exist file; o/w create new file


same
cmd &> file
cmd >& file
cmd > file 2>&1 

ls x* z* &> output         # stderr+stdout go to output
ls x* z* >output 2>&1   # stderr+stdout go to output
ls x* z* 2>&1 >output   # stderr not go to output 

ls x* z* 2> /dev/null 

sort -k2  1 apple
> 2 pear
> 3 banana 
> EOF
1 apple
3 banana 
2 pear

gunzip -c file.tar.bz2 | tar -xvf - 

tar -vcf pam.tar /etc/sysconfig >/dev/null 2>&1
updatedb >& /dev/null &
killall -HUP inetd sendmail &> /dev/null
tar -vcf pam.tar /etc/sysconfig/ >debug 2>debug

tee

print out on screen stdout also to log file

script |tee log
scripttee -a log

find, xargs

find ./ -name "*~" |xarg -d "" rm    # remedy for filename with space

rm find ./ -name "*~"

Quiz questions

1. What are the two methods to capture the output of a process and include it on the command line of another process?
2. How would you send the output of a process to a file named “output” and the errors to a file named “errors”?
3. What would you type in a script to have the standard input of the command defined in the script?
4. How do you send the output of one process as the input of another?
5. How do you send both standard output and standard error output to another program as its standard input?
6. How would you use tee to capture the intermediate results of find /etc before sorting them with sort? (Are there any differences between the unsorted and the sorted versions?)

Answers to quiz questions

1. a) $(process) and process b) xargs
2. process > output 2> errors
3. echo “input” | process; ... or ... process &1 | another-process
6. find /etc | tee unsorted | sort > sorted