coreutils: tee invocation

1 
1 17.1 ‘tee’: Redirect output to multiple files or processes
1 ==========================================================
1 
1 The ‘tee’ command copies standard input to standard output and also to
1 any files given as arguments.  This is useful when you want not only to
1 send some data down a pipe, but also to save a copy.  Synopsis:
1 
1      tee [OPTION]... [FILE]...
1 
1    If a file being written to does not already exist, it is created.  If
1 a file being written to already exists, the data it previously contained
1 is overwritten unless the ‘-a’ option is used.
1 
1    In previous versions of GNU coreutils (v5.3.0 - v8.23), a FILE of ‘-’
1 caused ‘tee’ to send another copy of input to standard output.  However,
1 as the interleaved output was not very useful, ‘tee’ now conforms to
1 POSIX which explicitly mandates it to treat ‘-’ as a file with such
1 name.
1 
11    The program accepts the following options.  Also see ⇒Common
 options.
1 
1 ‘-a’
1 ‘--append’
1      Append standard input to the given files rather than overwriting
1      them.
1 
1 ‘-i’
1 ‘--ignore-interrupts’
1      Ignore interrupt signals.
1 
1 ‘-p’
1 ‘--output-error[=MODE]’
1      Adjust the behavior with errors on the outputs, with the long form
1      option supporting selection between the following MODEs:
1 
1      ‘warn’
1           Warn on error opening or writing any output, including pipes.
1           Writing is continued to still open files/pipes.  Exit status
1           indicates failure if any output has an error.
1 
1      ‘warn-nopipe’
1           This is the default MODE when not specified, or when the short
1           form ‘-p’ is used.  Warn on error opening or writing any
1           output, except pipes.  Writing is continued to still open
1           files/pipes.  Exit status indicates failure if any non pipe
1           output had an error.
1 
1      ‘exit’
1           Exit on error opening or writing any output, including pipes.
1 
1      ‘exit-nopipe’
1           Exit on error opening or writing any output, except pipes.
1 
1    The ‘tee’ command is useful when you happen to be transferring a
1 large amount of data and also want to summarize that data without
1 reading it a second time.  For example, when you are downloading a DVD
1 image, you often want to verify its signature or checksum right away.
1 The inefficient way to do it is simply:
1 
1      wget https://example.com/some.iso && sha1sum some.iso
1 
1    One problem with the above is that it makes you wait for the download
1 to complete before starting the time-consuming SHA1 computation.
1 Perhaps even more importantly, the above requires reading the DVD image
1 a second time (the first was from the network).
1 
1    The efficient way to do it is to interleave the download and SHA1
1 computation.  Then, you’ll get the checksum for free, because the entire
1 process parallelizes so well:
1 
1      # slightly contrived, to demonstrate process substitution
1      wget -O - https://example.com/dvd.iso \
1        | tee >(sha1sum > dvd.sha1) > dvd.iso
1 
1    That makes ‘tee’ write not just to the expected output file, but also
1 to a pipe running ‘sha1sum’ and saving the final checksum in a file
1 named ‘dvd.sha1’.
1 
1    Note, however, that this example relies on a feature of modern shells
11 called “process substitution” (the ‘>(command)’ syntax, above; ⇒
 Process Substitution (bash)Process Substitution.), so it works with
1 ‘zsh’, ‘bash’, and ‘ksh’, but not with ‘/bin/sh’.  So if you write code
1 like this in a shell script, be sure to start the script with
1 ‘#!/bin/bash’.
1 
1    Note also that if any of the process substitutions (or piped stdout)
1 might exit early without consuming all the data, the ‘-p’ option is
1 needed to allow ‘tee’ to continue to process the input to any remaining
1 outputs.
1 
1    Since the above example writes to one file and one process, a more
1 conventional and portable use of ‘tee’ is even better:
1 
1      wget -O - https://example.com/dvd.iso \
1        | tee dvd.iso | sha1sum > dvd.sha1
1 
1    You can extend this example to make ‘tee’ write to two processes,
1 computing MD5 and SHA1 checksums in parallel.  In this case, process
1 substitution is required:
1 
1      wget -O - https://example.com/dvd.iso \
1        | tee >(sha1sum > dvd.sha1) \
1              >(md5sum > dvd.md5) \
1        > dvd.iso
1 
1    This technique is also useful when you want to make a _compressed_
1 copy of the contents of a pipe.  Consider a tool to graphically
1 summarize disk usage data from ‘du -ak’.  For a large hierarchy, ‘du
1 -ak’ can run for a long time, and can easily produce terabytes of data,
1 so you won’t want to rerun the command unnecessarily.  Nor will you want
1 to save the uncompressed output.
1 
1    Doing it the inefficient way, you can’t even start the GUI until
1 after you’ve compressed all of the ‘du’ output:
1 
1      du -ak | gzip -9 > /tmp/du.gz
1      gzip -d /tmp/du.gz | xdiskusage -a
1 
1    With ‘tee’ and process substitution, you start the GUI right away and
1 eliminate the decompression completely:
1 
1      du -ak | tee >(gzip -9 > /tmp/du.gz) | xdiskusage -a
1 
1    Finally, if you regularly create more than one type of compressed
1 tarball at once, for example when ‘make dist’ creates both
1 ‘gzip’-compressed and ‘bzip2’-compressed tarballs, there may be a better
1 way.  Typical ‘automake’-generated ‘Makefile’ rules create the two
1 compressed tar archives with commands in sequence, like this (slightly
1 simplified):
1 
1      tardir=your-pkg-M.N
1      tar chof - "$tardir" | gzip  -9 -c > your-pkg-M.N.tar.gz
1      tar chof - "$tardir" | bzip2 -9 -c > your-pkg-M.N.tar.bz2
1 
1    However, if the hierarchy you are archiving and compressing is larger
1 than a couple megabytes, and especially if you are using a
1 multi-processor system with plenty of memory, then you can do much
1 better by reading the directory contents only once and running the
1 compression programs in parallel:
1 
1      tardir=your-pkg-M.N
1      tar chof - "$tardir" \
1        | tee >(gzip -9 -c > your-pkg-M.N.tar.gz) \
1        | bzip2 -9 -c > your-pkg-M.N.tar.bz2
1 
1    If you want to further process the output from process substitutions,
1 and those processes write atomically (i.e., write less than the system’s
1 PIPE_BUF size at a time), that’s possible with a construct like:
1 
1      tardir=your-pkg-M.N
1      tar chof - "$tardir" \
1        | tee >(md5sum --tag) > >(sha256sum --tag) \
1        | sort | gpg --clearsign > your-pkg-M.N.tar.sig
1 
1    An exit status of zero indicates success, and a nonzero value
1 indicates failure.
1