Advanced Redirection Examples

[Jarret W. Buse]

Advanced Redirection Examples

With some articles, examples are always helpful. With some topics, extra examples may be required. In this case, I will offer extra examples to help anyone who may still have some confusion about the subject.

In two previous articles: Linux+: Linux Shell 13 – Shell Command Redirection and Linux+: Linux Shell 14 – Shell Command Redirection – Advanced I covered redirection commands. I hope to go into more detailed explanations and examples on the subject.

Let's start with the fact that each character is considered to be a literal filename in a redirection.

For example, to perform the command “ls >2” would cause the current directory listing to be generated and sent to a file named “2”. To specify that the “2” is not a filename, but should be standard error, an “&” should be placed before it. The command to send the output to standard error (2) is “ls >&2”.

NOTE: Be aware that there is no space between the “>” and “&” symbol.

Now, let's look a little more closely at the redirection command itself. The first section is the command to be performed. The command section will contain any and all parameters needed to perform the command. The command section is any command text used to perform a command in a shell. For example, to get a listing of the Home directory and the /etc/ folder, the command would be “ls ~ /etc”.

The next section is the redirection File Descriptor (FD), if needed. The File descriptors are as follows:

• FD 0 – standard in (stdin)
• FD 1 – standard out (stdout)
• FD 2 – standard error (stderr)
• & - both standard out and standard error
  

NOTE: I will not cover the Standard Input (0) in this article.

The next item is the redirector itself. The output redirectors are as follows:

• > - out
• >> - append out

After the redirector, the placement of the output need to be specified. In most cases, this is a filename. Any filename which is allowed by the file system is acceptable. Any non-standard characters not accepted as part of the filename can cause an error.

NOTE: Multiple redirectors can be used if multiple outputs will be used.

The basic syntax then is “command file_descriptor redirector filename”.

So, let's look at a few examples to start. Let's start with some basic output of an “ls” command.

1. ls 1 > output
2. ls 1 >> output
3. ls 1 > output 2 > error
4. ls 1 >> output 2 >> error

In example number 1, a listing is made of the current directory. The standard output (1) is sent to a file called “output”. If the file does not exist, it is created. If the “output” file does exist, it will be overwritten.

In example number 2, a listing is made of the current directory. The standard output (1) is sent to a file called “output”. If the file does not exist, it is created. If the “output” file does exist, the listing will be added to the existing information in the file.

In example number 3, a listing is made of the current directory. The standard output (1) is sent to a file called “output”. If the file does not exist, it is created. If the “output” file does exist, it will be overwritten. Standard errors (2) which are error messages, if any are generated, will be sent to a file called “error”. If “error” does not exist it will be created. If the file does exist, it will be overwritten.

In example number 4, a listing is made of the current directory. The standard output (1) is sent to a file called “output”. If the file does not exist, it is created. If the “output” file does exist, the output will be added to the existing information in the file. Standard errors (2) which are error messages, if any are generated, will be sent to a file called “error”. If “error” does not exist it will be created. If the file does exist, error messages will be added to the existing information in the file.

Now, we can look at the “&” symbol as a File Descriptor. The “&” allows for both the Standard Output (1) and standard Error (2) to be redirected as one stream. So, in the example command of “ls &>output” both the output and error messages generated by the command will be streamed to the same file, called “output”.

NOTE: Keep in mind that the filenames can be whatever you want them to be. I use “output” and “error” to help illustrate where the output is going.

It should be noted that the “&” as File Descriptor is allowed in BASH 4+. For BASH versions before version 4, the File Descriptors need to be as follows:

command file_descriptor_1>filename file_descriptor_2>&file_descriptor_1

The command to perform the same action as “ls &>output” would be “ls 1>output 2>&1”. Since the “&” is after the redirector, it signifies that the “1” is not a literal for the filename. The “&1” denotes the File Descriptor 1. Here, the assignment for File Descriptor 1 must be assigned before another is redirected to it. If the following command were run “ls ~ 2>&1 1>output” the Standard Output would still go to the file “output”, but any error messages would simply go to the Standard Error device (the screen).

For a fun test, you can swap FD 1 and FD 2 as follows:

cmd 3>&1 1>&2 2>&3

Here, the command is executed as FD 3 is mapped to FD 1. FD 1 is then mapped to FD 2 and finally FD 3 is mapped to FD 2.

NOTE: File Descriptors up to 1023 can be used for your own assignment as I will discuss momentarily.

Since both the output streams are sent to the screen, you cannot tell which is which. So let's try this example:

ls ~ /etd 1>out5 2>error5 3>&1 1>&2 2>&3

The example should output the Home directory. It should also generate an error since the /etd folder should not exist. If this folder does exist on your system, use a name which is not present.

Two files should be generated. The “out5” file should contain the error message that /etd does not exist. The “error5” file should contain a listing of the Home folder.

To create a custom FD to use as output, the following can be done:

exec 3 >file3

Now, within a terminal or a script, the FD 3 can be used to point to a file called “file3”. Commands can be executed using FD 3 for whatever purpose we require as an output file. Make sure the previous command was executed to create FD 3. For example, to use it as the Standard Out, we could do the following:

ls ~ 1>&3

Here, FD 1 is redirected to custom FD 3 which streams to the file “file3”.

To close a custom FD, perform the following:

exec 3>&-

Once closed, the FD can be recreated, but if you point it to the same file, use “>>” to append and not overwrite it.

To open a custom FD for reading and writing, use the following:

exec 3<>file3

The file can be used for input and output such as searching the file for specific information using grep.

1. exec 3>file3
2. ls ~ 1>&3
3. exec 3>&-
4. exec 3<>file3
5. grep err* <&3
6. exec 3>&-
   

On line 1, we create the custom FD 3 and point it to “file3”. Line 2 performs a list command of the Home directory and places standard out to custom FD 3. Next in line 3, we close FD 3. Line 4 shows the use of FD 3 as input and output for “file3”. On line 5, FD 3 is used as input for the GREP command to search for files with “err” in the filename. Line 6 closes FD 3.

NOTE: If in a terminal, all custom FD will be closed when the terminal is closed.

I hope this helps clear up any issues and makes the subject clearer for all who may find this confusing.