Bash Signal Handling with Trap: EXIT, ERR, INT

Resource management in Linux is a huge consideration for those writing complex server scripts. For the test scripts that we demonstrate on this NameHero site, it hardly matters, but some long-running scripts like backups or compilations can leave huge files behind – and if the script is interrupted for whatever reason, those files can be left corrupted, using up space at best, and at worst, left misconfigured, leading to unpredictable errors down the road. For this reason, scripts in bash often use signal handling for cleanup purposes when they either exit or encounter an error. This helps ensure that a script cleans up after itself.

I’ll cover three types of signals – two are generated from within scripts, and the other is generated by the shell itself.

Three Types of Signals

If a script wants to ensure that it cleans up after itself, it has to watch out for three signals.

The EXIT Signal

As the name suggests, the script receives this signal when it ends – either successfully or unsuccessfully. The script can use this signal to perform actions like clean up temporary files, close database connections, or release other resources like volumes, etc. If you’re using temporary files in your script, it’s good practice to always use the exit signal to clean up.

The ERR Signal

This signal is sent whenever a command exits with a non-zero exit status – the zero indicates success. Note that this is not when the script exits when a non-zero status, but when any command within that script is unsuccessful. As I mentioned in my earlier tutorial on the set -e command, a bash script usually just continues to execute when a command within it returns an error status, and those errors can cascade down towards unpredictable executions of future commands. When you use “set -e”, the script exits whenever an error occurs.

As the command returns a non-success exit status, it generated the ERR signal. Scripts can use this ERR signal to perform cleanup acts.

As the script exits after the error, the EXIT signal is also generated, giving you two opportunities to perform final actions before the script wraps up.

The INT Signal

The previous two signals are known as “pseudo-signals” because they don’t correspond to regular operating system signals and are instead generated by scripts. The INT signal, on the other hand, is a real signal recognized by the operating system. As users, we most commonly generate it using the “Ctrl + c” keyboard combination. We know that this usually interrupts any command and terminates it, rather than merely putting it to sleep like “Ctrl + z” does.

Within scripts, we can trap even INT signals and perform cleanup functions before letting the script exit gracefully. We can use this opportunity to print some messages as well, letting the user know what’s happening.

Other signals can be trapped as well. For example, the “Ctrl + z” signal generates the TSTP signal that we can also trap.

Which Signals Cannot be Trapped?

If you have a paranoid bent of mind, you might be wondering at this point how we can defend against a malicious script that exploits the trap command to refuse to exit. A script can intercept an INT signal issued via “Ctrl+c” and even print a misleading message – something like “Gracefully exiting” and then simply continue to execute! In fact, this was one of the earliest concerns about the trap command, so people were very aware of it.

Addressing these concerns, two signals can’t be trapped. SIGSTOP and SIGKILL

SIGKILL

SIGKILL is the forceful termination of a process. The signal isn’t handled by the script, but by the operating system kernel. The process is given no time warning and no chance at redemption. The OS terminates the process ASAP. For malicious scripts, this is the only thing to do.

The disadvantage of using SIGKILL, of course, is that the process gets no time to perform cleanup actions like closing database connections or deleting temporary files. So you shouldn’t normally use this unless you have to.

The command for SIGKILL is:

kill -SIGKILL <PID>

Where <PID> is the process name.

SIGSTOP

The SIGSTOP signal is used to forcefully stop a process – suspending it, rather than killing it. It’s like the SIGTSTP signal, but SIGSTOP can’t be trapped by a script, so it can’t be ignored. If you want to pause a process forcefully and need some time to evaluate whether or not it’s dangerous, you can use SIGSTOP.

The command for issuing THE SIGSTOP signal is:

kill -SIGSTOP <PID>

Once the process is stopped, you can re-enable it with the SIGCONT signal like this:

kill -SIGCONT <PID>

Or you could terminate it directly if you determine it to be malicious.

How to Trap Signals in Bash

Now that we’ve seen what signals are and why it’s necessary for scripts to be able to handle them, let’s see how it works in practice.

First Use “set -e” for Trapping Errors

If you want to trap ERR signals, then you need to include the following command at the beginning of your script:

set -e

We’d seen earlier how set -e allows you to exit scripts when commands generate an error, and if you want to trap ERR statements, you need to use “set -e”, otherwise, it won’t work. Workarounds exist for allowing us to trap ERR signals without exiting, but they’re messy and beyond our scope right now.

Define Functions for Handling Different Signals

The next step is to tell bash which functions will handle which signals. For example, I’m defining two functions to handle the EXIT and ERR signals:

trap ‘err_handler’ ERR
trap ‘exit_handler’ EXIT

In this code, the script tells bash that if an ERR signal is generated, then the “err_handler” function should be executed. Similarly, when the script exits with an EXIT signal, the “exit_handler” function will perform the final actions.

Once these functions are defined and coded, we can write the rest of the script as is.

A Working Example

Here’s a bare-bones working example of the above process:

#!/bin/bash

set -e

err_handler() {
    echo "ERR trap: An error occurred."
}

exit_handler() {
    echo "EXIT trap: Performing final cleanup."
}

trap 'err_handler' ERR
trap 'exit_handler' EXIT

echo "Script started."

# Successful command
echo "Successful step."
sleep 1

# Failing command (triggers ERR trap, then EXIT trap due to set -e)
echo "Attempting to fail..."
false

# This line is never reached
echo "This line is after the failure."

In this example, we first set “set -e”, and then define our two functions with simple “echo” statements stating that they’re executing. Then we define the signal handlers in the next two lines. Finally, we deliberately fail a command with the “false” statement, ensuring that the script generates an ERR signal, followed by an EXIT signal. Note that the final statement is never executed because of the error in the previous line.

Here’s the output:

As you can see, we trapped both the ERR and EXIT signals, and their respective functions were executed. We could have also trapped the INT signal. But note that when defining the function for trapping INT, we must include an “exit” statement as well, otherwise the script will continue running as if it were a malicious script!

Here’s what an INT function would look like:

graceful_exit() {
    echo "Performing cleanup..."
    exit 1
}
# Set the INT trap
trap 'graceful_exit' INT

In the above code, the INT trap function exits with a non-zero exit code, and thus honors the intent of the INT signal.

Conclusion

For simple scripts that run almost instantly, you don’t need to trap signals to perform cleanups. But if your script is long-running, or uses a lot of resources, then you should free up those resources when the script ends – either normally, or due to an error. For this, you can “trap” the error, exit, and interrupt signals to ensure that your script terminates in a well-behaved manner.