How to Use the xxd Command in Linux

We use the xxd command in Linux to convert a file into its hex equivalent and to show the hex output side by side with the ASCII representation. This might seem like a niche use case scenario, but it’s surprisingly useful for debugging. There are certain file formats, for example, that are supposed to start with a very specific sequence of bytes, many of which are unprintable ASCII characters, so you won’t be able to verify the correct format if you just look at the ASCII code. But when viewed in a hex format, you can see the exact byte-level representation, allowing you to catch subtle errors that aren’t visible with other tools.

Syntax for the xxd Command

To display a file or an input sequence in hex, we use the following command:

xxd testfile.txt

For example, I have a script file on my server, and if I feed it into xxd as shown above, it looks like this:

You can see that the command shows 16 bytes at a time, starting with the offset on the left-hand column. It then shows me the ASCII representation of those bytes in the right-hand column. In this example, since I’m using a text file as an input, all the hex characters are visible as ASCII representations.

If a particular byte sequence isn’t translatable to ASCII, then the tool prints a “.” instead. One example is the newline character represented by the hexadecimal combination “0a”. On the second line in the above screenshot, we see that it ends with:

3e0a

That’s why you see the line ending with a dot (.) because this is a newline character that can’t be printed using ASCII.

However, let’s see what happens when we have a file that is full of unprintable characters.

In the above screenshot, I’ve generated a file with a lot of unprintable characters that have no ASCII equivalent. You can see that when I use the following command:

cat unprintable.txt

It shows nothing. But when I use:

xxd unprintable.txt

It shows a whole string of hexadecimal characters. On the right-hand side, however, it shows nothing but a series of dots (.), indicating that none of the characters are printable. This then, is the real benefit of the xxd command. When you view a file in hex format, you can see it on a byte level, without leaving anything out. Simply printing it in a text editor is very misleading, since you’ll either miss the majority of the content, or you may see completely garbled text that conveys no information.

Uses of the xxd Command

Let’s look at the many ways in which we can use the xxd command to debug or inspect files that we can’t view in the normal way. This can include things like memory dumps, image files, or even binaries like .exe files. Seeing the hexadecimal representation along with a partial ASCII representation can be very useful.

Verifying the Validity of Files

Many file formats are required to start with a certain sequence for easy identification when a program tries to open them. If the sequence is missing or corrupted, the program will throw an error that can be very hard to debug, because it’s possible that everything else looks normal – it might have the right file extension, for example.

PNG files are a good example to look at. According to the specification, all PNG files must start with the following sequence:

89 50 4E 47 0D 0A 1A 0A

If we have a PNG file with a corrupted header, it won’t work in an image editor, because the editor will think it’s an invalid PNG file.

Here’s an example. I downloaded a PNG file to my server and used the following command:

xxd pngimage.png | head

Here’s the output:

You can see in the above screenshot, that the PNG file is a valid one since it contains the required opening sequence of bytes. Not just that, the ASCII representation of the file also has the first line ending in “IHDR”, which is another requirement of a valid PNG file, since it specifies the dimensions of the image. So using the xxd tool, we can check if a file is built correctly.

Searching for Strings in Binary Files

Binary files are special files that contain mostly non-readable characters. They’re not meant to be readable by humans but by machines. EXE files, for example, are meant to be interpreted directly by humans, and you wouldn’t open one in a text editor. So generally, searching through an EXE file for specific information would be meaningless. However, sometimes you want to parse an EXE file for specific strings anyway. These files don’t contain only non-readable characters, they also contain a smattering of actual strings. But opening them in a text editor like VIM won’t necessarily show them because the unprintable characters cause weird things to happen, and they can mess up VIM’s operations. Even displaying these characters on the terminal screen can cause unexpected behavior.

Similarly, using a command like “cat” and “grep” to search for these strings runs into the same problems. An alternative is to view the file in hex format using xxd, and this works nicely. For example, a file can contain the following hex representation:

0061 646d 696e 00

This contains the word “admin”, but a command like “cat” won’t show it.

In the above example, you can see that the “cat” command doesn’t display anything. This is because of the unprintable characters that cause a problem with cat’s operation. You can probably use other text editors that will show you the word “admin”, but the unprintable characters might interfere with their operation in strange ways. It’s just safer overall to use a tool like xxd that clearly shows you not just the string, but also the precise line number at which it occurs.

For larger binary files, xxd is a clean way to scan for strings that might contain sensitive information like usernames or passwords that might be hardcoded in the file, and which you’re worried about.

Analyzing Memory Dumps

When programs crash, they often leave behind records of the state of their memory before exiting. These are called “memory dumps” and they’re usually files containing the raw data, often in binary form. If you send an error report, you can often choose to send this file to the developers for debugging.

The xxd command is very useful for analyzing memory dumps because their content is mostly unreadable by traditional text editors. Developers who know what to look for can zero in on potential problems that they know exist. For example a command like this:

xxd -s 0x100 -l 64 memdump.bin

This will read 64 bytes from the memory dump file, starting at a specific offset. Without xxd, developers would have a lot of difficulty debugging this file.

Conclusion

While it may seem that xxd has a very niche use, like many Linux tools, it’s invaluable for what it does. Suppose you need to examine the binary content of a file or search for a specific string in a file consisting of mostly non-readable characters. In that case, xxd is the most useful tool, since a text editor won’t do the job.