sodcalculator

10 Tech How-To

How to Fix Linux Broken Pipe Errors

Guide 10-56498 · Updated · Step by step

Broken pipe errors in Linux are a common frustration for developers, system administrators, and anyone working with command-line pipelines. They typically appear as Broken pipe or write: Broken pipe and occur when a program writes to a pipe or socket that has been closed by the reader. This guide walks you through the most effective fixes, from simple shell adjustments to code-level solutions, so you can eliminate these errors and make your pipelines robust.

What You’ll Need

  • A Linux terminal (any distribution works).
  • Basic familiarity with the command line and shell syntax.
  • Access to the source code if you are debugging a custom script or program.
  • Optional: strace or gdb for deeper debugging (install via your package manager if needed).

Step-by-Step

  1. Identify the exact error and context. Run your command again and note the full error message. Is it Broken pipe from a shell, or write: Broken pipe from a specific program? Also note which command in the pipeline is failing—this matters because the fix differs.
  2. Understand the cause. A broken pipe happens when a process writes to a pipe or socket whose read end has been closed. The most common scenario is using head or grep -m1 to stop reading early, causing the upstream command to receive SIGPIPE (signal 13). By default, this signal terminates the process, and the shell may print an error.
  3. Use head or tail with --pipe (if available). Some versions of head support a --pipe option that makes it read in larger chunks, reducing the chance of a broken pipe. Try command | head --pipe -n 10 as a quick test.
  4. Redirect stderr to avoid noise. If the error is just a warning and doesn’t break your pipeline, you can suppress it by adding 2>/dev/null to the failing command. For example: yes | head -n 5 2>/dev/null. This is a cosmetic fix but useful when the error is harmless.
  5. Use trap in shell scripts. If you’re writing a shell script and want to ignore broken pipe signals, add trap '' PIPE at the top. This tells the shell to ignore SIGPIPE, so the script won’t terminate abruptly. For example: trap '' PIPE; yes | head -n 3.
  6. For Python scripts, handle BrokenPipeError. In Python, wrap your output code in a try/except block. A common pattern is:
  7. try:
        # your output loop
        sys.stdout.flush()
    except BrokenPipeError:
        devnull = os.open(os.devnull, os.O_WRONLY)
        os.dup2(devnull, sys.stdout.fileno())
        sys.exit(1)
  8. For C/C++ programs, ignore SIGPIPE or handle EPIPE. Add signal(SIGPIPE, SIG_IGN); at the start of your program to ignore the signal, then check the return value of write() and handle EPIPE gracefully. Alternatively, use send() with the MSG_NOSIGNAL flag for sockets.
  9. For Java, catch IOException. When writing to a closed pipe, Java throws an IOException. Catch it and exit cleanly. For example, in a loop that writes to System.out, wrap it in a try-catch and break on IOException.
  10. Use stdbuf to change buffering. Sometimes the issue is that a program buffers output and writes a large chunk after the reader has closed. Use stdbuf -oL to force line-buffering: command | stdbuf -oL head -n 10. This can prevent the broken pipe in some cases.
  11. Check for socket timeouts. If the broken pipe occurs on a network socket (e.g., in a server), it usually means the client disconnected. In your server code, handle EPIPE or BrokenPipeError and close the socket cleanly. Also consider setting SO_KEEPALIVE to detect dead connections.

Common Mistakes

  • Ignoring the error completely. Suppressing stderr with 2>/dev/null can hide real issues. Only do this if you are certain the error is harmless.
  • Using trap in the wrong shell. The trap '' PIPE syntax works in Bash and POSIX shells, but in some older shells the signal name might be SIGPIPE instead of PIPE. Test it.
  • Forgetting to flush buffers. In Python, if you don’t flush stdout, the BrokenPipeError might not appear until the program exits, causing a traceback. Always flush or use sys.stdout.flush().
  • Assuming it’s a hardware issue. Broken pipe is almost always a software issue, not a hardware failure. Don’t waste time checking cables.
  • Overusing stdbuf. Changing buffering can affect performance and behavior. Use it only when necessary.

Tips

  • Use yes to test pipelines. The command yes | head -n 5 is a classic way to reproduce a broken pipe. Test your fixes with this simple pipeline.
  • Set PIPESTATUS to check all commands. In Bash, after a pipeline, echo ${PIPESTATUS[@]} shows the exit status of each command. A status of 141 (128+13) indicates a process killed by SIGPIPE. This helps identify which command is failing.
  • Consider using pv for large data transfers. The pv command can monitor progress and handle broken pipes more gracefully. Use command | pv | head -n 10.
  • In systemd services, set IgnoreSIGPIPE=yes. If your service runs as a systemd unit, add IgnoreSIGPIPE=yes in the [Service] section to prevent the service from crashing on broken pipes.
  • Use set -o pipefail in scripts. This makes a pipeline return the exit status of the last command that failed, which helps in debugging. But note it doesn’t prevent broken pipes.

Troubleshooting

If the error persists after trying the above, follow these steps:

  • Trace the system calls. Use strace -f -e trace=write,read,close,pipe your_command to see exactly which file descriptor is failing. Look for EPIPE in the output.
  • Check for zombie processes. Sometimes a background process might hold a pipe open. Use ps aux | grep your_command to see if any related processes are still running.
  • Test without the pipeline. Run the command directly to see if it works without a pipe. If it does, the issue is with the pipeline, not the command itself.
  • Check file descriptor limits. Use ulimit -n to see the max open files. If you’re running out of file descriptors, you might get broken pipes. Increase with ulimit -n 4096 if needed.
  • Verify the reader is actually reading. If you wrote a custom reader, ensure it reads until EOF and doesn’t close early. Add debugging output to confirm.

Frequently Asked Questions

Why do I get “Broken pipe” when using `head`?

When head reads the number of lines it needs, it exits, closing the read end of the pipe. The producer (e.g., yes) then tries to write and receives SIGPIPE, causing the error. This is normal behavior.

Is a broken pipe error dangerous?

No, it’s usually harmless. It just means the consumer stopped reading. However, in production scripts, it can cause unexpected exits, so it’s best to handle it properly.

How do I ignore broken pipe in a bash script?

Add trap '' PIPE at the start of your script. This tells bash to ignore the SIGPIPE signal, so your script won’t terminate when a pipe breaks.

Why does Python show a traceback on broken pipe?

Python’s default behavior is to raise BrokenPipeError when writing to a closed pipe. If you don’t catch it, the interpreter prints a traceback. Wrap your output code in a try/except to handle it cleanly.

Can I prevent broken pipes by using `stdbuf`?

Sometimes. stdbuf -oL forces line-buffering, which reduces the amount of data written at once, making it less likely that a large write occurs after the reader closes. It’s not a guarantee, but it helps in many cases.

What does exit code 141 mean?

Exit code 141 is 128+13, meaning the process was terminated by signal 13 (SIGPIPE). This is the typical cause of a broken pipe error.

Related

General steps for current mainstream versions; menus can differ by device and version. See the disclaimer or report an error.

More in this bed

The whole Tech How-To bed →