Connecting Commands with Pipes and tee

🎯 Learning Objective

Master the pipe operator (|) to chain commands together, creating powerful data processing workflows by connecting the output of one command directly to the input of another.

📚 Concept Introduction

Imagine you're working with a massive log file and need to find the most recent error entries, then view them one page at a time. Without pipes, you'd need to save intermediate results to temporary files, creating clutter and slowing down your workflow.

Pipes solve this elegantly by creating "data highways" between commands. Think of pipes as connecting LEGO blocks - each command does one thing well, and pipes let you combine them into sophisticated operations without any mess.

📁 Pre-created for this unit:

• long_document.txt - A sample file with multiple lines for practicing pipe operations

🔗 Understanding Pipes

In Linux, you can use a pipe (|) to send the output of one command directly as the input to another. This creates seamless data flow without temporary files, enabling powerful one-liners and workflows made of small, simple commands.

The beauty of pipes lies in the Unix philosophy: "Write programs that do one thing and do it well. Write programs to work together." Pipes are the mechanism that makes this cooperation possible.

🛠️ How Pipes Work

Syntax:

command1 | command2

Here's what happens behind the scenes:

1. command1 runs and produces output
2. The standard output (stdout) of command1 is not sent to the screen
3. Instead, it flows directly as standard input (stdin) to command2
4. command2 processes that input and produces its own output

Think of it as an assembly line where each command performs a specific operation on the data before passing it to the next station.

You can chain even more commands:

command1 | command2 | command3

Each command in the chain processes the data and passes the result forward, creating sophisticated data processing pipelines.

🔍 Practical Examples

◆ Viewing File Sections with tail and less

Instead of overwhelming your screen with an entire file, you can preview just the end and scroll through it comfortably:

tail -20 long_document.txt | less

What's happening here:

• tail -20 extracts the last 20 lines from the file
• less receives those 20 lines and provides a scrollable interface
• You can search within those lines (/ to search) and exit cleanly (q)

This pattern is invaluable when monitoring log files where the most recent entries are usually the most relevant.

◆ Extracting Middle Sections with head and tail

You can combine head and tail to view lines from the middle of a file - a powerful technique for sampling data. To display lines 11 to 15 from long_document.txt:

head -15 long_document.txt | tail -5

Breaking this down:

• head -15 takes the first 15 lines from the original file
• tail -5 then takes the last 5 lines from that subset
• The result: lines 11-15 from the original file

This technique is perfect for inspecting specific sections of large files without loading the entire file into an editor. It's commonly used for analyzing structured data files where you need to examine content at specific positions.

📋 Essential Command Reference

💡 Key Takeaways

Pipes transform individual commands into powerful data processing workflows by connecting outputs to inputs seamlessly. They eliminate the need for temporary files and enable the Unix philosophy of combining simple tools to solve complex problems. Mastering pipes is essential for efficient command-line productivity and forms the foundation for advanced Linux automation.

🎯 Learning Objective

Master the tee command to simultaneously save command output to files while viewing it on screen, enabling effective logging and monitoring workflows without losing real-time visibility.

📚 Concept Introduction

Picture this scenario: you're running a critical system update and need to both monitor the progress in real-time and keep a detailed log for later review. Traditional redirection forces you to choose - either see the output or save it, but not both.

The tee command solves this dilemma elegantly. Think of it like a plumbing T-junction that splits water flow in two directions - it takes one stream of data and sends it to both your screen and a file simultaneously.

📁 Pre-created for this unit:

• system_events.log - A sample log file with initial content for practicing append operations

🚰 Understanding tee

Sometimes, you want to save the output of a command to a file and also see it on your screen at the same time. Traditional redirection with > sends output to a file but hides it from your terminal. The tee command bridges this gap perfectly.

The name "tee" comes from plumbing - like a T-shaped pipe fitting that splits flow in multiple directions. This visual metaphor perfectly captures what the command does with data streams.

◆ Basic tee Syntax

The basic format is:

command | tee filename

This accomplishes two things simultaneously:

• Saves the output of command to the specified file
• Displays the same output on your screen

Important: If the file already exists, it will be overwritten by default.

◆ Simple Example: Capturing Timestamps

Let's get the current date and time, display it, and save it to a file:

date | tee current_datetime.log

What happens here:

• The current date and time appears on your screen
• The exact same information gets saved to current_datetime.log
• You maintain visibility while creating a permanent record

📝 Appending with tee -a

If you want to add output to a file without overwriting its existing content, use the -a (append) option with tee. This is crucial for building cumulative log files over time.

Syntax:

command | tee -a filename

This works just like basic tee, but adds new content to the end of the existing file instead of replacing it.

◆ Building Log Files

Let's append a new event to our pre-created system_events.log file.

First, examine the current content:

cat system_events.log

Now, add a new event:

echo "ALERT: Disk space low on /var" | tee -a system_events.log

This adds the new line to system_events.log while preserving existing content. You see the message on screen and know it's been logged permanently.

📋 Multiple File Output

tee can write the same output to multiple files simultaneously - useful for creating backups or sending information to different log systems:

command | tee file1.txt file2.txt file3.txt

◆ Redundant Logging Example

echo "Backup complete." | tee backup.log backup_copy.txt

This writes the message to both backup.log and backup_copy.txt while displaying it on screen. This redundancy ensures critical information isn't lost if one file becomes corrupted or inaccessible.

📋 Essential Command Reference

💡 Key Takeaways

The tee command eliminates the traditional choice between seeing output and saving it by doing both simultaneously. It's essential for logging, monitoring, and debugging scenarios where maintaining real-time visibility is crucial. Whether you're documenting system operations, troubleshooting pipelines, or building audit trails, tee ensures you never lose important information while keeping full visibility of ongoing processes.