Introduction to Stream Editing (sed)

🎯 Learning Objective

Master the fundamental sed substitute command to perform find-and-replace operations on text streams, understanding the difference between viewing output and modifying files directly.

📚 Concept Introduction

Imagine you have a configuration file with hundreds of lines where you need to change every occurrence of "localhost" to "production-server". Manually editing each line would be time-consuming and error-prone. This is where sed (Stream Editor) excels.

sed is a powerful command-line utility that processes text line by line, making it perfect for automated find-and-replace operations, text transformations, and batch file editing. It's an essential tool in system administration and scripting.

📁 Pre-created for this unit:

• sed_practice.txt - Sample text file for practicing output redirection
• sed_practice_inplace.txt - Sample text file for practicing in-place editing

🔧 The Substitute Command

The most commonly used sed feature is the substitute command (s), which performs find-and-replace operations.

◆ Basic Syntax

Syntax:

sed 's/pattern/replacement/flags' filename

• s: The substitute command
• pattern: Text to search for (can be plain text or regex)
• replacement: Text to replace it with
• flags: Optional modifiers (covered below)

Let's start with a simple example - replacing the first occurrence of "old" with "new":

sed 's/old/new/' sed_practice.txt

◆ Common Flags

To replace ALL occurrences of "old" with "new" on each line:

sed 's/old/new/g' sed_practice.txt

💻 Viewing vs. Saving Output

By default, sed displays the modified text on your terminal but does not change the original file. This "preview first" behavior is actually a safety feature - it lets you test your transformations before committing to them.

Think of this as sed asking "Is this what you wanted?" before making permanent changes. This approach prevents accidental data loss and lets you experiment with different patterns until you get exactly the result you need.

◆ Display Output Only

This command shows you what the changes would look like:

sed 's/old/new/g' sed_practice.txt

The original file remains unchanged - you're just previewing the results. This is incredibly valuable when working with important files, as you can verify that your pattern matching works correctly before applying changes.

◆ Save Output with Redirection

To save the modified text to a new file, use output redirection:

sed 's/old/new/g' sed_practice.txt > test_output.txt

This creates a new file test_output.txt with the changes, keeping your original file safe. This approach is often the best practice - it preserves your original data while giving you the transformed version.

✏️ In-Place File Modification

When you want to modify the original file directly, use the -i (in-place) flag. This is powerful but requires caution - once you modify a file in-place, the original content is gone unless you create a backup.

The in-place option is perfect for automated scripts where you're confident in your transformations, or when working with temporary files where preserving the original isn't important.

◆ Without Backup (Use with Caution)

This modifies the original file directly with no backup:

sed -i 's/old/new/g' filename

This approach is fast and efficient, but risky. Use it only when you're certain of your pattern or when the original file isn't critical.

◆ With Backup (Recommended)

This creates a backup before modifying the original:

sed -i.backup 's/old/new/g' filename

The backup file will be named filename.backup. This gives you the convenience of in-place editing with the safety of preserving the original. Many system administrators use this approach routinely when modifying configuration files.

📋 Essential Command Reference

💡 Key Takeaways

sed is a powerful stream editor for text transformations. The substitute command (s) is your primary tool for find-and-replace operations. Always preview changes first, use redirection to save output safely, and create backups when editing files in-place. The global flag (g) ensures all occurrences on each line are replaced, not just the first one.

🎯 Learning Objective

Master sed commands for selectively printing and deleting lines based on line numbers, patterns, and ranges to extract specific content from text files.

📚 Concept Introduction

While substitution is sed's most common use, its line manipulation capabilities are equally powerful. Imagine you need to extract only the error lines from a 10,000-line log file, or remove header lines from a data file before processing. These tasks require precise line selection and deletion.

sed can print specific lines (like a smart head or tail) or delete unwanted lines (like an advanced filter), all based on line numbers, patterns, or ranges you specify.

📁 Pre-created for this unit:

• sed_practice.txt - Sample text file with multiple lines for practicing line operations

Sample content:

The old house has an old door and an old window.
This is a simple test, simple and clear.
Old McDonald had a farm, old and green.
Replace all OLD instances if case is ignored.
Another old example for old times sake.

🔍 Printing Specific Lines

By default, sed prints every line after applying commands. To control exactly what gets printed, you need two components working together. This precise control over output is what makes sed so powerful for extracting specific information from large files.

◆ The -n Flag and p Command

Syntax:

sed -n 'address p' filename

• -n: Suppresses automatic printing of all lines
• p: Explicitly prints only the selected lines
• address: Specifies which lines to target

The -n flag is crucial here - without it, sed would print every line plus the ones you specifically request, giving you duplicated output. Think of -n as telling sed to "be quiet unless I specifically ask you to print something."

Let's start with printing a single line:

sed -n '3p' sed_practice.txt

This prints only line 3 from the file. This technique is incredibly useful when you know exactly which line contains the information you need.

◆ Common Address Patterns

◆ Pattern-Based Printing

To print lines containing specific text (similar to grep):

sed -n '/house/p' sed_practice.txt

This prints all lines containing the word "house". While grep might be more familiar for pattern searching, sed's pattern printing becomes powerful when combined with other sed operations in the same command.

❌ Deleting Lines

The d command removes specified lines from the output. Unlike printing, you don't need the -n flag - deleted lines simply won't appear. This is perfect for filtering out unwanted content from files.

Think of deletion as the opposite of printing - instead of saying "show me only these lines," you're saying "show me everything except these lines." This approach is often more efficient when you want to remove a small number of lines from a large file.

◆ Basic Deletion Syntax

Syntax:

sed 'address d' filename

To delete the first line:

sed '1d' sed_practice.txt

This shows all lines except the first one. This pattern is commonly used to remove header lines from data files before processing.

◆ Common Deletion Patterns

◆ Pattern-Based Deletion

To delete lines containing specific text:

sed '/OLD/d' sed_practice.txt

This removes all lines containing "OLD" (case-sensitive). This technique is invaluable for cleaning up log files by removing irrelevant entries or filtering out known noise from data sets.

◆ Deleting Specific Positions

To remove the last line from a file:

sed '$d' sed_practice.txt

The $ symbol represents the last line in the file. This is particularly useful when you need to remove trailing content like footers or summary lines from generated reports.

📋 Essential Command Reference

💡 Key Takeaways

sed provides precise line control through printing and deletion commands. Use -n with p to print only specific lines, suppressing the default output. Use d to remove unwanted lines. Both commands work with line numbers, ranges, and pattern matching, making them powerful tools for extracting or filtering content from text files.