Lesson  in  Test Linux for DevOps Engineers

Finding Text and Files

Learn how to search for specific patterns grep and locate files and directories using find.

Finding Text in Files with grep

🎯 Learning Objective

Master the grep command to search for specific text patterns within files, a fundamental skill for debugging, analyzing logs, and finding information on the command line.

📚 Concept Introduction

Imagine you're troubleshooting a problem and need to find every line containing the word "ERROR" in a log file with thousands of entries. Or maybe you need to find a specific configuration setting in a file you didn't write. Manually reading through them would be impossible.

This is the problem grep (Global Regular Expression Print) solves. It's a powerful command-line utility that scans files or input and prints out lines that match a specific pattern.

📁 Pre-created for this unit:

  • A file named story.txt - Contains simple text for basic pattern matching.
  • A file named data.log - A mock log file for practicing log analysis.

The basic syntax for grep is straightforward. It's best practice to always wrap your search pattern in double quotes to prevent the shell from interpreting special characters unexpectedly.

Syntax: grep "pattern" filename

Think of grep as your digital detective - it reads through every line of a file looking for your specified pattern and reports back with exactly what it found. This simple search is incredibly powerful for quick investigations.

Let's find all lines in story.txt that contain the word "fox":

grep "fox" story.txt

This command scans every line and prints only those containing "fox". Notice how grep preserves the entire line context - you're not just getting the word "fox" but the complete sentence it appears in.

◆ Essential grep Options

You can modify grep's behavior with options (or flags). You can even combine them, like grep -in "pattern" filename. These options transform grep from a simple search tool into a sophisticated pattern-matching engine.

OptionDescriptionUse Case
-iIgnore case (case-insensitive)Finding "Error", "ERROR", or "error"
-nShow line numbersLocating exact positions in files
-vInvert match (show non-matching lines)Filtering out unwanted entries
-cCount matching lines onlyGetting statistics without seeing content
-wMatch whole words onlyFinding "cat" but not "category"
-rRecursive search through directoriesSearching entire project folders
-lList filenames with matches onlyFinding which files contain patterns

Sometimes you don't know the exact capitalization of what you're looking for:

grep -i "house" story.txt

This finds "house", "House", "HOUSE", or any other capitalization variation. This is essential when searching through files where capitalization isn't consistent.

Show Line Numbers

When you need to know exactly where matches occur:

grep -n "fox" story.txt

This prefixes each matching line with its line number, making it easy to locate and edit specific content later.

Count Matches Only

To get just the number of matching lines without seeing the content:

grep -c "fox" story.txt

This outputs only a number, perfect for scripts or when you just need statistics.

◆ Anchoring to the Start or End of a Line

Sometimes you need more precision than just "contains this pattern." Anchoring lets you specify exactly where in the line your pattern should appear.

AnchorDescriptionExample
^patternPattern at the start of linegrep "^The" story.txt
pattern$Pattern at the end of linegrep "end.$" story.txt

Start of Line

To find lines that begin with a specific word:

grep "^The" story.txt

This only matches lines where "The" is the very first word, not lines where "The" appears in the middle.

End of Line

To find lines ending with specific text:

grep "window.$" story.txt

This matches lines ending with "window." (including the period). The $ ensures you're finding lines that actually end with this pattern, not lines where it appears elsewhere.


📋 Essential Command Reference

CommandPurposeDevOps Use Case
grep "pattern" fileBasic text searchFinding specific error messages in application logs.
grep -i ...Case-insensitive searchFinding a term (Error, error, ERROR) regardless of capitalization.
grep -c ...Count matching linesCounting the number of failed login attempts in an auth log.
grep -r ...Recursive searchLocating a specific API endpoint or variable across an entire codebase.
grep -v ...Invert matchHiding routine messages (INFO, DEBUG) to focus on warnings or errors.
grep ^...Anchor to start of lineIsolating log lines that start with a specific severity level (e.g., ^ERROR).

💡 Key Takeaways

  • grep is your primary tool for searching for text within files from the command line.
  • Always enclose your search pattern in quotes (e.g., "my pattern").
  • Use options like -i (ignore case), -n (line number), -c (count), and -v (invert) to refine your search.
  • Use -r to search recursively through entire directories.
  • Use ^ and $ to anchor your search to the start or end of a line, respectively.

Mastering grep is a cornerstone of command-line productivity, enabling you to quickly diagnose issues, audit configurations, and process large amounts of text data with precision.

Finding Files and Directories with find

🎯 Learning Objective

Master the basic features of the find command to locate files and directories based on their name, type, and path, a crucial skill for navigating complex file systems.

📚 Concept Introduction

While grep searches for text inside files, find searches for the files and directories themselves. Imagine you know a file named report.txt exists somewhere in a project folder, but you don't know exactly where. The find command is the perfect tool to locate it. It recursively searches through a directory tree and filters items based on criteria you provide.

📁 Pre-created for this unit:

  • A directory structure at /home/laborant/search_area/ containing various files and subdirectories to practice searching.

◆ Basic find Syntax

The find command is structured as follows:

Syntax:

find [path_to_search] [expression]
  • [path_to_search]: Where find should start looking (e.g., ., /home/laborant, /).
  • [expression]: The test or condition to apply (e.g., -name "filename").

Think of find as a tireless assistant that walks through every directory and subdirectory, examining each file and folder against your criteria. Unlike simple directory listings, find digs deep into nested folders, making it perfect for complex file systems.

◆ Finding by Name

The most common use of find is to search by name. This is your go-to approach when you know what a file is called but not where it lives.

  • -name: Performs a case-sensitive search.
  • -iname: Performs a case-insensitive search.

To find a file named exactly report.txt within the search_area directory:

find /home/laborant/search_area -name "report.txt"

This command starts at /home/laborant/search_area and recursively examines every subdirectory for files named exactly "report.txt". The quotes around the filename are important - they prevent the shell from expanding special characters.

To find all files named report.txt regardless of case (report.txt, Report.txt, etc.):

find /home/laborant/search_area -iname "report.txt"

◆ Finding by Type

Sometimes you want to distinguish between files and directories, or find only specific types of items:

TypeDescriptionExample
-type fRegular files onlyfind . -type f -name "*.txt"
-type dDirectories onlyfind . -type d -name "backup*"

To find only directories (folders) that start with "project":

find /home/laborant/search_area -type d -name "project*"

This helps you locate organizational structures without getting cluttered by individual files.

◆ Using Wildcards with find

You can use wildcards to match patterns, but remember to quote them properly:

To find all text files (files ending in .txt):

find /home/laborant/search_area -name "*.txt"

The asterisk (*) matches any number of characters. The quotes prevent your shell from expanding the pattern before find sees it.

◆ Combining Expressions

The real power of find comes from combining expressions to create highly specific searches. find assumes an "AND" logic, meaning all conditions must be met.

To find a directory that is also named "config_files":

find /home/laborant/search_area -type d -name "config_files"

This combines type filtering (-type d for directories only) with name matching config_files. The ability to chain expressions makes find incredibly powerful for complex file management tasks.


📋 Essential Command Reference

CommandPurposeDevOps Use Case
find . -name "app.log"Finds files with an exact name in the current directory.Locating a specific log file without knowing its full path.
find /etc -iname "*.conf"Finds files by name, ignoring case.Finding all configuration files (.conf, .CONF) in the /etc directory.
find . -type dFinds all directories.Listing all subdirectories of a project to understand its structure.
find . -type fFinds all regular files.Getting a complete list of all files in a directory tree, excluding subdirectories.
find /var/log -type f -name "*.log"Combines tests to find files of a certain type and name.Locating all files ending in .log within the /var/log directory structure.

💡 Key Takeaways

  • find locates files and directories based on their attributes, not their content.
  • Use -name for case-sensitive searches and -iname for case-insensitive searches.
  • The -type flag lets you filter for files (f), directories (d), or other types.
  • You can combine expressions to create very specific search queries.
  • Be mindful of your starting path; searching from / can be slow and generate permission errors.

Mastering these basic find expressions is the first step toward efficiently navigating and managing even the most complex Linux file systems.

Advanced File Searching with find

🎯 Learning Objective

Master advanced find expressions to locate files based on size, modification time, and permissions, enabling you to perform sophisticated system administration and cleanup tasks.

📚 Concept Introduction

Searching by name and type is just the beginning. The true power of find is its ability to locate files based on their metadata. Imagine you need to find all configuration files modified in the last 24 hours to audit a recent change, or you need to locate all files larger than 100MB to free up disk space. These are the kinds of complex questions find can answer.

📁 Pre-created for this unit:

  • A directory structure at /home/laborant/search_area/ with files of various sizes, permissions, and timestamps.

◆ Finding by Size (-size)

A common task is finding files that are taking up too much space. The -size expression filters files based on their size. This is invaluable for disk space management and identifying files that might need archiving or cleanup.

Syntax:

find [path] -size [N][unit]
  • Prepend N with + to find files larger than N.
  • Prepend N with - to find files smaller than N.
UnitMeaning
cbytes
kKilobytes (1024 bytes)
MMegabytes
GGigabytes

To find all regular files in the search area larger than 1 Kilobytes:

find /home/laborant/search_area -type f -size +1k

◆ Finding by Modification Time (-mtime)

Often you need to find files based on when they were last modified. This is crucial for backup operations, security audits, or cleanup tasks.

The -mtime option measures time in days:

ExpressionMeaning
-mtime +7Modified more than 7 days ago
-mtime -7Modified less than 7 days ago
-mtime 7Modified exactly 7 days ago

To find files modified in the last 2 days:

find /home/laborant/search_area -type f -mtime -2

This is perfect for identifying recently changed configuration files after system updates, or finding files that were modified during a specific troubleshooting session.

◆ Limiting Search Depth (-maxdepth)

Sometimes you want to limit how deep find searches into subdirectories. This speeds up searches and prevents them from going into areas you don't care about.

To search only in the current directory:

find /home/laborant/search_area -maxdepth 1 -type f

The command above will only search for files within /home/laborant/search_area itself and will not descend into subdirectories like archive/.

◆ Finding by Permissions (-perm)

You can also search for files based on their permission settings. This is essential for security audits and system administration.

To find files with specific permissions (e.g., readable, writable, and executable by the owner):

find /home/laborant/search_area -type f -perm 755

To find all files with exactly the permissions rw------- (600 in octal), like a private key:

find /home/laborant/search_area -type f -perm 600

📋 Essential Command Reference

CommandPurposeDevOps Use Case
find . -size +100MFinds files larger than 100MB.Identifying large log files that need to be rotated or archived.
find . -mtime -1Finds files modified in the last 24 hours.Auditing which configuration files were changed during a recent deployment.
find . -mmin -60Finds files modified in the last 60 minutes.Finding a file you just saved but cannot remember where.
find . -perm 600Finds files with exact permissions of 600.Locating all private keys on a system to ensure they are properly secured.
find . -maxdepth 1Limits the find search to the current directory.Searching for a file in /etc without getting lost in all its subdirectories.
find . -type f -emptyFinds empty files.Cleaning up zero-byte files created by failed script executions.

💡 Key Takeaways

  • find can search for files using a wide range of metadata, not just names.
  • Use -size to find files based on their size, which is useful for disk cleanup.
  • Use -mtime (days) or -mmin (minutes) to find files based on their last modification time.
  • Use -perm to locate files with specific permission settings, which is great for security audits.
  • Combine multiple expressions (-size, -mtime, -type, -name, etc.) to build highly specific and powerful search queries.

By mastering these advanced find expressions, you can perform detailed system audits, manage files based on their properties, and automate complex administrative tasks.

Previous lesson
File Ownership and Links