Linux Terminal Command: uniq

The uniq command is an essential tool in File Manipulation & Viewing. In this tutorial, we will explore what uniq does, look at everyday examples, and cover advanced options to supercharge your command-line workflow.

Concept & Explanation

The uniq command filters out adjacent duplicate lines from input streams.

Common Options & Syntax

uniq [options] [arguments]

Here are the most common flags used with uniq:

Simple Usage: Basic default commands.
Detailed View: Shows diagnostic information.
Advanced Actions: Can chain parameters for scripting.

1. Interactive Example (Simple)

Here is how most people run the command:

# Example
uniq data.txt

What it does: Omits adjacent duplicate lines from ‘data.txt’.

2. Power-User Example (Advanced)

For scripting and advanced diagnostics, use this configuration:

# Advanced
sort ip_logs.txt | uniq -c | sort -nr

What it does: Sorts logs, counts occurrences of each duplicate line (-c), and sorts them numerically in reverse to show the most frequent entries.

⚙️ Warning & Common Pitfalls

[!WARNING] uniq only removes adjacent duplicate lines. If duplicates are scattered, they will not be removed. Always run sort before uniq to group duplicates.

Here are some related posts on cli_tty1 you might want to check out:

sort : Sort lines of text files.