1. Home
  2. Tools
  3. Text
  4. Line
  5. Duplicate Line Remover
Share

Remove Duplicate Lines

TEXT   DEDUPLICATION   DATA CLEANSING  
Duplicate line remover is an online text mechanic to remove duplicate lines of text. Lines can even be compared in a case insensitive way. Empty or blank lines can be deleted & whitespace can be trimmed. All the removed lines can be seen or checked along with their line numbers using this tool.


Input: Paste text below



Settings



Background Information

We all use text files everyday. Be it a list of file paths, names of people or things to do. There is often a need to remove duplicate lines of text. This tool allows you to do just that. Read more at wikipedia.

Settings Explained

Case Sensitive

Use this to optionally perform a case-insensitive matching when trying to find duplicate lines. This is useful when you are comparing things like windows file paths where case does not matter.

The output stays the same as the original line.
Case Sensitive On
She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog
She sells sea Shells on the sea shore
Case Sensitive Off
She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog
Case Conversion

Case is changed using one of the following styles before duplicate lines are removed

The output will be changed depending on the case conversion rule selected.
None
No case Change
Lower Case
lower case change
Upper Case
UPPER CASE CHANGE
Camel Case
camelCaseChange
Pascal Case
PascalCaseChange
Title Case
Title Case Change
Snake Case
snake-or-kebaberized-case-change
Sentence Case
Sentence case change
Remove Empty Lines
When turned on, empty lines are removed from the output.
Remove Empty Lines On
She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog
She sells sea Shells on the sea shore
Remove Empty Lines Off
She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog

She sells sea Shells on the sea shore
Trim Type
The following trim types are available
None
In this mode no whitespace is trimmed
She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog
   She sells sea Shells on the sea shore
Start
In this mode only the starting whitespace is trimmed or removed
She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog
She sells sea Shells on the sea shore
End
In this mode only the ending whitespace is trimmed or removed
She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog   
   She sells sea Shells on the sea shore
Both
In this mode both the starting and the ending whitespace is trimmed or removed
She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog
She sells sea Shells on the sea shore
Show Removed Lines
If selected, removed lines are shown along with their corresponding line numbers
Remove Double Space
If selected, contiguous or multiple spaces are replaced by a single space before removing the duplicates
➤