We all use text files everyday. Be it a list of file paths, names of people or things to do. There is often a need to remove duplicate lines of text. This tool allows you to do just that. Read more at wikipedia.
Use this to optionally perform a case-insensitive matching when trying to find duplicate lines. This is useful when you are comparing things like windows file paths where case does not matter.
The output stays the same as the original line.She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog
She sells sea Shells on the sea shoreShe sells sea shells on the sea shore
Quick brown fox jumps over the lazy dogCase is changed using one of the following styles before duplicate lines are removed
The output will be changed depending on the case conversion rule selected.No case Changelower case changeUPPER CASE CHANGEcamelCaseChangePascalCaseChangeTitle Case Changesnake-or-kebaberized-case-changeSentence case changeWhen turned on, empty lines are removed from the output.
She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog
She sells sea Shells on the sea shoreShe sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog
She sells sea Shells on the sea shoreThe following trim types are available
In this mode no whitespace is trimmed
She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog
She sells sea Shells on the sea shoreIn this mode only the starting whitespace is trimmed or removed
She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog
She sells sea Shells on the sea shoreIn this mode only the ending whitespace is trimmed or removed
She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog
She sells sea Shells on the sea shoreIn this mode both the starting and the ending whitespace is trimmed or removed
She sells sea shells on the sea shore
Quick brown fox jumps over the lazy dog
She sells sea Shells on the sea shoreIf selected, removed lines are shown along with their corresponding line numbers
If selected, all unique lines along with a count of how many times they appeared is shown
If selected, contiguous or multiple spaces are replaced by a single space before removing the duplicates
This is the main setting as it governs the sorting logic/strategy. The following strategies are available:-
This is the default sorting order which utilizes a case insensitive & culture neutral comparison between items. Numbers are treated as normal ASCII characters. For more information, checkout the wikipedia article on alphabetical order.
007, James Bond
09, John Doe
1, Jane DoeThis is similar to alphabetical sorting, but treats numbers differently. Consecutive numbers are grouped together and treated as a single character. For example, "1, Jane Doe" will appear before "007, James Bond". For more information, checkout the wikipedia article on natural sort order.
1, Jane Doe
007, James Bond
09, John DoeIn this type of sorting, lines are sorted on the basis of their lengths in characters. So, you can make the smaller ones bubble to the top or use the descending sort order to make them sink to the bottom with the larger ones taking the top spots.
1, Jane Doe
09, John Doe
007, James BondThis is useful mainly if you want to reverse the order of the lines in the original input. Sort Order is not supported in this mode.
09, John Doe
1, Jane Doe
007, James Bond007, James Bond
1, Jane Doe
09, John DoeThis sort type randomizes the lines. Sort Order is not supported in this mode.
Some sort types support ascending & descending modes. Setting descending order reverses the order of the items while still applying the sorting logic.
Random & Reverse sort types do not support sort order.
Comments 0