Remove Duplicate Lines
Remove repeated lines while keeping the first occurrence of each exact line. Unlike a sort-and-uniq workflow, this operation does not reorder the input.
What this tool actually changes
The tool reads the input as lines, keeps a set of lines already seen, and emits a line only the first time it appears. Equality is exact: capitalization, spaces, and punctuation are part of the comparison.
Edge cases to know
A line with leading spaces is different from the same text without them. An empty line is also a line, so use Remove Empty Lines first if blank lines should disappear. CRLF and LF input are accepted and the output uses LF.
Example
Input: alpha beta alpha gamma beta Output: alpha beta gamma
Good use cases
Pasted identifiers, configuration entries, import lists, log fragments, and other line-oriented data where original order carries meaning.
Workflow tip
Choose the operation that matches your intent. If whitespace is causing unexpected duplicates, normalize it before deduplicating rather than assuming the duplicate operation should ignore formatting.
Frequently asked questions
Does this ignore capitalization?
No. Comparison is exact, so Alpha and alpha are different lines.
Does it preserve order?
Yes. The first occurrence is kept and later identical lines are removed.
What about whitespace?
Whitespace is significant. Trim the input first if leading or trailing whitespace should not distinguish lines.