gratislab100% FREE

Remove Duplicate Lines from a List

Paste a list — email addresses, roll numbers, URLs, phone numbers — and this page hands it back with every repeat gone and the order you had it in kept. It also states how many lines went in, how many were unique and how many were dropped, so the arithmetic is yours to check rather than ours to claim.

One entry per line · emails, roll numbers, URLs, anything · the order is kept0 words · 0 characters

Which lines survive, and in what order
When a line repeats, keep

Keep the last one when a list has been added to and the later entry is the corrected one. It is written into the first one's position, so the order you were reading survives.

Line endings in the saved file
Write the result with

Keeping what came in is right almost always: a file that arrived from Windows goes back to Windows and nothing downstream notices a change it did not ask for. Force one when the system at the far end insists.

Nothing uploadedNo sign-upNo watermarkHow that works

How to remove duplicate lines from a list

  1. Paste the list, one entry per line.
  2. Decide whether Ram and ram are the same entry, and whether a stray space at the end counts.
  3. Press Remove the repeats.
  4. Check how many were dropped, then copy the cleaned list out or save it as a .txt.

Two lines that look the same and are not

This is the whole difficulty. A trailing space, one capital letter, or the same word typed two different ways keeps a repeat alive through most tools. An accented word built from a plain letter plus a floating accent, and the same word built from a single accented letter, are different sequences that draw identically on screen — and a phone keyboard and a desktop paste will happily produce one of each in the same column.

So the comparison runs on a tidied copy of each line: the two spellings are folded together first, then trimming and case-folding on top if you asked for them. What comes back out is your original line, character for character. Only the comparison ever sees the tidied version, because silently rewriting your text would be the worst bug this page could have.

Which copy survives, and in what order

By default the first occurrence stays where it was and everything after it goes. Ask it to keep the last one instead and the final version is written into the first one's position, so the order you were reading is the order you get back. That is the setting you want when a list has been appended to and the later entry is the corrected one.

Sorting is optional and starts off. Turned on, numbers sort the way a person expects: item 2 before item 10 rather than after it. You can also invert the whole job and ask for only the lines that appeared more than once, which is how you find out what was duplicated instead of quietly removing it.

What it will not catch

It compares whole lines and nothing smaller. Two rows describing the same person with a different phone number are two different lines, and both survive. A typo is a different line too. There is no fuzzy matching here and there deliberately is not, because a tool that decides two nearly-identical entries are the same one will eventually delete a real record.

Questions

Are blank lines treated as duplicates?
Not unless you ask. Blank lines usually separate the sections of a list, and deleting all but the first would run them together, so they are left where they are. Switch that off and they are compared like any other line.
Will Windows line endings come back as Windows line endings?
Yes. The page works out what the text arrived with and writes the result the same way, so a list pasted out of Notepad and back into a spreadsheet behaves itself. You can force one or the other when the system at the far end insists.
How long a list can it take?
Far more than anybody pastes by hand. The de-duplication is one pass over the lines, and the optional sort reuses a single comparer rather than building a new one for every comparison — which on a long list is the difference between a second and what looks like a frozen page.
Does the list reach a server?
No, it is processed on your own device. That matters for the lists people actually clean up here: a column of customer emails or student roll numbers is not something to hand to a website that uploads it first.

Text & utilities