All tools

    Remove Duplicate Lines

    Paste a list and every repeated line is deleted, keeping the first occurrence of each and the original order. The tool shows how many duplicates it removed, and you can choose whether "Apple" and "apple" count as the same line.

    Characters
    0
    Words
    0
    Lines
    0

    Your text never leaves your device.

    How to use

    1. Paste your list with one item per line: emails, keywords, URLs, names.
    2. Choose the options: Case-sensitive keeps "Apple" and "apple" apart; Ignore spaces at line ends treats "apple " and "apple" as the same.
    3. Copy the deduplicated list. The status line tells you how many lines were removed.

    Examples

    • Before
      apple banana Apple apple cherry banana
      After
      apple banana Apple cherry
    • Before
      apple banana Apple cherry banana
      After
      apple banana cherry

    How lines are compared

    Two lines are duplicates when they contain the same text. With the default settings:

    • Case matters. “Apple” and “apple” are different lines.
    • Spaces at the ends do not. “apple “ with a trailing space matches “apple”.
    • Accents are compared by meaning. An “é” typed as one character and an “é” built from “e” plus a combining accent look the same and are treated as the same, because both lines are normalized before comparing.

    Empty lines are never treated as duplicates, so the blank lines between groups of items stay where they are. To remove them, use remove empty lines.

    Typical uses

    • Email and contact lists merged from several exports, where the same address appears many times.
    • Keyword lists collected from different SEO tools.
    • URLs from crawls and logs, before you check them one by one.
    • Tags, product codes and IDs pasted from spreadsheets.

    In the first example above, “Apple” stays because case matters by default, “apple “ is removed because its trailing space is ignored, and the second “banana” is removed: two duplicates in total. In the second example, Case-sensitive is turned off, so “Apple” is removed as a copy of “apple”.

    Common mistakes

    • Sorting first and then deduplicating by hand. Deduplicate first; the order of your list is preserved. Sort afterwards only if you need it.
    • Leaving case-sensitivity on for emails. “Anna@example.com” and “anna@example.com” reach the same inbox. Turn Case-sensitive off for addresses and domains.
    • Expecting near-duplicates to be found. “Acme Inc.” and “Acme Inc” differ by a period, so both stay. Clean up punctuation with find and replace first.
    • Pasting comma-separated values. The tool works on lines. Replace “, “ with a line break using find and replace, then deduplicate.

    Frequently asked questions

    How do I remove duplicate lines from a list?

    Paste the list into the box above. Each line that already appeared earlier is deleted, the first copy stays where it was, and the order of the remaining lines does not change. The result is ready to copy, with a count of removed lines.

    Which copy is kept, the first or the last?

    The first. If "banana" appears on lines 2 and 6, line 2 stays and line 6 is removed. The order of everything else is preserved, so the list is not sorted. If you also want it in alphabetical order, run sort lines afterwards.

    Are "Apple" and "apple" duplicates?

    With Case-sensitive on (the default), no: both lines are kept. Turn it off to treat them as the same line; the first spelling is the one that stays. For email addresses, which are effectively case-insensitive, turn it off.

    Why were two lines that look identical both kept?

    Usually because of an invisible difference: a space at the end of one line, or a tab. Ignore spaces at line ends (on by default) handles spaces at the start and end. A difference in the middle, such as a non-breaking space between words, still makes lines different; run remove extra spaces first to normalize it.

    How do I remove duplicates in Excel or Google Sheets?

    Excel has Data → Remove Duplicates, and Google Sheets has Data → Data cleanup → Remove duplicates, or the formula =UNIQUE(A:A). Both compare cells without regard to case. For a list in a text file or an email, pasting it here is faster.

    • Sort Lines Alphabetically

      Paste a list with one item per line and it is sorted alphabetically as you type. Switch to Z → A, natural order (item 2 before item 10), length, reverse or shuffle with one click; accents and capitals sort where a reader expects them.

    • Remove Empty Lines

      Paste text and every blank line is deleted, including lines that look empty but contain spaces or tabs. All other lines stay exactly as they were, and the tool tells you how many lines it removed.

    • Remove Line Breaks

      Paste text with hard line breaks, for example from a PDF, and every line is joined into flowing text: "the lazy⏎dog" becomes "the lazy dog". Choose whether lines are joined with a space or with nothing, and whether paragraphs stay apart.

    • Remove Extra Spaces

      Paste text and every run of two or more spaces becomes a single space, tabs become spaces, and spaces at the start and end of each line are trimmed. Line breaks are kept, so lists and paragraphs keep their shape.

    • Find and Replace Text

      Type what to find and what to replace it with, and every match in your text is replaced as you type. Use plain text for simple swaps, or regex mode for patterns such as dates, emails and repeated spaces; the tool shows how many matches it replaced.