🔧 Advanced List Deduplicator & Cleaner

×

📚 Help & Instructions

🚀 Getting Started

Paste or type your list in the Input List area (one item per line), then click "Process List" to clean it.

🎯 Deduplication Options

• Remove exact duplicates: Removes items that are exactly the same
• Case-insensitive duplicates: Treats "Apple" and "apple" as duplicates
• Near-duplicates (fuzzy): Removes similar items based on similarity threshold
• Whitespace-only duplicates: Removes items that are only whitespace
• Remove empty entries: Removes blank lines
• Trim whitespace: Removes leading/trailing spaces from items
• Sort while deduplicating: Sorts the list alphabetically
• Preserve original order: Keeps the original order of items

📏 Similarity Threshold

Adjust the slider to set how similar items must be to be considered near-duplicates (0-100%). Higher values mean items must be more similar to be removed.

🔄 Keep First/Last Occurrence

Choose whether to keep the first or last occurrence when duplicates are found.

🔍 Advanced Filtering

• Regex Pattern: Enter a regex pattern to filter items (e.g., ^[A-Z] for items starting with uppercase)
• Pattern to Remove: Enter a text pattern to remove from all items (e.g., http:// to remove URLs)

🔗 List Comparison

Click "Compare Lists" to show the comparison section. You can:

• Find Common Items: Items that exist in both lists
• Find Unique Items: Items that exist in only one list
• Compare Lists: Detailed comparison statistics

💾 Save/Load Rules

Save your current settings as a rule for quick reuse later. Rules are stored in your browser.

📥 Export

Export your cleaned list as a text file or copy it to the clipboard.

🌐 API Access

Use the deduplicatorAPI object in the browser console for programmatic access.