Duplicate customer rows in a data migration: common fixes

Duplicate customer rows can inflate your migration, create repeated accounts, or make later updates hard to reconcile. The tricky part is that two rows with the same name are not always the same customer, and two records for one customer may contain different details.

Start with a copy of the source CSV, decide which field identifies a customer, then check and clean the file before importing it. The CSV Editor can help you find, sort, edit, and remove duplicate rows in your browser.

Check the customer rows before importing

1. Make a working copy of the source CSV and keep the original unchanged. Open the copy in the CSV Editor so you can inspect the data without changing your source file.
2. Check the column headers and identify the strongest customer key available. A stable customer ID is usually safer than a name. Email may help when IDs are missing, but it can be shared, changed, or entered differently.
3. Sort by the chosen key, or search for values you already suspect are repeated. Look at the complete rows around each match, not just the name or email field.
4. Remove exact duplicate rows with the duplicate-removal feature. If repeated records have different values, inspect them individually before deciding whether to keep, edit, merge, or remove a row.
5. Save the cleaned copy, then check the row count and review a sample of records before importing. If the migration has a staging area, import there first and verify the results before moving the data into production.

Do not treat a lower row count as proof that the file is correct. The goal is one appropriate record per customer, with useful details preserved and no unrelated customers combined.

What the repeated rows mean

The complete rows are identical

This is the clearest case for removing a duplicate. The same customer and the same values appear more than once, often because the source export included repeated records or an earlier import was run twice. Keep one copy, but retain the original file in case you need to trace the change.

The customer ID matches, but some details differ

These rows likely refer to one customer with conflicting or updated information. Do not simply remove one. Decide which source is authoritative for each field, such as the newest verified phone number or the approved billing address, and combine the correct values into one record.

The name or email matches, but the customer IDs differ

This may be a true duplicate, a shared email address, a family account, or two people with similar names. Check other fields and the source system before merging. Names alone are weak identifiers, and a shared address or inbox does not prove two rows belong to one person.

Rows look repeated only after formatting differences

Values such as alex@example.com and Alex@example.com may look like separate values because of capitalization or an extra space. Normalize the values you plan to compare, then review matches again. Keep the original values available so you can confirm that cleanup did not alter meaningful information.

How to fix duplicate records safely

1. Stop before importing the file again. If the destination already contains some of these customers, a second import may create another set of duplicates. Check the migration status and find out whether it updates existing records or always creates new ones.
2. Choose a stable match key. Use a unique customer ID from the source when possible. If the destination uses different IDs, create or maintain a mapping between source IDs and destination IDs rather than assuming that matching names identify the same person.
3. Make comparisons consistent. Review spaces, capitalization, punctuation, and formatting in candidate fields such as email or phone number. Use the CSV Editor to find, edit, and sort values. Avoid merging records based only on a similar name or address.
4. Set a rule for conflicting fields. Before merging, decide which source wins when values disagree. For example, you might trust the latest verified contact detail, while preserving a billing value from the system designated as authoritative. Record the rule so another migration run produces the same result.
5. Remove only confirmed duplicates. Use the CSV Editor's duplicate-removal feature for identical rows. For records that share a key but differ in other fields, compare the full rows, retain the right values, and remove a row only after confirming it represents the same customer.
6. Test the cleaned file in a safe destination. If available, run a small sample or use a staging database first. Check that the expected customer count is reasonable, that IDs are unique where required, and that related orders or accounts still point to the right customers.
7. Make repeat imports safe. Configure the migration to match existing records by a stable key and update them, instead of creating a fresh record every time. This behavior is often called an upsert. Test it on a small sample before using it for the full migration.

Why duplicates return after cleanup

Cleaning a CSV fixes the file in front of you, but it does not automatically correct the source system or migration rules. Duplicates can return when users create a second account, an export combines overlapping sources, or each migration run inserts rows without checking for an existing customer.

For a repeatable migration, keep a record of the matching key and merge rules, and use the same rules on every run. A CSV editor is useful for inspecting and correcting a file, but it does not resolve relationships across database tables or decide which customer record your business should trust.

The CSV Editor processes the file in your browser, and the file stays on your device.