PYTHON
Remove Duplicates from a Python List While Preserving Order
Discover Pythonic ways to remove duplicate elements from a list without altering the original order of the remaining unique items. Essential for maintaining data integrity.
# Original list with duplicates
data = [1, 5, 2, 1, 9, 5, 8, 2, 10]
strings = ['apple', 'banana', 'apple', 'orange', 'banana', 'grape']
# Method 1: Using a temporary set and a list comprehension (common and efficient)
seen = set()
unique_data_comprehension = [x for x in data if x not in seen and not seen.add(x)]
# print(unique_data_comprehension) # [1, 5, 2, 9, 8, 10]
seen_strings = set()
unique_strings_comprehension = [x for x in strings if x not in seen_strings and not seen_strings.add(x)]
# print(unique_strings_comprehension) # ['apple', 'banana', 'orange', 'grape']
# Method 2: Using OrderedDict (Python 3.7+ preserves insertion order for regular dicts)
# Note: For Python < 3.7, you would explicitly use collections.OrderedDict
unique_data_ordereddict = list(dict.fromkeys(data))
# print(unique_data_ordereddict) # [1, 5, 2, 9, 8, 10]
unique_strings_ordereddict = list(dict.fromkeys(strings))
# print(unique_strings_ordereddict) # ['apple', 'banana', 'orange', 'grape']
How it works: This snippet presents two effective methods for removing duplicate elements from a list while ensuring the original order of the unique elements is maintained. The first method uses a `set` to track seen elements in conjunction with a list comprehension for efficiency. The second, more concise method, leverages `dict.fromkeys()`. Since Python 3.7+, regular dictionaries preserve insertion order, making `list(dict.fromkeys(my_list))` a simple and elegant solution for this problem.