PYTHON

Deduplicate a List While Preserving Order

Learn to efficiently remove duplicate elements from a Python list while maintaining the original insertion order, using a combination of sets and list traversal.

def deduplicate_list_preserve_order(input_list):
    seen = set()
    deduplicated = []
    for item in input_list:
        if item not in seen:
            seen.add(item)
            deduplicated.append(item)
    return deduplicated

# Example Usage:
my_list = [1, 2, 2, 3, 4, 3, 5, 1]
unique_list = deduplicate_list_preserve_order(my_list)
# unique_list will be [1, 2, 3, 4, 5]
print(f"Original list: {my_list}")
print(f"Deduplicated (order preserved): {unique_list}")

# For simple deduplication where order doesn't matter, you can use:
# unique_set = list(set(my_list))
How it works: This snippet defines a function `deduplicate_list_preserve_order` that takes a list and returns a new list containing only unique elements, maintaining their original order of appearance. It achieves this by iterating through the input list, using a `set` called `seen` to keep track of elements already encountered. If an element has not been seen before, it's added to both the `seen` set and the `deduplicated` result list. This approach ensures efficient `O(1)` average time complexity for checking uniqueness while preserving the element order.

Need help integrating this into your project?

Our team of expert developers can help you build your custom application from scratch.

Hire DigitalCodeLabs