How to find duplicates in a list
Short answer
Set.add() returns false if the element was already there. Collect such elements into a separate set:
void main() {
var items = List.of("a", "b", "a", "c", "b", "a");
var seen = new HashSet<String>();
var duplicates = new LinkedHashSet<String>();
for (String item : items) {
if (!seen.add(item)) {
duplicates.add(item);
}
}
IO.println(duplicates);
}Output:
[a, b]With a count
Grouping shows how many times each element repeats:
void main() {
var items = List.of("a", "b", "a", "c", "b", "a");
var repeated = items.stream()
.collect(Collectors.groupingBy(s -> s, TreeMap::new, Collectors.counting()))
.entrySet().stream()
.filter(e -> e.getValue() > 1)
.toList();
IO.println(repeated);
}Output:
[a=3, b=2]Are there any duplicates
If you only need a "yes" or "no", compare the sizes of the list and the set:
void main() {
var ids = List.of(1, 2, 3, 2);
IO.println(new HashSet<>(ids).size() != ids.size());
}Output:
trueCommon mistake
Finding duplicates with two nested loops. It works, but on 100,000 elements it performs billions of comparisons. A HashSet walks the list once.
Functions used in this recipe
- HashSet — A set without duplicates backed by a hash table; order is not guaranteed.
- Collectors.groupingBy() — A collector that sorts elements into groups that share a key.
- Collections.frequency() — Counts how many times an element occurs in a collection.
- Set.add() — Adds an element if it is not in the set yet.