without - remove duplicates from list java 8




How do I remove repeated elements from ArrayList? (20)

I have an ArrayList<String>, and I want to remove repeated strings from it. How can I do this?


If you are using model type List< T>/ArrayList< T> . Hope,it's help you.


Here is my code without using any other data structure like set or hashmap

for(int i = 0; i < Models.size(); i++) {
     for(int j = i + 1; j < Models.size(); j++) {                                
       if(Models.get(i).getName().equals(Models.get(j).getName())){    
                                Models.remove(j);

                                j--;
                            }
                        }
                    }

Code:

List<String> duplicatList = new ArrayList<String>();
duplicatList = Arrays.asList("AA","BB","CC","DD","DD","EE","AA","FF");
//above AA and DD are duplicate
Set<String> uniqueList = new HashSet<String>(duplicatList);
duplicatList = new ArrayList<String>(uniqueList); //let GC will doing free memory
System.out.println("Removed Duplicate : "+duplicatList);

Note: Definitely, there will be memory overhead.


As said before, you should use a class implementing the Set interface instead of List to be sure of the unicity of elements. If you have to keep the order of elements, the SortedSet interface can then be used; the TreeSet class implements that interface.


Here's a way that doesn't affect your list ordering:

ArrayList l1 = new ArrayList();
ArrayList l2 = new ArrayList();

Iterator iterator = l1.iterator();

        while (iterator.hasNext())
        {
            YourClass o = (YourClass) iterator.next();
            if(!l2.contains(o)) l2.add(o);
        }

l1 is the original list, and l2 is the list whithout repeated items (Make sure YourClass has the equals method acording to what you want to stand for equality)


If you don't want duplicates, use a Set instead of a List. To convert a List to a Set you can use the following code:

// list is some List of Strings
Set<String> s = new HashSet<String>(list);

If really necessary you can use the same construction to convert a Set back into a List.


If you want to preserve your Order then it is best to use LinkedHashSet. Because if you want to pass this List to an Insert Query by Iterating it, the order would be preserved.

Try this

LinkedHashSet link=new LinkedHashSet();
List listOfValues=new ArrayList();
listOfValues.add(link);

This conversion will be very helpful when you want to return a List but not a Set.


If you're willing to use a third-party library, you can use the method distinct() in Eclipse Collections (formerly GS Collections).

ListIterable<Integer> integers = FastList.newListWith(1, 3, 1, 2, 2, 1);
Assert.assertEquals(
    FastList.newListWith(1, 3, 2),
    integers.distinct());

The advantage of using distinct() instead of converting to a Set and then back to a List is that distinct() preserves the order of the original List, retaining the first occurrence of each element. It's implemented by using both a Set and a List.

MutableSet<T> seenSoFar = UnifiedSet.newSet();
int size = list.size();
for (int i = 0; i < size; i++)
{
    T item = list.get(i);
    if (seenSoFar.add(item))
    {
        targetCollection.add(item);
    }
}
return targetCollection;

If you cannot convert your original List into an Eclipse Collections type, you can use ListAdapter to get the same API.

MutableList<Integer> distinct = ListAdapter.adapt(integers).distinct();

Note: I am a committer for Eclipse Collections.


In Java 8:

List<String> deduped = list.stream().distinct().collect(Collectors.toList());

Please note that the hashCode-equals contract for list members should be respected for the filtering to work properly.


Java 8 streams provide a very simple way to remove duplicate elements from a list. Using the distinct method. If we have a list of cities and we want to remove duplicates from that list it can be done in a single line -

 List<String> cityList = new ArrayList<>();
 cityList.add("Delhi");
 cityList.add("Mumbai");
 cityList.add("Bangalore");
 cityList.add("Chennai");
 cityList.add("Kolkata");
 cityList.add("Mumbai");

 cityList = cityList.stream().distinct().collect(Collectors.toList());

How to remove duplicate elements from an arraylist


LinkedHashSet will do the trick.

String[] arr2 = {"5","1","2","3","3","4","1","2"};
Set<String> set = new LinkedHashSet<String>(Arrays.asList(arr2));
for(String s1 : set)
    System.out.println(s1);

System.out.println( "------------------------" );
String[] arr3 = set.toArray(new String[0]);
for(int i = 0; i < arr3.length; i++)
     System.out.println(arr3[i].toString());

//output: 5,1,2,3,4


Suppose we have a list of String like:

List<String> strList = new ArrayList<>(5);
// insert up to five items to list.        

Then we can remove duplicate elements in multiple ways.

Prior to Java 8

List<String> deDupStringList = new ArrayList<>(new HashSet<>(strList));

Note: If we want to maintain the insertion order then we need to use LinkedHashSet in place of HashSet

Using Guava

List<String> deDupStringList2 = Lists.newArrayList(Sets.newHashSet(strList));

Using Java 8

List<String> deDupStringList3 = strList.stream().distinct().collect(Collectors.toList());

Note: In case we want to collect the result in a specific list implementation e.g. LinkedList then we can modify the above example as:

List<String> deDupStringList3 = strList.stream().distinct()
                 .collect(Collectors.toCollection(LinkedList::new));

There is also ImmutableSet from Guava as an option (here is the documentation):

ImmutableSet.copyOf(list);

This three lines of code can remove the duplicated element from ArrayList or any collection.

List<Entity> entities = repository.findByUserId(userId);

Set<Entity> s = new LinkedHashSet<Entity>(entities);
entities.clear();
entities.addAll(s);

When you are filling the ArrayList, use a condition for each element. For example:

    ArrayList< Integer > al = new ArrayList< Integer >(); 

    // fill 1 
    for ( int i = 0; i <= 5; i++ ) 
        if ( !al.contains( i ) ) 
            al.add( i ); 

    // fill 2 
    for (int i = 0; i <= 10; i++ ) 
        if ( !al.contains( i ) ) 
            al.add( i ); 

    for( Integer i: al )
    {
        System.out.print( i + " ");     
    }

We will get an array {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}


this can solve the problem:

private List<SomeClass> clearListFromDuplicateFirstName(List<SomeClass> list1) {

Map<String, SomeClass> cleanMap = new LinkedHashMap<String, SomeClass>();
for (int i = 0; i < list1.size(); i++) {
     cleanMap.put(list1.get(i).getFirstName(), list1.get(i));
}
List<SomeClass> list = new ArrayList<SomeClass>(cleanMap.values());
return list;
}

you can use nested loop in follow :

ArrayList<Class1> l1 = new ArrayList<Class1>();
ArrayList<Class1> l2 = new ArrayList<Class1>();

        Iterator iterator1 = l1.iterator();
        boolean repeated = false;

        while (iterator1.hasNext())
        {
            Class1 c1 = (Class1) iterator1.next();
            for (Class1 _c: l2) {
                if(_c.getId() == c1.getId())
                    repeated = true;
            }
            if(!repeated)
                l2.add(c1);
        }

    ArrayList<String> list = new ArrayList<String>();
    HashSet<String> unique = new LinkedHashSet<String>();
    HashSet<String> dup = new LinkedHashSet<String>();
    boolean b = false;
    list.add("Hello");
    list.add("Hello");
    list.add("how");
    list.add("are");
    list.add("u");
    list.add("u");

    for(Iterator iterator= list.iterator();iterator.hasNext();)
    {
        String value = (String)iterator.next();
        System.out.println(value);

        if(b==unique.add(value))
            dup.add(value);
        else
            unique.add(value);


    }
    System.out.println(unique);
    System.out.println(dup);

ArrayList<String> city=new ArrayList<String>();
city.add("rajkot");
city.add("gondal");
city.add("rajkot");
city.add("gova");
city.add("baroda");
city.add("morbi");
city.add("gova");

HashSet<String> hashSet = new HashSet<String>();
hashSet.addAll(city);
city.clear();
city.addAll(hashSet);
Toast.makeText(getActivity(),"" + city.toString(),Toast.LENGTH_SHORT).show();

import java.util.*;
class RemoveDupFrmString
{
    public static void main(String[] args)
    {

        String s="appsc";

        Set<Character> unique = new LinkedHashSet<Character> ();

        for(char c : s.toCharArray()) {

            System.out.println(unique.add(c));
        }
        for(char dis:unique){
            System.out.println(dis);
        }


    }
}

public Set<Object> findDuplicates(List<Object> list) {
        Set<Object> items = new HashSet<Object>();
        Set<Object> duplicates = new HashSet<Object>();
        for (Object item : list) {
            if (items.contains(item)) {
                duplicates.add(item);
                } else { 
                    items.add(item);
                    } 
            } 
        return duplicates;
        }




duplicates