java without - How do I remove repeated elements from ArrayList?




values find (25)

I have an ArrayList<String>, and I want to remove repeated strings from it. How can I do this?


Answers

Here's a way that doesn't affect your list ordering:

ArrayList l1 = new ArrayList();
ArrayList l2 = new ArrayList();

Iterator iterator = l1.iterator();

        while (iterator.hasNext())
        {
            YourClass o = (YourClass) iterator.next();
            if(!l2.contains(o)) l2.add(o);
        }

l1 is the original list, and l2 is the list whithout repeated items (Make sure YourClass has the equals method acording to what you want to stand for equality)


As said before, you should use a class implementing the Set interface instead of List to be sure of the unicity of elements. If you have to keep the order of elements, the SortedSet interface can then be used; the TreeSet class implements that interface.


If you don't want duplicates in a Collection, you should consider why you're using a Collection that allows duplicates. The easiest way to remove repeated elements is to add the contents to a Set (which will not allow duplicates) and then add the Set back to the ArrayList:

List<String> al = new ArrayList<>();
// add elements to al, including duplicates
Set<String> hs = new HashSet<>();
hs.addAll(al);
al.clear();
al.addAll(hs);

Of course, this destroys the ordering of the elements in the ArrayList.


LinkedHashSet will do the trick.

String[] arr2 = {"5","1","2","3","3","4","1","2"};
Set<String> set = new LinkedHashSet<String>(Arrays.asList(arr2));
for(String s1 : set)
    System.out.println(s1);

System.out.println( "------------------------" );
String[] arr3 = set.toArray(new String[0]);
for(int i = 0; i < arr3.length; i++)
     System.out.println(arr3[i].toString());

//output: 5,1,2,3,4


Probably a bit overkill, but I enjoy this kind of isolated problem. :)

This code uses a temporary Set (for the uniqueness check) but removes elements directly inside the original list. Since element removal inside an ArrayList can induce a huge amount of array copying, the remove(int)-method is avoided.

public static <T> void removeDuplicates(ArrayList<T> list) {
    int size = list.size();
    int out = 0;
    {
        final Set<T> encountered = new HashSet<T>();
        for (int in = 0; in < size; in++) {
            final T t = list.get(in);
            final boolean first = encountered.add(t);
            if (first) {
                list.set(out++, t);
            }
        }
    }
    while (out < size) {
        list.remove(--size);
    }
}

While we're at it, here's a version for LinkedList (a lot nicer!):

public static <T> void removeDuplicates(LinkedList<T> list) {
    final Set<T> encountered = new HashSet<T>();
    for (Iterator<T> iter = list.iterator(); iter.hasNext(); ) {
        final T t = iter.next();
        final boolean first = encountered.add(t);
        if (!first) {
            iter.remove();
        }
    }
}

Use the marker interface to present a unified solution for List:

public static <T> void removeDuplicates(List<T> list) {
    if (list instanceof RandomAccess) {
        // use first version here
    } else {
        // use other version here
    }
}

EDIT: I guess the generics-stuff doesn't really add any value here.. Oh well. :)


It is possible to remove duplicates from arraylist without using HashSet or one more arraylist.

Try this code..

    ArrayList<String> lst = new ArrayList<String>();
    lst.add("ABC");
    lst.add("ABC");
    lst.add("ABCD");
    lst.add("ABCD");
    lst.add("ABCE");

    System.out.println("Duplicates List "+lst);

    Object[] st = lst.toArray();
      for (Object s : st) {
        if (lst.indexOf(s) != lst.lastIndexOf(s)) {
            lst.remove(lst.lastIndexOf(s));
         }
      }

    System.out.println("Distinct List "+lst);

Output is

Duplicates List [ABC, ABC, ABCD, ABCD, ABCE]
Distinct List [ABC, ABCD, ABCE]

There is also ImmutableSet from Guava as an option (here is the documentation):

ImmutableSet.copyOf(list);

This is used for your Custom Objects list

   public List<Contact> removeDuplicates(List<Contact> list) {
    // Set set1 = new LinkedHashSet(list);
    Set set = new TreeSet(new Comparator() {

        @Override
        public int compare(Object o1, Object o2) {
            if (((Contact) o1).getId().equalsIgnoreCase(((Contact) o2).getId()) /*&&
                    ((Contact)o1).getName().equalsIgnoreCase(((Contact)o2).getName())*/) {
                return 0;
            }
            return 1;
        }
    });
    set.addAll(list);

    final List newList = new ArrayList(set);
    return newList;
}

Although converting the ArrayList to a HashSet effectively removes duplicates, if you need to preserve insertion order, I'd rather suggest you to use this variant

// list is some List of Strings
Set<String> s = new LinkedHashSet<>(list);

Then, if you need to get back a List reference, you can use again the conversion constructor.


If you want to preserve your Order then it is best to use LinkedHashSet. Because if you want to pass this List to an Insert Query by Iterating it, the order would be preserved.

Try this

LinkedHashSet link=new LinkedHashSet();
List listOfValues=new ArrayList();
listOfValues.add(link);

This conversion will be very helpful when you want to return a List but not a Set.


import java.util.*;
class RemoveDupFrmString
{
    public static void main(String[] args)
    {

        String s="appsc";

        Set<Character> unique = new LinkedHashSet<Character> ();

        for(char c : s.toCharArray()) {

            System.out.println(unique.add(c));
        }
        for(char dis:unique){
            System.out.println(dis);
        }


    }
}

Suppose we have a list of String like:

List<String> strList = new ArrayList<>(5);
// insert up to five items to list.        

Then we can remove duplicate elements in multiple ways.

Prior to Java 8

List<String> deDupStringList = new ArrayList<>(new HashSet<>(strList));

Note: If we want to maintain the insertion order then we need to use LinkedHashSet in place of HashSet

Using Guava

List<String> deDupStringList2 = Lists.newArrayList(Sets.newHashSet(strList));

Using Java 8

List<String> deDupStringList3 = strList.stream().distinct().collect(Collectors.toList());

Note: In case we want to collect the result in a specific list implementation e.g. LinkedList then we can modify the above example as:

List<String> deDupStringList3 = strList.stream().distinct()
                 .collect(Collectors.toCollection(LinkedList::new));

for(int a=0;a<myArray.size();a++){
        for(int b=a+1;b<myArray.size();b++){
            if(myArray.get(a).equalsIgnoreCase(myArray.get(b))){
                myArray.remove(b); 
                dups++;
                b--;
            }
        }
}

public static void main(String[] args){
    ArrayList<Object> al = new ArrayList<Object>();
    al.add("abc");
    al.add('a');
    al.add('b');
    al.add('a');
    al.add("abc");
    al.add(10.3);
    al.add('c');
    al.add(10);
    al.add("abc");
    al.add(10);
    System.out.println("Before Duplicate Remove:"+al);
    for(int i=0;i<al.size();i++){
        for(int j=i+1;j<al.size();j++){
            if(al.get(i).equals(al.get(j))){
                al.remove(j);
                j--;
            }
        }
    }
    System.out.println("After Removing duplicate:"+al);
}

This three lines of code can remove the duplicated element from ArrayList or any collection.

List<Entity> entities = repository.findByUserId(userId);

Set<Entity> s = new LinkedHashSet<Entity>(entities);
entities.clear();
entities.addAll(s);

ArrayList<String> city=new ArrayList<String>();
city.add("rajkot");
city.add("gondal");
city.add("rajkot");
city.add("gova");
city.add("baroda");
city.add("morbi");
city.add("gova");

HashSet<String> hashSet = new HashSet<String>();
hashSet.addAll(city);
city.clear();
city.addAll(hashSet);
Toast.makeText(getActivity(),"" + city.toString(),Toast.LENGTH_SHORT).show();

When you are filling the ArrayList, use a condition for each element. For example:

    ArrayList< Integer > al = new ArrayList< Integer >(); 

    // fill 1 
    for ( int i = 0; i <= 5; i++ ) 
        if ( !al.contains( i ) ) 
            al.add( i ); 

    // fill 2 
    for (int i = 0; i <= 10; i++ ) 
        if ( !al.contains( i ) ) 
            al.add( i ); 

    for( Integer i: al )
    {
        System.out.print( i + " ");     
    }

We will get an array {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}


Java 8 streams provide a very simple way to remove duplicate elements from a list. Using the distinct method. If we have a list of cities and we want to remove duplicates from that list it can be done in a single line -

 List<String> cityList = new ArrayList<>();
 cityList.add("Delhi");
 cityList.add("Mumbai");
 cityList.add("Bangalore");
 cityList.add("Chennai");
 cityList.add("Kolkata");
 cityList.add("Mumbai");

 cityList = cityList.stream().distinct().collect(Collectors.toList());

How to remove duplicate elements from an arraylist


If you don't want duplicates, use a Set instead of a List. To convert a List to a Set you can use the following code:

// list is some List of Strings
Set<String> s = new HashSet<String>(list);

If really necessary you can use the same construction to convert a Set back into a List.


public Set<Object> findDuplicates(List<Object> list) {
        Set<Object> items = new HashSet<Object>();
        Set<Object> duplicates = new HashSet<Object>();
        for (Object item : list) {
            if (items.contains(item)) {
                duplicates.add(item);
                } else { 
                    items.add(item);
                    } 
            } 
        return duplicates;
        }

Code:

List<String> duplicatList = new ArrayList<String>();
duplicatList = Arrays.asList("AA","BB","CC","DD","DD","EE","AA","FF");
//above AA and DD are duplicate
Set<String> uniqueList = new HashSet<String>(duplicatList);
duplicatList = new ArrayList<String>(uniqueList); //let GC will doing free memory
System.out.println("Removed Duplicate : "+duplicatList);

Note: Definitely, there will be memory overhead.


    ArrayList<String> list = new ArrayList<String>();
    HashSet<String> unique = new LinkedHashSet<String>();
    HashSet<String> dup = new LinkedHashSet<String>();
    boolean b = false;
    list.add("Hello");
    list.add("Hello");
    list.add("how");
    list.add("are");
    list.add("u");
    list.add("u");

    for(Iterator iterator= list.iterator();iterator.hasNext();)
    {
        String value = (String)iterator.next();
        System.out.println(value);

        if(b==unique.add(value))
            dup.add(value);
        else
            unique.add(value);


    }
    System.out.println(unique);
    System.out.println(dup);

In Java 8:

List<String> deduped = list.stream().distinct().collect(Collectors.toList());

Please note that the hashCode-equals contract for list members should be respected for the filtering to work properly.


You can also do it this way, and preserve order:

// delete duplicates (if any) from 'myArrayList'
myArrayList = new ArrayList<String>(new LinkedHashSet<String>(myArrayList));

I am a little bit surprised that nobody didn't mention Integer constructor that takes String as a parameter.
So, here is:

String myString = "1234";
int i1 = new Integer(myString);

Java 8 - Integer(String).

Of course, the constructor will return type Integer, and unboxing operation converts value to int.


It's important to mention
This constructor calls parseInt method.

public Integer(String var1) throws NumberFormatException {
    this.value = parseInt(var1, 10);
}






java list collections arraylist duplicates