java中刪除數(shù)組中重復(fù)元素方法探討

更新時間：2013年10月02日 00:16:18 作者：

這個是一個老問題，但是發(fā)現(xiàn)大多數(shù)人說的還不夠透。小弟就在這里拋磚引玉了，歡迎拍磚

問題：比如我有一個數(shù)組（元素個數(shù)為0哈），希望添加進去元素不能重復(fù)。

　　拿到這樣一個問題，我可能會快速的寫下代碼，這里數(shù)組用ArrayList.

private static void testListSet(){
        List<String> arrays = new ArrayList<String>(){
            @Override
            public boolean add(String e) {
                for(String str:this){
                    if(str.equals(e)){
                        System.out.println("add failed !!!  duplicate element");
                        return false;
                    }else{
                        System.out.println("add successed !!!");
                    }
                }
                return super.add(e);
            }
        };

        arrays.add("a");arrays.add("b");arrays.add("c");arrays.add("b");
        for(String e:arrays)
            System.out.print(e);
    }

這里我什么都不關(guān)，只關(guān)心在數(shù)組添加元素的時候做下判斷（當(dāng)然添加數(shù)組元素只用add方法），是否已存在相同元素，如果數(shù)組中不存在這個元素，就添加到這個數(shù)組中，反之亦然。這樣寫可能簡單，但是面臨龐大數(shù)組時就顯得笨拙：有100000元素的數(shù)組天家一個元素，難道要調(diào)用100000次equal嗎？這里是個基礎(chǔ)。

問題：加入已經(jīng)有一些元素的數(shù)組了，怎么刪除這個數(shù)組里重復(fù)的元素呢？

　　大家知道java中集合總的可以分為兩大類：List與Set。List類的集合里元素要求有序但可以重復(fù)，而Set類的集合里元素要求無序但不能重復(fù)。那么這里就可以考慮利用Set這個特性把重復(fù)元素刪除不就達(dá)到目的了，畢竟用系統(tǒng)里已有的算法要優(yōu)于自己現(xiàn)寫的算法吧。

復(fù)制代碼代碼如下:

public static void removeDuplicate(List<People> list){
       HashSet<People> set = new HashSet<People>(list);
       list.clear();
       list.addAll(set);
    }　　private static People[] ObjData = new People[]{
        new People(0, "a"),new People(1, "b"),new People(0, "a"),new People(2, "a"),new People(3, "c"),
    };　

復(fù)制代碼代碼如下:

public class People{
    private int id;
    private String name;

    public People(int id,String name){
        this.id = id;
        this.name = name;
    }

    @Override
    public String toString() {
        return ("id = "+id+" , name "+name);
    }    
}

上面的代碼，用了一個自定義的People類，當(dāng)我添加相同的對象時候（指的是含有相同的數(shù)據(jù)內(nèi)容），調(diào)用removeDuplicate方法發(fā)現(xiàn)這樣并不能解決實際問題，仍然存在相同的對象。那么HashSet里是怎么判斷像個對象是否相同的呢？打開HashSet源碼可以發(fā)現(xiàn)：每次往里面添加數(shù)據(jù)的時候，就必須要調(diào)用add方法：

復(fù)制代碼代碼如下:

@Override 
     public boolean add(E object) { 
         return backingMap.put(object, this) == null; 
     }

這里的backingMap也就是HashSet維護的數(shù)據(jù)，它用了一個很巧妙的方法，把每次添加的Object當(dāng)作HashMap里面的KEY，本身HashSet對象當(dāng)作VALUE。這樣就利用了Hashmap里的KEY唯一性，自然而然的HashSet的數(shù)據(jù)不會重復(fù)。但是真正的是否有重復(fù)數(shù)據(jù)，就得看HashMap里的怎么判斷兩個KEY是否相同。

復(fù)制代碼代碼如下:

@Override public V put(K key, V value) {
        if (key == null) {
            return putValueForNullKey(value);
        }

        int hash = secondaryHash(key.hashCode());
        HashMapEntry<K, V>[] tab = table;
        int index = hash & (tab.length - 1);
        for (HashMapEntry<K, V> e = tab[index]; e != null; e = e.next) {
            if (e.hash == hash && key.equals(e.key)) {
                preModify(e);
                V oldValue = e.value;
                e.value = value;
                return oldValue;
            }
        }

        // No entry for (non-null) key is present; create one
        modCount++;
        if (size++ > threshold) {
            tab = doubleCapacity();
            index = hash & (tab.length - 1);
        }
        addNewEntry(key, value, hash, index);
        return null;
    }

總的來說，這里實現(xiàn)的思路是：遍歷hashmap里的元素，如果元素的hashcode相等（事實上還要對hashcode做一次處理），然后去判斷KEY的eqaul方法。如果這兩個條件滿足，那么就是不同元素。那這里如果數(shù)組里的元素類型是自定義的話，要利用Set的機制，那就得自己實現(xiàn)equal與hashmap（這里hashmap算法就不詳細(xì)介紹了，我也就理解一點）方法了：

復(fù)制代碼代碼如下:

public class People{
    private int id; //
    private String name;

    public People(int id,String name){
        this.id = id;
        this.name = name;
    }

    @Override
    public String toString() {
        return ("id = "+id+" , name "+name);
    }

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    @Override
    public boolean equals(Object obj) {
        if(!(obj instanceof People))
            return false;
        People o = (People)obj;
        if(id == o.getId()&&name.equals(o.getName()))
            return true;
        else
            return false;
    }

    @Override
    public int hashCode() {
        // TODO Auto-generated method stub
        return id;
        //return super.hashCode();
    }
}

這里在調(diào)用removeDuplicate(list)方法就不會出現(xiàn)兩個相同的people了。

好吧，這里就測試它們的性能吧：

復(fù)制代碼代碼如下:

public class RemoveDeplicate {

    public static void main(String[] args) {
        // TODO Auto-generated method stub
        //testListSet();
        //removeDuplicateWithOrder(Arrays.asList(data));
        //ArrayList<People> list = new ArrayList<People>(Arrays.asList(ObjData));

        //removeDuplicate(list);

        People[] data = createObjectArray(10000);
        ArrayList<People> list = new ArrayList<People>(Arrays.asList(data));

        long startTime1 = System.currentTimeMillis();
        System.out.println("set start time --> "+startTime1);
        removeDuplicate(list);
        long endTime1 = System.currentTimeMillis();
        System.out.println("set end time --> "+endTime1);
        System.out.println("set total time --> "+(endTime1-startTime1));
        System.out.println("count : " + People.count);
        People.count = 0;

        long startTime = System.currentTimeMillis();
        System.out.println("Efficient start time --> "+startTime);
        EfficientRemoveDup(data);
        long endTime = System.currentTimeMillis();
        System.out.println("Efficient end time --> "+endTime);
        System.out.println("Efficient total time --> "+(endTime-startTime));
        System.out.println("count : " + People.count);

    }
    public static void removeDuplicate(List<People> list)
    {
     HashSet<People> set = new HashSet<People>(list);
     list.clear();
     list.addAll(set);
    }

    public static void removeDuplicateWithOrder(List<String> arlList)
    {
       Set<String> set = new HashSet<String>();
       List<String> newList = new ArrayList<String>();
       for (Iterator<String> iter = arlList.iterator(); iter.hasNext();) {
          String element = iter.next();
          if (set.add( element))
             newList.add( element);
       }
       arlList.clear();
       arlList.addAll(newList);
    }


    @SuppressWarnings("serial")
    private static void testListSet(){
        List<String> arrays = new ArrayList<String>(){
            @Override
            public boolean add(String e) {
                for(String str:this){
                    if(str.equals(e)){
                        System.out.println("add failed !!! duplicate element");
                        return false;
                    }else{
                        System.out.println("add successed !!!");
                    }
                }
                return super.add(e);
            }
        };

        arrays.add("a");arrays.add("b");arrays.add("c");arrays.add("b");
        for(String e:arrays)
            System.out.print(e);
    }

    private static void EfficientRemoveDup(People[] peoples){
        //Object[] originalArray; // again, pretend this contains our original data
        int count =0;
        // new temporary array to hold non-duplicate data
        People[] newArray = new People[peoples.length];
        // current index in the new array (also the number of non-dup elements)
        int currentIndex = 0;

        // loop through the original array...
        for (int i = 0; i < peoples.length; ++i) {
            // contains => true iff newArray contains originalArray[i]
            boolean contains = false;

            // search through newArray to see if it contains an element equal
            // to the element in originalArray[i]
            for(int j = 0; j <= currentIndex; ++j) {
                // if the same element is found, don't add it to the new array
                count++;
                if(peoples[i].equals(newArray[j])) {

                    contains = true;
                    break;
                }
            }

            // if we didn't find a duplicate, add the new element to the new array
            if(!contains) {
                // note: you may want to use a copy constructor, or a .clone()
                // here if the situation warrants more than a shallow copy
                newArray[currentIndex] = peoples[i];
                ++currentIndex;
            }
        }

        System.out.println("efficient medthod inner count : "+ count);

    }

    private static People[] createObjectArray(int length){
        int num = length;
        People[] data = new People[num];
        Random random = new Random();
        for(int i = 0;i<num;i++){
            int id = random.nextInt(10000);
            System.out.print(id + " ");
            data[i]=new People(id, "i am a man");
        }
        return data;
    }
｝

測試結(jié)果：

復(fù)制代碼代碼如下:

set end time -->  1326443326724
set total time -->  26
count : 3653
Efficient start time --> 1326443326729
efficient medthod inner  count : 28463252
Efficient end time -->  1326443327107
Efficient total time -->  378
count : 28463252

您可能感興趣的文章: