How do you remove duplicate values from a list?

kpa4941

Let's say that I have a list that contains (1,2,2,3,4).

How would I get rid of the duplicate 2 to make the list look like (1,2,3,4)?

Keep in mind that the list could possibly contain 20 or so duplicates with different numbers.

Thanks.
_________________
--kpa4941

shkaboinka · Posted: 18 Jan 2012 12:18:25 pm Post subject:

If it's okay to sort the list, you can do this (and I will pretend that the list is in L1):

1: use SortA(L1) or SortD(L1) to sort the list
2: Clear another list which you will copy the values into
3: Loop through the list, and only when an item is different than the previous item, add it to the new list. This works because all the duplicates will be next to eachother (since the list is sorted).

You can do this by storing the last item in a variable which you can use to compare the next item to. Here is the code, if you like (ignore if you want to figure it out by yourself):

If 2>dim(L1
Return
SortA(L1
{L1(1->L2
Not(L1(1->A
For(X,2,dim(L1
If A#L1(X
augment(L2,L1(X->L2
End
L2->L1

Xeda112358 · Power User Joined: 09 Nov 2010 Posts: 354

Hmm, I am pretty sure I saw a neat idea on TIBD that could do this, I will see if I can find it ... (I doubt I will)

EDIT: This is from Timothy Foster on TIBD:

Weregoose · Cemetech Expert Joined: 23 Oct 2009 Posts: 464

...and then there's this:

Xeda112358 · Power User Joined: 09 Nov 2010 Posts: 354

Haha, awesome! I like that code, though I also like the use of xor in the other code, too XD

shkaboinka · Posted: 19 Jan 2012 10:05:25 pm Post subject:

Weregoose's code is excellent because it does not affect another list variable at all! Smile

(I thought that Ans would be destroyed by the "For", but if that works, that's awesome!) Anyway, if L1 is not sorted ahead of time, then you have to add "SortD(L1" to the start of that for it to work

Xeda112358 · Power User Joined: 09 Nov 2010 Posts: 354

Yeah, For( does not modify Ans (which is very useful for algorithms).

Weregoose · Cemetech Expert Joined: 23 Oct 2009 Posts: 464

There's no need to sort the list beforehand.

L1={5 1 7 6 2 8} returns L1={5 1 7 6 2 8}.
L1={5 3 3 4 4 3} returns L1={5 3 4}.

Try it out.
_________________
Common Errors in English · How To Ask Questions The Smart Way

shkaboinka · Posted: 21 Jan 2012 12:12:15 am Post subject:

How does that work out!? Each time it's comparing just to the min value in the list, and what guarantees that the current value ("Xth") in the loop is unique if it is not equal to the min value so far? I would suppose that would only be true if the list was sorted in decending order...?

Weregoose · Cemetech Expert Joined: 23 Oct 2009 Posts: 464

shkaboinka · Posted: 21 Jan 2012 04:48:55 pm Post subject:

...oh... I was reading it as "min(Ans)≠L1(X" (because for some reason, my "leave off the parenthesis" mechanism applied the invisible ")" to before the "≠") ... That is an AWESOME technique! I've not used whole-list operations much, so I should give them a closer look (I knew they existed, but never thought how to apply them).