Benchmarking all TI-84 Plus CE OS Version [5.0.0-5.6.0] - Cemetech | Forum | Technology & Calculator Open Topic [Topic]

TheLastMillennial
2
Official Cemetech Cake Manager (Posts: 2030)

Benchmarking all TI-84 Plus CE OS Version [5.0.0-5.6.0]
25 Sep 2017 07:49:06 pm
Last edited by TheLastMillennial on 03 Nov 2020 04:45:33 pm; edited 5 times in total

A few days ago I watched a video about benchmarking every iOS. This gave me an idea, benchmark all CE OSes! I came up with a few small programs to benchmark, I used this program to time everything.

Here are my results. Green=fastest time, red= slowest time.

ABENCH (runs every benchmark and stores the results into a list)

Code:




SetUpEditor L₆


9→dim(L₆


prgmT01RAND


Ans→L₆(1


prgmT02GRAPH


Ans→L₆(2


prgmT03GRAPH


Ans→L₆(3


prgmT04SUM


Ans→L₆(4


prgmT05LIST


Ans→L₆(5


prgmT06LIST


Ans→L₆(6


prgmT07LIST


Ans→L₆(7


L₆

T01RAND (random integer test)

Code:

0→B


For(A,1,5


Asm(prgmTIMER


rand(100


Asm(prgmTIMER


Ans+B→B


End


B/5

T02GRAPH (Horizontal line test)

Code:

0→W


For(A,1,5


Asm(prgmTIMER


ClrDraw


For(B,Ymax,Ymin,Y


Horizontal B


End


Asm(prgmTIMER


Ans+W→W


End


W/5

T03GRAPH (vertical line test)

Code:

0→W


For(A,1,5


Asm(prgmTIMER


ClrDraw


For(B,Xmax,Xmin,X


Vertical B


End


Asm(prgmTIMER


Ans+W→W


End


W/5

T04SUM (summation test)

Code:

0→W


For(A,1,5


Asm(prgmTIMER


Σ(X²,X,0,1000


Asm(prgmTIMER


Ans+W→W


End


W/5

T05LIST (creating list test)

Code:

0→W


For(A,1,5


DelVar L₁


Asm(prgmTIMER


999→dim(L₁


Asm(prgmTIMER


Ans+W→W


End


W/5

T06LIST (sorting list of only 0s)

Code:

0→W


For(A,1,5


DelVar L₁


100→dim(L₁


Asm(prgmTIMER


SortA(L₁


Asm(prgmTIMER


Ans+W→W


End


W/5

T07LIST (sorting list of only 7s)

Code:

0→W


For(A,1,5


DelVar L₁


100→dim(L₁


Fill(7,L₁


Asm(prgmTIMER


SortA(L₁


Asm(prgmTIMER


Ans+W→W


End


W/5

RAYTRSPR ( this is a 3D modeling test using raycasting) (I didn't make this program, this was made by TheScienceElf, you can view his video here.)

Code:

Asm(prgmTIMER


ClrDraw:1→G


{12,24,23,22,21,20}→L₁





For(Y,0,162,2G)


(81-Y)/164→B


For(X,0,262,2G)


(X-131)/164→A





A²+B²+1→E:2B-12→F


34.75→H:F²-4EH→D





1/B→T





AT→U:T→V


remainder(abs(int(U)+int(V)),2)→C





12→L:12→M


12-(T>0)(C+1)→C





If D≥0:Then


(F-√(D))/(2E)→T


AT→I:BT-1→J:6-T→K


J/1.5→M





If M>0:Then


int(10M)→M


L₁(1+int(M/2))→L


L₁(1+int(M/2+.5))→M


Else:12→M


End





{I,J,K}/1.5→L₃


{A,B,1}→L₂


2sum(L₃L₂)→S





SL₃-L₂→L₄


L₄(1)→P:L₄(2)→Q:L₄(3)→R





(J+2)/Q→T





If T>0:Then


PT+I→U:RT+6-K→V


11-remainder(abs(int(U)+int(V)),2)→C


Else:12→C:End





Else:If T>0:Then


1→E:4→F


U²+V²-12V+37.75→H


F²-4EH→D





12(D≥0)+C(D<0)→L


L→M


End:End





Pxl-On(Y,X,C)


Pxl-On(Y+1,X,L)


Pxl-On(Y,X+1,M)


Pxl-On(Y+1,X+1,C)


End:End


Asm(prgmTIMER


Ans

If you spot any errors in my testing, or want me to benchmark some other aspect of the CE, please post below!

iPhoenix
3
friendly neighborhood site admin (Posts: 1912)

25 Sep 2017 08:02:47 pm

I made a raycaster once. You could use mine (look up ti-basic raycaster in the Cemetech search thingie)

oldmud0
Power User (Posts: 416)

25 Sep 2017 08:11:21 pm

Thus begin the conspiracies…

john35588
Member (Posts: 170)

26 Sep 2017 06:17:07 pm

It is all barely a difference to me, I don't notice.

mr womp womp
Official Cemetech Cat Manager (Posts: 1796)

26 Sep 2017 06:51:08 pm

what on earth happened to the list sorting on 5.0.0?
first, did you re-test that one to confirm that the value was correct?
If so, why is there such a discrepancy? As far as I know, there was no mention of the sorting algorithm being improved. Did you try SortD() to see if similar results appear? What about a list that has different values rather than 0s?
I think the fact that the sortA() command seems 7x faster is worth looking into Confused

TheLastMillennial
2
Official Cemetech Cake Manager (Posts: 2030)

26 Sep 2017 06:55:32 pm

mr womp womp wrote:

what on earth happened to the list sorting on 5.0.0?
first, did you re-test that one to confirm that the value was correct?
If so, why is there such a discrepancy? As far as I know, there was no mention of the sorting algorithm being improved. Did you try SortD() to see if similar results appear? What about a list that has different values rather than 0s?
I think the fact that the sortA() command seems 7x faster is worth looking into Confused

This was a startling result, but I ran the thing 5 times and got the same result.
I have no idea why it was so slow.
I did not try SortD( I'll try that later since my calculator recently recovered from 3 OS fails.
I'll also try values other than 0.

Good points! Smile

TheLastMillennial
2
Official Cemetech Cake Manager (Posts: 2030)

25 Aug 2019 08:15:15 pm
Last edited by TheLastMillennial on 28 Aug 2019 12:27:14 pm; edited 1 time in total

Hey, coming at you with a 2 year necro-follow-up!

Since there's been several OS's released since my last benchmark, I've decided to re-do all of them. Hopefully the table is a little bit easier to understand than last time since Excel color coded all of them. I decided to check some of womp's questions about sortA( handling non-zero numbers, that became T07LIST. Unfortunately I forgot to make a test for sortD( Sad

Anyways, enjoy the graph: green is the best time and red is the worst time!

Each test was performed from the RAM, on the same calculator, in the same order, and after a full mem clear. The calculator was plugged in and fully charged the entire time. Each test, with the exception of prgmRAYTRSPR, was completed 5 times then the times were averaged into the table above. I only ran prgmRAYTRSPR once because it takes over 30 minutes per run and ain't nobody got time to do that 5 times over! I have yet to retest OS 5.1.0, 5.1.5, and 5.2.1 for new prgmRAYTRSPR times.

Overall it seems if you want the best BASIC performance, then OS 5.0.0 or OS 5.3.1 is the way to go. However, if you just care about the speed of specific functions, then you can't go wrong with OS 5.1.x or 5.2.1.

I'd like to dig into the numbers a bit more though.

something

So there you go, personally I think OS 5.3.0 has the best performance to match its features (Asm84CEprgm token, yay!). If you'd like to try out the programs yourself, I'll be updating the first post with all the programs I ran.

mr womp womp
Official Cemetech Cat Manager (Posts: 1796)

25 Aug 2019 09:38:55 pm

The reason I asked about sorting numbers that weren't all 0s is because some algorithms will take more or less time depending on how sorted the initial list is. I know the calculator used a version of selection sort, but that might have changed. Selection sort has quadratic time complexity (O(n²)), while Quicksort takes at best quasilinear time (O(n log(n))) and at worst quadratic time (ie. the pivot was chosen to be a unique minimum or maximum)

I would assume they would stay away from some fancy stuff like Timsort, but maybe because lists will often contain few elements, they might have gone with insertion or something else if memory usage is a concern, since insertion sort has the same space complexity as selection sort. For best-case scenarios, selection sort is really one of the worst, so it would make sense that if they changed the algorithm, we would expect a drastic improvement for best-case scenarios like the one you tested.

It wasn't so much that the test case was all 0s, but rather that the list was already sorted, which isn't a good way to measure a sorting algorithm. It is hard to figure out what would be a good test case though, since different algorithms and implementations benefit from different inputs. I would suggest trying a sorted list (like 0-99) and a random list of the same length, but using the same random list for all tests. Generally, a sorted list (sorted in the right way of course) will be the best case for most common sorting algorithms, so its a pretty good metric.

TheLastMillennial
2
Official Cemetech Cake Manager (Posts: 2030)

New results 2020!
03 Nov 2020 02:02:50 am

OK, a heck of a lot has changed since last I did this test, so I decided to go back and do all these tests over again on my Pre-A TI-84 Plus CE. I made a few changes so the graph won't line up completely with last test's results.

Specifically I changed:

I added a few more OSes to test. 5.2.0, 5.2.2, 5.4.1, 5.5.5, and 5.6.0
T06LIST to SortA( a randomly generated list with the seed 1->rand.
T07LIST to SortD( a randomly generated list with the seed 1->rand.
I added T08ASM which tests how long it took for the timer program to launch. This measures the speed overhead it took for the program to start and return.
I removed the ray trace benchmark for now because it just takes too long to test.
I added an extra row that shows how much faster a certain OS was over the baseline (the slowest OS).

You can download the whole benchmark group here.

Enough talk, let's see the numbers!
Green is the best time and Red is the worst time

Each test was performed from the RAM, on the same calculator, in the same order, and after a full mem clear. The calculator was plugged in and fully charged the entire time. Each test was completed 5 times, then the times were averaged into the table above. arTIfiCE and AsmHOOK were only installed on OS versions that required them; all jail breaking utilities were stored in the Archive.

OK, we see a lot of similarities to last years results. OS 5.0.0 continues to be the slowest at sorting but it's generally pretty darn fast at everything else.

As you get to the newer OSes which ~~added~~ removed features, it's clear that the calculator is getting slower and slower at doing every function. Only by about 1% from OS 5.3.0, but if we see another drop in performance like OS 5.3.0 brought for the graphing tests, there could be significant speed issues for those with a pre-M TI-84 Plus CE.

One result I was surprised to see was that OS 5.5+ was negligibly slower at launching ASM programs than OSes that supported ASM. I figured TI would have done more to block ASM programs from launching.

So those are the results for this year! I'm probably going to stop re-testing every single OS every time a new OS gets released because this took 4 hours to organize, test, collect data, and post. I'll just be testing the newest OS and tacking it onto the end of my Excel sheet. I hope to improve my Excel sheet next time because I know my current method isn't super great at providing an accurate depiction of the OSes speed, it's just a rough measurement for now. Let me know what you thought of the results! Anything you guys want me to test next time? Are there any results that surprised you?

TheLastMillennial
2
Official Cemetech Cake Manager (Posts: 2030)

05 Nov 2020 05:33:49 pm

I received some criticism on my last table but thanks to the help of Kerm and LogicalJoe, I've created a new table that's actually useful and doesn't exaggerate the color of a negligible difference! Not only that, but I'm sharing my results so you can mess with the data however you'd like! Go take a look at it here: https://1drv.ms/x/s!Al_ZukYl7XyyyzQRzhPZcp0ZZCD4?e=6fXiGA!

This table compares the speed of each OS to OS 5.0.0 which I used as the baseline:

By the request of Adriweb, I also made a table that compares the speed of a new OS over the previous OS:

My previous conclusion needs some adjusting. Although new OS versions are still generally getting slower, most of the time the difference per generation is negligible and you'll never notice it. The only exception to this is OS 5.1.0 significantly speeding up sorting lists by about 4 times! Along with OS 5.3.0 slowing down graphing by almost 14%.

So that's the new table format! Thanks again to those who helped me figure out Excel and requested data!