Weird shit happens with rotation. 

Also, CalcCapture is acting weird. At the end, you can see that it inserted a picture of itself into a few frames.

It always did that for me. You see CalcCapture at the end in almost all screenshots I made with it (see this screenshot from Early 2004 for example)



It also has poor framerate (although it still does the job I guess). Anyway nice to see more work being done on Mode 7 lately for the Nspire

I suspect the problem is inaccurate trig or more likely fixed point multiplication routines.

I was originally calculating a lookup table for every pixel and then translating the values for each pixel based on the camera position and rotation. Now, I am only doing a lookup table for every row of the screen, and interpolating values within the row, which is actually how the GBA and SNES do it. It also improved the framerate as you can see in the FPS counter.

I still have to add back in rotation. Fortunately, it shouldn't be as much of a performance hit as you saw last time, because all the translation will be done per row, instead of per pixel.

Well, I've got rotation by a fixed angle working, and it still runs >60FPS at full resolution. (Not that it matters given how slow the response time of the LCD on the actual thing is.) Now to write *accurate* trig functions so I can change the angle dynamically. Also - screen tearing is from the emulator, not the program.

If you're measuring 64FPS on the emulator you can be sure it's lower oncalc. The emulator doesn't have very accurate timing, because it executes every instruction in 1 cycle and doesn't emulate memory access delays (not that I'd expect it to).

Since I'm getting quite interested in Mode7, I was planning to see how it worked.
I though of a way, that would look a bit like like the screenshots you give: Each "pixel" is shows as a trapezoid, from small(up) to big(down).
However when I look at screenshots of F-zero(68k) for example: http://www.ticalc.org/archives/files/fileinfo/381/38175.html this doesn't look the case.
Is that then a different kind of mode7?

The problem is something with the uart_printf() and not your sin() and cos(). I don't know much about printf specifiers, but %4l doesn't seem to be valid and results in a warning (if you'd use printf instead of uart_printf, obviously).
Why do you use uart_printf and not the normal printf?

There's also a bug in this code, (a & 0x3000) >> 16 always gives zero. I think you meant to use 0x30000?