Oh, it just uses a fairly simple trial division algorithm. I at one point uses a Fermat prime test modulo 2, but that algorithm is just too slow for anything smaller than 9 digits (in TI-BASIC it is, anyways). While it can return if a number is prime many times faster than trial division, it tends to be much slower than trial division at returning whether or not a number is composite (which is most numbers). So in the end, I found that it saved time to just remove the code for the Fermat test.
So instead, since all composite numbers except 2,3,5 are of the form 30k+{7,11,13,17,19,23,29,31}, I just check to see if any of those numbers are factors of the number up until the square root. The relevant code is:
If min(remainder(C,{3,7,11 ;Check primality
Then
√(C→D
{13,17,19,23,29,31,37,41
While max(Ans)≤D and min(remainder(C,Ans
Ans+30
End
N-not(not(min(remainder(C,Ans→N
End