I'm guessing they use Gaussian Elimination

@Quigibo: I'll look into it. Even if the calc doesn't use it, it looks interesting.

@nemo: When I was learning how to do it over the summer for a summer packet (school started last week...), I came across that same youtube tutorial. It _is_ good. The other one's a little over my head though.

You would use either floating point numbers (recommended) and regular gaussian elimination algorithms or you could use the division-free variants if you had arbitrary precision integer operations available. I'd recommend obtaining a copy of "Algorithms for Computer Algebra" by Geddes et al. if you want an in depth explanation.