There are two main kinds of libraries: static libraries (in *nix, usually ".a" files; in Windows, often ".lib" files), used at compile time, and dynamic libraries (".so" in *nix, ".dll" in Windows), used at runtime.
Sometimes, for using a dynamic library, a small static library containing imports for the dynamic library is linked statically with a program.

There's no support for dynamic libraries in Ndless (yet), so you would have to make a static build of GMP, and then link another program against that static library: the code of the GMP functions you use, and their transitive dependencies, would be embedded into your program (and for the common subset, copied across multiple different programs which use different sets of functions from said static library).

* in your C source code itself, indeed, you would just say "#include <gmp.h>";
* you'd also have to add a "-lgmp" flag to the linker flags in your Makefile (or whatever you use for triggering the build);
* you should add a -L flag too, to tell the linker where to find the "libgmp.a" library you've compiled for the current ARM platform.


I've just made and tested a quick script for compiling the GMP static library I mentioned above:
#!/bin/sh
# This creates a "gmp" subdirectory whereever you launch the script, and compiles and installs in subdirectories thereof.
mkdir gmp
cd gmp
WORKING_DIR=`pwd`
wget ftp://ftp.gmplib.org/pub/gmp-5.0.4/gmp-5.0.4.tar.bz2 || exit 1
tar xvjf gmp-5.0.4.tar.bz2 || exit 1
mv gmp-5.0.4 src
mkdir build
mkdir prefix
cd build
../src/configure --host=arm-none-eabi --prefix="$WORKING_DIR/prefix" --enable-shared=no --enable-assert || exit 1
make && make install
Upon successful execution of the script, in the "prefix" subdirectory, you have include/gmp.h and lib/libgmp.a, so you'd want a -L<whereever>/prefix/lib linker flag

NOTE: when cross-compiling, it's hard to run "make check" and verify the compilation. This script doesn't tackle the problem.

[EDIT: clarified a sentence.]

-L and -l should be in LDFLAGS.
CFLAGS should contain -I /path/to/the/include/directory, that contains gmp.h.

Upgrade your Ndless version to the latest version


Off-topic: I know that you're not making an Lua project, but this topic somehow made me wonder whether there are Lua bindings for GMP... and I found one:  http://www.dkrogers.com/doug/lua/ . But it was made years ago, so if it is to be ported to the Nspire, it would require quite a bit of work, potentially on the Lua binding side and definitely on the GMP side.
Combining that, with the support for native Lua extensions of the latest Ndless SVN versions (Omnimaga news / TI-Planet news), could showcase an interesting cooperation between Lua and native code. This is exactly the sort of thing that would benefit both users and TI, but TI will certainly kill it in ~10 days alongside the current iteration of Ndless in ~10 days, notwithstanding our freedom to tinker.