I am not sure whether to go with writing out programs by letters (which would be easier to display and make the editor smaller) or by tokens (which would make the editor much bigger but the saved files smaller).
It looks like reading and writing the programs will be the hardest part (so I can compile code written in the editor or edit it in the basic editor also).

If you want your files to be usable by anything else on the calculator, tokens are the required format.

It would be really cool if it could display code like "{L1 +8}" as "boss1hp" if the user declares the name. That way variables can have really long custom names and there (theoretically) is not a name limit.

For editing Axe programs, which it looks like is what you're alluding to, I don't think such a feature would need to be supplied by the editor. The Axe language already lets users declare named constants/variables. In your example, the user would declare L₁+8→°boss1hp and could then refer to it as the one-byte variable boss1hp^r throughout the rest of the code.

Is there any way to convert between the program editor tokens and the display tokens without way too many if statements?

You mean displaying the string represented by a token code? Using the ▶Tok conversion with a display command does this.

For the tokens I have decided to have the program written out in letters so it is easier to display and have it convert each command to token form when the user quits.

I wouldn't suggest this method, but it's ultimately your decision.

As I understand Axe has a limit on the number of named variables (around 150?) and this would bypass that since when it is written to the program it would be written as {L1 +8}

It depends on how long your names are, but it's closer to 1500 than 150. I can't imagine a program using that many named constants/variables, or if it did, it probably wouldn't matter because it would be too large to fit in RAM and actually be editable.

I must be missing something on the >Tok command. When I ask it to display {A} (or {A}r) >Tok, it displays a completely different token.

The OS calls for displaying tokens require a pointer to the token to be displayed. And Axe uses these calls to implement ▶Tok, so the same applies. I presume this decision was made because tokens are almost always displayed as part of a program or string in RAM, and putting the read in the call saves callers from having to do so. Since you're also editing a program in RAM, this decision should actually make your code simpler. Just drop the curly brackets for the memory read.

How would I write tokens back to the program? "token" -> A doesn't work

→{A} for one-byte tokens and →{A}ʳ for two-byte tokens.

EDIT: And if you're using tokens, you probably shouldn't have to have their string data anywhere in your program. You should just be using the token codes, which (as previously suggested) are one- or two-byte values. In case you weren't aware, you can get the value of most tokens in Axe with the ^Ttoken syntax.

Is there a way to tell if a token is one or two byte when you display it?

All two-byte tokens have one of the following hexadecimal values as their first (low) byte:
5C 5D 5E 60 61 62 63 7E AA BB EF

For the purpose of displaying a token in Axe, you shouldn't need to do anything different for one- or two-byte tokens. I guess that could be another reason why the input for displaying a token is a pointer rather than an actual token value. The display function can tell if it's a one- or two-byte token after reading the first byte and then only read the second byte if necessasry.

Awesome! The only other thing I need to know (for now) is what is the best way to find a program's size?
Is it with the length( command or with {ptr - 2}?

{P-2}^r is the correct way to read the size of a program, appvar, picture, string, or equation variable pointed to by P. Note the ^r modifier, which was missing from your otherwise correct second suggestion.

edit: is there a way to get (not set) the cursor's position?

Since you're working on a small font program editor, I'm going to assume you mean the graph screen cursor, which is actually called the "pen," rather than the home screen cursor.

The pen cloumn is stored at penCol = 86D7h and the pen row is stored at penRow = 86D8h. Each are one-byte values, so you could read them with {ᴇ86D7} and {ᴇ86D8}. But if you plan on referencing them a lot, you should probably turn these into named variables with statements like ᴇ86D7→°PX and ᴇ86D7→°PY and then reference them with PXʳ and PYʳ.

So far I have everything done but the ability to add to the program (you can delete).
Is there a way to display the custom tokens? I can't seem to find a good way to choose tokens without creating a program with them or by chaining if statements.

When I display tokens to the screen it shows the basic tokens. Is there a way to allow the program to check which tokens should be changed?
I have decided to just create a program to display the tokens.