Author Topic: Two-Byte Tokens ? (Read 3308 times)

Hayleia · « **on:** June 03, 2013, 02:56:05 pm »

So I am currently experimenting things a bit with tokens, and I hit the problem most of us surely met: two-byte token. See the screenshot, everything is displaying well... except that there are "?" behind all the "L1" and a "}" behind the "AxesOff". So I guess that "L1" and "AxesOff" actually are two-byte tokens and since my code currently displays a token at each byte, I must be displaying L1, then an unknow one-byte token that is actually part of L1 and was already displayed.

So the question is: How to recognize a two-byte token so I don't display something that shouldn't ?

edit Also, I posted this in "Axe Language" since I did my program in Axe, but this question could also apply to ASM since it only requires ASM knowledge, so if you think this is not the right place to ask this, feel free to move it.

jacobly · « **Reply #1 on:** June 03, 2013, 05:43:21 pm »

Iff a byte is equal to ᴇ5C, ᴇ5D, ᴇ5E, ᴇ60, ᴇ61, ᴇ62, ᴇ63, ᴇ7E, ᴇBB, ᴇAA, or ᴇEF, then it is the first byte of a two byte token, and it can not be the second byte of a two byte token.

shmibs · « **Reply #2 on:** June 03, 2013, 06:45:58 pm »

^ yup, it's that simple.
take a look at tibasicdev if you want to see which is which in more detail.

Hayleia · « **Reply #3 on:** June 04, 2013, 01:28:39 am »

Ok, thanks both of you. I knew there was a simple answer but I didn't know that answer.

Author Topic: Two-Byte Tokens ? (Read 3308 times)

Hayleia

Two-Byte Tokens ?

jacobly

Re: Two-Byte Tokens ?

shmibs

Re: Two-Byte Tokens ?

Hayleia

Re: Two-Byte Tokens ?