Like I promised I create this thread for my (de-)tokenizer. There is a download attached to this post, and a tokens xml-file is downloadable from coupler.netai.net/tokens.php. Remove the ads at the bottom of the file first, before loading it into CediT. If you don't, the program will die! (just once, not forever though )

some screens:

oops, forgot to explain that... :$
It is a pretty conventional tokenizer and detokenizer, and it reads it's tokens from an xml file, or from a database.
For this feature you need a password, but I'm not gonna type it on public pages, so it could be easily found...
Just PM me for it
I think you know what a tokenizer is, but for those who don't:
it's a program which converts typed in tokens to hex, and vice versa. This way you can write your own calc programs on your PC!
It's not ready yet, but it has the following features implemented:
- immediate conversion
- connection with a database (MySQL)
- changing typed tokens to their nicer equivalents when Shift-Space is pressed, like -> will become an arrow. Those tokens and replacements have to be declared in a file, which can be loaded on start of the program, or declared in a file called alts01.xml at the application startup path.
- single token conversion (mainly for testing, but also for showing the description of a token if available)
- saving database data to an XML file
in the future those features will be added:
- replacement chars in the database
- enable modifying descriptions and suggesting them to the database
- saving to an 8xp file

and if I've forgotten anything, just give me a shout

yes, that's right... but the extra feature this program has is MySQL database connection, and it's different in many aspects concerning the way it works and looks

no, in the database all tokens and their hex codes are stored...

You have a B:\ drive? How's that?

EDIT: I finally understood what this does Thanks to the screenshots. Is it open-source? I'd really like to see your detokenizing engine

Originally, A:\ and B:\ were for floppy drives and C to Z for the rest, since most of the computers at the time had 2 of them. Windows now lets you map drives to B:\ (and maybe even A:\ if you have no floppy). Myself, I have a B:\ drive for a MS-DOS partition.

Fun fact: I managed to map non-alphabetical drives once ($:\ or 1:\)

Quote from: juju2143 on January 13, 2011, 04:19:32 am

Originally, A:\ and B:\ were for floppy drives and C to Z for the rest, since most of the computers at the time had 2 of them. Windows now lets you map drives to B:\ (and maybe even A:\ if you have no floppy). Myself, I have a B:\ drive for a MS-DOS partition.

Fun fact: I managed to map non-alphabetical drives once ($:\ or 1:\)

Quote from: graphmastur on January 12, 2011, 06:47:19 pm

Quote from: ScoutDavid on January 12, 2011, 03:55:30 pm

You have a B:\ drive? How's that?

Technically, it does it in order, so you can have a B:\ drive. I have an A:\ drive fora  3 1/2 floppy, I don't remember what hardware piece B:\ would be.  C:\ is obviously the first partition on my hard-drive.  Also, it's \ not /.  / is for linux/unix and usually will work in windows. But \ is more technically correct. Just some fun info there. ;-)

Thank you guys for the detailed explanation.

no problem.  I'm wondering if I should hook any hardware up to that B:/ part. Maybe an SD card reader or something.

I like it, but i dont get the token change func... shift-space doesnt seem to work...

And, am i right that you can use the hex code using asmprgm??
so,
program BBBB
Asmprgm
[Hex]
and run it with asm(prgmBBBB
or am i completely missing the point?

EDIT: No, it didnt work. what is the point of the hex code, then? (its a good prog, though)