« previous next »
Pages: [1]   Go Down

Author Topic: Tokenizer DE v1.0 Released  (Read 9102 times)

0 Members and 3 Guests are viewing this topic.

Offline KermMartian

  • Editor
  • LV7 Elite (Next: 700)
  • *******
  • Posts: 500
  • Rating: +233/-20
    • View Profile
    • Cemetech
Tokenizer DE v1.0 Released
« on: August 06, 2010, 02:16:54 am »
In answer to all the users that have requested a custom oncalc TI-BASIC editor, Tokenizer DE takes a first step in that direction. Just as SourceCoder (http://sc.cemetech.net) can convert text files into .8xp programs online, Tokenizer can take text documents typed in Document DE 6 or Document DE 7 and convert them into fully-tokenized BASIC programs editable with the TI-OS's TI-BASIC editor. Weighing in at slightly over 1KB, this small but efficient program takes advantage of the power of the DCS GUI system, and requires Doors CS 6.8 beta or higher.  It uses a naive reverse substring-match brute force algorithm that takes very little memory (roughly 300 bytes for the core tokenization algorithm) at the expense of going at a moderate rather than blazing speed.  Future improvements may integrate this with a Document DE-style editor; feel free to give it a try and post any comments or suggestions in the associated topic.  This release also includes the source code to Tokenizer DE for those wishing to learn more about using the Doors CS ASM libraries.

Download
Tokenizer DE v1.0

Logged


Offline bwang

  • LV7 Elite (Next: 700)
  • *******
  • Posts: 634
  • Rating: +30/-11
    • View Profile
Re: Tokenizer DE v1.0 Released
« Reply #1 on: August 06, 2010, 05:20:40 am »
Excellent! This is certainly useful for people who want to edit more than 7 lines of text at a time :)
Logged

Offline qazz42

  • LV9 Veteran (Next: 1337)
  • *********
  • Posts: 1130
  • Rating: +30/-12
  • hiiiiiiiii
    • View Profile
Re: Tokenizer DE v1.0 Released
« Reply #2 on: August 06, 2010, 09:04:10 am »
Hmmm, very interesting, I see I will have 2 ways to write programs now :D
Logged

Offline meishe91

  • Super Ninja
  • LV11 Super Veteran (Next: 3000)
  • ***********
  • Posts: 2946
  • Rating: +115/-11
    • View Profile
    • DeviantArt
Re: Tokenizer DE v1.0 Released
« Reply #3 on: August 06, 2010, 09:16:50 am »
This is very cool! I don't know how much I'd use it but I'm sure some will find it very helpful. Thanks, Kerm :)
Logged
Spoiler For Spoiler:



For the 51st time, that is not my card! (Magic Joke)

Offline ztrumpet

  • The Rarely Active One
  • CoT Emeritus
  • LV13 Extreme Addict (Next: 9001)
  • *
  • Posts: 5712
  • Rating: +364/-4
  • If you see this, send me a PM. Just for fun.
    • View Profile
Re: Tokenizer DE v1.0 Released
« Reply #4 on: August 06, 2010, 02:23:19 pm »
This is indeed cool.  Nice work Kerm! ;D
Logged

Offline Silver Shadow

  • Beta Tester
  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 817
  • Rating: +27/-7
    • View Profile
Re: Tokenizer DE v1.0 Released
« Reply #5 on: August 06, 2010, 03:09:51 pm »
Very nice! :)
Logged
Former Coder of Tomorrow

Offline qazz42

  • LV9 Veteran (Next: 1337)
  • *********
  • Posts: 1130
  • Rating: +30/-12
  • hiiiiiiiii
    • View Profile
Re: Tokenizer DE v1.0 Released
« Reply #6 on: August 06, 2010, 03:56:25 pm »
People with nspires, it seems that nspire 2.0 will crash in 84 keypad mode when using tokenizer
Logged

Offline TIfanx1999

  • ಠ_ಠ ( ͡° ͜ʖ ͡°)
  • CoT Emeritus
  • LV13 Extreme Addict (Next: 9001)
  • *
  • Posts: 6173
  • Rating: +191/-9
    • View Profile
Re: Tokenizer DE v1.0 Released
« Reply #7 on: August 06, 2010, 05:00:47 pm »
Wow, that's pretty awesome. I can't believe how much functionality Doors has. Great work!
Logged

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55943
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: Tokenizer DE v1.0 Released
« Reply #8 on: August 06, 2010, 07:44:41 pm »
Wow pretty nice indeed. Nice tool again Kerm. Pretty small too :)
Logged

Offline qazz42

  • LV9 Veteran (Next: 1337)
  • *********
  • Posts: 1130
  • Rating: +30/-12
  • hiiiiiiiii
    • View Profile
Re: Tokenizer DE v1.0 Released
« Reply #9 on: August 06, 2010, 08:03:03 pm »
Ohh, check that, the latest build has nspire compat, it is not released yet..
Logged

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55943
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: Tokenizer DE v1.0 Released
« Reply #10 on: August 06, 2010, 08:04:17 pm »
You have access to a secret Cemetech download folder? Tell us tell us! j/k ;D

Or do you mean he posted about the compatibility on his forums? I should check more often x.x
« Last Edit: August 06, 2010, 08:04:30 pm by DJ Omnimaga »
Logged

Offline qazz42

  • LV9 Veteran (Next: 1337)
  • *********
  • Posts: 1130
  • Rating: +30/-12
  • hiiiiiiiii
    • View Profile
Re: Tokenizer DE v1.0 Released
« Reply #11 on: August 06, 2010, 10:34:06 pm »
No, I mean he sent me an email with the latest build for testing :)
« Last Edit: August 06, 2010, 10:34:30 pm by qazz42 »
Logged

Offline KermMartian

  • Editor
  • LV7 Elite (Next: 700)
  • *******
  • Posts: 500
  • Rating: +233/-20
    • View Profile
    • Cemetech
Re: Tokenizer DE v1.0 Released
« Reply #12 on: August 06, 2010, 10:37:47 pm »
Quote from: qazz42 on August 06, 2010, 10:34:06 pm
No, I mean he sent me an email with the latest build for testing :)
Both you and DJ are correct.  I posted about fixing the Nspire compatibility, and will shortly release the version I emailed Qazz.
Logged


Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55943
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: Tokenizer DE v1.0 Released
« Reply #13 on: August 06, 2010, 11:07:01 pm »
Cool to hear :)

The TI-Nspire is usually the first calc I have that gets used when it comes to testing/using stuff on-calc, because it's my only z80-compatible calc that I can connect through USB (I do not have a Silverlink and my computer got no serial port)
Logged
Pages: [1]   Go Up
« previous next »