Author Topic: Zigel - Variable Storage (Read 5245 times)

Iambian · « **on:** April 12, 2007, 04:58:00 am »

I had started this topic up on UnitedTI, but I wanted to see what the people here think of it. Between first mentioning it and posting it here, I came up with a name for the project, completely in the spirit of how I named Celtic.

This project is a filesystem program for the TI-83 Plus Silver Edition and all models of TI-84 Plus, capable of storing and retrieving variables of many kinds in 64KB of the available 96KB not in use by the OS. Information in this memory cannot be executed, but it can be swapped in and out so as to provide some alternative to archiving objects (which can wear down on Flash and cause frequent garbage collects).

Input and output will work in a fashion similar to Celtic, the commands being to move a said variable into "ExRAM", move a variable back into "User RAM", check the integrity of "ExRAM", one to determine the amount of "ExRAM" that is free, and one to format it.

Two different filesystems have been planned out for this program, so two different editions will be available nearing the end of developing this project.

The first one will operate like the TI-OS's RAM allocation and deallocation methods. Working with 64KB of RAM, the said operations can become slow, especially under the worst possible conditions where performing said functions can take upward to a full second.

The second one will work similar to FAT, allocating files a certain number of "chunks". That method will work lightning-fast, but a lot of memory will be wasted in the "slack" produced by allocating an entire sector to a small variable. For this, I plan to divide up the 64KB of RAM into 256 sectors, each 256 bytes in length.

Memory management interface is planned to work almost identical to the TI-OS's memory manager.

I do understand that applications will cause conflicts (Omnicalc, virtual calculator, RAM recovery) that could result in the data being destroyed. For that, I can only suggest that this project be used in RAM-intensive applications and programs where the user would not be able to interact with any conflicting applications during the use of program (i.e. a game). It will then become the programmer's responsibility to ensure that nothing important remains in the area of RAM at the end of the program.

Have you any thoughts regarding the project? Perhaps suggesting which edition I work on first? A better filesystem? Or do you think such a program would be, in general, useless? Maybe there's a command that you'd like to see in addition to those described?

josephmarin · « **Reply #1 on:** April 12, 2007, 05:53:00 am »

It certainly would be great to use

Radical Pi · « **Reply #2 on:** April 12, 2007, 05:55:00 am »

Extra memory is always a good thing. If you can make this and figure out how to get past some of the negatives you listed, I would certainly use this

josephmarin · « **Reply #3 on:** April 12, 2007, 05:58:00 am »

Definately. Great for BASIC RPGs when you need more memory. Could you store programs in there? Theoretically Id say yes...

Iambian · « **Reply #4 on:** April 12, 2007, 10:01:00 am »

QuoteBegin-Sonata+12 Apr, 2007, 11:58-->

QUOTE (Sonata @ 12 Apr, 2007, 11:58)

Definately. Great for BASIC RPGs when you need more memory. Could you store programs in there? Theoretically Id say yes...

You would be able to store programs, but not run them directly from "ExRAM".

Also, I was thinking of keeping track of the last address allocated in memory, so that allocation and deallocation would run faster by moving only what is needed. That is with the first TI-OS style memory allocation method. I still haven't developed anything for the second scheme, so that's still iffy.

I'm still planning this thing out on paper, since it wouldn't be the easiest thing to debug. Know that I can't use PTI to emulate this since PTI doesn't have support for the memory I'm trying to use, so development will naturally be slower than if I had an emulator that would work with this.

Any suggestions for commands that might be useful for interfacing with this move-only filesystem program? Maybe filesystem ideas?

So far, these are the specifics of the idea:

In addition to ROM pages 0 thru 7F (logically), there are RAM pages 0 thru 7, addressable as-is on port 5, or addressable through port 6 as $80 thru $87. Because of logic simplicity and the fact that the TI-OS on the TI-84 series uses RAM page 2 (for USB protocol), my program will use RAM pages 4 thru 7 and address it as if it was a single 64KB chunk. For that, I have a routine that transparently translates an address in a register (usually HL) to reflect a page and position in the $400-7FFF memory bank.

Programs are allocated from $0000 up to the end of the table, the table starts at $FFEF and grows downward. The spare 16 bytes beyond the table is reserved for filesystem variables, so they need not be stored in user RAM, and is then safe from other utilities that will use the standard saferam areas.

At the start of the table, each entry will be allocated 12 bytes per entry, irrespective of the variable's type. This is to ensure that traversing the table will happen as quickly and painlessly as possible, although a dynamic table size is certainly probable. Each entry is written backwards in memory, but each entry, shown forwards is as follows in offsets:
+0 : Data type
+1 : LSB of memory location
+2 : MSB of memory location
+3 : Reserved (possible use is a name size byte or something.
+4 to +11 : 8 bytes allocated to data's name. Null-padded.

Upon formatting this memory, one entry will be already on the table, and the variable created contains four bytes, two for its size and two for the data itself. It's name will be "!", and it serves the purpose of being a placeholder so there won't be a table with zero entries. Using the memory routines, an error will be generated if one attempts to delete "!" out of the memory.

At memory location $FFFE will reside a two-byte entry showing how much "Free RAM" there is. Although it could be calculated each time, I would like to keep it to verify the integrity of the filesystem.

At memory location $FFFC will contain a two-byte entry showing how many symbol table entries there are. This is used as a loop counter as the symbol table is traversed.

At memory location $FFFA will be the address of the last piece of data allocated in ExRAM. This value is used to calculate where the end of ExRAM is so that memory allocation and deallocation can run much more efficiently, so that they don't try to move unused memory (which would cause a massive slowdown if there was a large enough chunk between the end of the symbol table and the end of ExRAM).

More variables will be inserted into this "reserved" space as I see fit.

bfr · « **Reply #5 on:** April 12, 2007, 10:30:00 am »

This looks like it could be very useful. Good idea, Iambian.

Speler · « **Reply #6 on:** April 12, 2007, 10:41:00 am »

How safe (or un-safe) would this be to use?

josephmarin · « **Reply #7 on:** April 12, 2007, 11:17:00 am »

you could use calcsys to verify that its writing where you want it and the like

DJ Omnimaga · « **Reply #8 on:** October 10, 2007, 06:19:00 pm »

BrandonW · « **Reply #9 on:** October 23, 2007, 04:49:00 pm »

QuoteBegin-Super Speler+12 Apr, 2007, 16:41-->

QUOTE (Super Speler @ 12 Apr, 2007, 16:41)

How safe (or un-safe) would this be to use?

Lots of other programs (Virtual Calc, Omnicalc, msd8x, probably others) will use this RAM as scratch space, so that would mean your data is corrupted when you try to get those variables back. You can check to make sure your data's valid, but not recover it.

But if you know you don't use those, then you're safe.

The OS also doesn't touch these RAM pages when the calculator resets, so it's a good way of keeping RAM variables after reset, too.

Jon · « **Reply #10 on:** October 23, 2007, 04:55:00 pm »

Port 5 eh? I'm impressed

I'd make it the same as VAT entries, and have a VAT at $FFFF of each page, working down to $C000

DJ Omnimaga · « **Reply #11 on:** October 23, 2007, 04:57:00 pm »

I wonder if the hidden ram was unlocked if we could actually see the data of those apps in the mem/mngmt/del->all menu?

That said if you don't use those apps we could have one heck of a massive ASM RPG or a huge pure BASIC game

Jon · « **Reply #12 on:** October 23, 2007, 05:01:00 pm »

nope,

TIOS doesn't check those pages at all.
Its VAT entries contain a "page" byte, and if the "page" byte is any number other then 0, it's assumed that the variable in question is archived.

A bit of a hassle actually, bit 7 of the page byte determines whether it indicates an archived page or a RAM page, so the 0=RAM system that TIOS uses is rather superfluous

PS: MY $FFst POST!!!!

DJ Omnimaga · « **Reply #13 on:** October 23, 2007, 05:23:00 pm »

Congrats on post #255. I wonder why did TI put those hidden ram pages on the 83+SE anyway tho?

Jon · « **Reply #14 on:** October 23, 2007, 05:31:00 pm »

cuz it's the SE and they were just too lazy to edit the TIOS to utilize those pages. the 84+(SE) OS is almost identical to the 83+(SE) OS with the exception of USB code and some minor revisions. The OS was designed to be universal, which is why it checks bit 7 of port 2 in alot of its routines. (that bit is reset for 83+ BE, set for everything else)