Author Topic: 8xp File Structure  (Read 6925 times)

0 Members and 1 Guest are viewing this topic.

Offline BlakPilar

  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 734
  • Rating: +44/-1
    • View Profile
8xp File Structure
« on: July 26, 2011, 04:52:19 pm »
I have a document (I can't remember where I got it from) and I was just wondering if it is 100% accurate. If it is, can someone explain the checksum? Thanks.

Code: [Select]
All values are in decimal.

8XP File:
(1-11)Header (11 bytes): 42,42,84,73,56,51,70,42,26,10,0
(12-52)Comment (42 bytes): 42 bytes of ASCII, if the comment is <42 bytes then fill the rest of the space with 0s
(53)Comment Delimiter: 0
(54-55)Length of the Data Section (2 bytes): Length of the data section+19
(56-57)Random Stuff (2 bytes): 13,0
(58-59)Length of the Data Section (2 bytes): Length of the data section+2
(60)Protected (1 byte): 6 if yes, 5 if no
(61-68)Program Name (8 bytes): Program Name (in ASCII)-fill with 0s if <8 bytes
(69-70)More Random Stuff (2 bytes): 0,0
(71-72)Length of the Data Section (2 bytes): Length of the data section+2
(73-74)Length of the Data Section (2 bytes): Length of the data section
(75-?)Data Section (Varies): The program's data
Checksum: Checksum of 56 bytes to the byte before the checksum

Offline Munchor

  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 6199
  • Rating: +295/-121
  • Code Recycler
    • View Profile
Re: 8xp File Structure
« Reply #1 on: July 26, 2011, 05:01:39 pm »
That is not very reliable, use merthsoft's tables, they are better.

Checksum:
A byte has a value (numerical), it's the sum of the numerical value of all the bytes.

Offline Scipi

  • Omni Kitten Meow~ =^ω^=
  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1547
  • Rating: +192/-3
  • Meow :3
    • View Profile
    • ScipiSoftware
Re: 8xp File Structure
« Reply #2 on: July 26, 2011, 05:06:24 pm »
Would the checksum include itself? And the would the checksum's value = 74 + bytes of data?

Imma Cat! =^_^= :3 (It's an emoticon now!)
Spoiler For Things I find interesting:
Spoiler For AI Programming:
Spoiler For Shameless advertising:

Spoiler For OldSig:





Spoiler For IMPORTANT NEWS!:
Late last night, Quebec was invaded by a group calling themselves, "Omnimaga". Not much is known about these mysterious people except that they all carried calculators of some kind and they all seemed to converge on one house in particular. Experts estimate that the combined power of their fabled calculators is greater than all the worlds super computers put together. The group seems to be holding out in the home of a certain DJ_O, who the Omnimagians claim to be their founder. Such power has put the world at a standstill with everyone waiting to see what the Omnimagians will do...

Wait... This just in, the Omnimagians have sent the UN a list of demands that must be met or else the world will be "submitted to the wrath of Netham45's Lobster Army". Such demands include >9001 crates of peanuts, sacrificial blue lobsters, and a wide assortment of cherry flavored items. With such computing power stored in the hands of such people, we can only hope these demands are met.

In the wake of these events, we can only ask, Why? Why do these people make these demands, what caused them to gather, and what are their future plans...

Offline Munchor

  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 6199
  • Rating: +295/-121
  • Code Recycler
    • View Profile
Re: 8xp File Structure
« Reply #3 on: July 26, 2011, 05:09:12 pm »
Would the checksum include itself? And the would the checksum's value = 74 + bytes of data?

Not sure if it's 74, but it makes sense.

It's like the whole file is 74 bytes (header), the program code is X bytes. So the checksum is 74+numericalvalueOf(X).

Offline Michael_Lee

  • LV9 Veteran (Next: 1337)
  • *********
  • Posts: 1019
  • Rating: +124/-9
    • View Profile
Re: 8xp File Structure
« Reply #4 on: July 26, 2011, 05:13:26 pm »
This is what I use (all values that start with '$' are in hex):

Code: [Select]
METADATA:

Sizes:
      Offset    Length  Description

Packet
|
| Header
| |   0         8       **TI83F*
| |   8         3       $1A $0A $00
| |   11        42      Comment - padded with zeros or spaces.
| |   53        2       Length of data section.  Should be equal to filesize - 57
| End Header
|
| Data
| |   55        2       Always has a value of $0B or $0D? (prefer $0B?)
| |   57        2       Length of Variable Data Section
| |   59        1       Variable type ID ($05 for programs)
| |   60        8       Variable name (padded with zeros)
| |   68        1       Version (??), usually $00.  (Present if byte 55 == $0D)
| |   69        1       Archive flag.  $80 archived, $00 if not. (Present if byte 55 == $0D)
| |   70        2       Length of Variable Data Section (repeat of bytes @ 57)
| |
| | Variable Data
| | | 72        2       Length of Actual Data / 2
| | | 74        n       Actual Data
| | End Variable Data
| |
| End Data
|
|    74 + n    2       Checksum (sum of Data section)
|
End Packet

And the checksum is the sum of everything in the Data section only.
« Last Edit: July 26, 2011, 05:27:34 pm by Michael_Lee »
My website: Currently boring.

Projects:
Axe Interpreter
   > Core: Done
   > Memory: Need write code to add constants.
   > Graphics: Rewritten.  Needs to integrate sprites with constants.
   > IO: GetKey done.  Need to add mostly homescreen IO stuff.
Croquette:
   > Stomping bugs
   > Internet version: On hold until I can make my website less boring/broken.

Offline BlakPilar

  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 734
  • Rating: +44/-1
    • View Profile
Re: 8xp File Structure
« Reply #5 on: July 26, 2011, 08:00:26 pm »
@ephan, I tried their tables but they weren't very in-depth, unless I was looking at the wrong ones. By numerical value, do you mean like if the program was >DMS>DMS then the data section would be 01-01, so that value would be 2? Or is it just the length of the DataSection?

@Michael_Lee, thanks! I'll try doing everything that way when I get back to my computer. Just a question though... Isn't the part where it says "final size - 57" the same as Data Section + 19 (76, 74 bytes before the Data Section and the checksum, minus that 57)?

Offline Michael_Lee

  • LV9 Veteran (Next: 1337)
  • *********
  • Posts: 1019
  • Rating: +124/-9
    • View Profile
Re: 8xp File Structure
« Reply #6 on: July 26, 2011, 08:19:25 pm »
@Michael_Lee, thanks! I'll try doing everything that way when I get back to my computer. Just a question though... Isn't the part where it says "final size - 57" the same as Data Section + 19 (76, 74 bytes before the Data Section and the checksum, minus that 57)?

Erm, I'm not really sure what you mean -- could you clarify?

I do know that in the Data section, the length of the Variable Data section is repeated twice, if that's what I think you're asking.
My website: Currently boring.

Projects:
Axe Interpreter
   > Core: Done
   > Memory: Need write code to add constants.
   > Graphics: Rewritten.  Needs to integrate sprites with constants.
   > IO: GetKey done.  Need to add mostly homescreen IO stuff.
Croquette:
   > Stomping bugs
   > Internet version: On hold until I can make my website less boring/broken.

Offline BlakPilar

  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 734
  • Rating: +44/-1
    • View Profile
Re: 8xp File Structure
« Reply #7 on: July 26, 2011, 08:23:46 pm »
Okay, um... In yours, at the end of its header section, it says "filesize - 57". Wouldn't that be the same as where in mine it says "Data Section + 19"? Because the 74 bytes before the data section plus the 2 from the checksum is 76. Then 76 minus 57, from yours, is +19. It's a dumb question lol, I'm just checking.

Offline Michael_Lee

  • LV9 Veteran (Next: 1337)
  • *********
  • Posts: 1019
  • Rating: +124/-9
    • View Profile
Re: 8xp File Structure
« Reply #8 on: July 26, 2011, 08:31:24 pm »
Ok, I see what you mean.  Yep, I think the two should be the same.

A quick comment: in your document, you have a comment delimiter at byte 53: I don't think there's actually one.  If you remove that, and shift down all the offsets accordingly, your file format should output something identical to mine.
My website: Currently boring.

Projects:
Axe Interpreter
   > Core: Done
   > Memory: Need write code to add constants.
   > Graphics: Rewritten.  Needs to integrate sprites with constants.
   > IO: GetKey done.  Need to add mostly homescreen IO stuff.
Croquette:
   > Stomping bugs
   > Internet version: On hold until I can make my website less boring/broken.

Offline BlakPilar

  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 734
  • Rating: +44/-1
    • View Profile
Re: 8xp File Structure
« Reply #9 on: July 26, 2011, 09:18:11 pm »
Hmm... My exporting seems to be slightly off... It works generally, but it's a little broken...

Here's what happens with DRIFTER as an example program. I open it, then export it as a different file. Opening works perfectly.

Spoiler For Open:
Code: [Select]
:"Drifter
StoreGDB 1
/E/9rand→W
Float:Full:FnOff :AxesOff:GridOff:PlotsOff
ZStandard
104→Xmax
⁻72→Ymin
ZInteger
"(X]X(:@y:@y:[[X[[ [YY ++@y(++|cross|[|cross|++(@y+++^[^+ ]:+ ^+|cross|+^ +:] |cross|+^+|cross|í℮J  x√^oY    Y^ox√  J℮í→Str2
SetUpEditor DRIFT
1→dim(˪DRIFT
˪DRIFT(1→A
Ans<0→Q
abs(A
Ans>/E/2→R
{abs(A/(8Ans+1→DRIFT
Goto M
Lbl 4
End
Lbl 3
End
End
Lbl 1
End
Lbl M
DelVar ZClrDraw
For(A,1,7
For(B,17,18
StorePic 1
Text(⁻1,2,7A+B,sub("DRIFTER",A,1
RecallPic 1
End
End
Text(10,17,"BY:ZACHARY TULLER
rand(8
getKey
23
Text(20,Ans,"1. PLAY DRIFTER
Text(28,Ans,"2. PLAY CUSTOM LEVEL
Text(36,Ans,"3. LEVEL EDITOR
Text(44,Ans,"4. EXIT
Repeat max(Ans={22,45,82,92,93,94
getKey
End
If Ans=22 or 1=abs(Ans-93
Then
For(A,20,49
Line(20,⁻A,91,⁻A,0
End
End
Spoiler For Open, then save, then open again:
Code: [Select]
:"D
StoreGDB 1
/E/9→W
Float:F:FO :AO:GO:PO
ZStandard
104→X
⁻72→Y
ZInteger
"(X]X(:::[[X[[ [YY ++(++[++(+++^[^+ ]:+ ^++^ +:] +^+J  ^Y    Y^  J→S2
DRIFT
1→(˪DRIFT
˪DRIFT(1→A
Ans<0→Q
abs(A
Ans>/E/2→R
{(A/(8A+1→DRIFT
Goto M
Lbl 4
End
Lbl 3
End
End
Lbl 1
End
Lbl M
ZCD
For(A,1,7
For(B,17,18
StorePic 1
Text(⁻1,2,7A+B,("DRIFTER",A,1
RecallPic 1
End
End
Text(10,17,"BY:ZACHARY TULLER
rand(8
getKey
23
Text(20,A,"1. PLAY DRIFTER
Text(28,A,"2. PLAY CUSTOM LEVEL
Text(36,A,"3. LEVEL EDITOR
Text(44,A,"4. EXIT
Repeat max(Ans={22,45,82,92,93,94
getKey
End
If Ans=22  1=(A-93
Then
For(A,20,49
Line(20,⁻A,91,⁻A,0
End
End
« Last Edit: July 27, 2011, 04:21:33 pm by BlakPilar »

Offline Munchor

  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 6199
  • Rating: +295/-121
  • Code Recycler
    • View Profile
Re: 8xp File Structure
« Reply #10 on: July 27, 2011, 04:26:53 am »
Hmm... My exporting seems to be slightly off... It works generally, but it's a little broken...

Here's what happens with DRIFTER as an example program. I open it, then export it as a different file. Opening works perfectly.

Spoiler For Open:
Code: [Select]
:"Drifter
StoreGDB 1
/E/9rand→W
Float:Full:FnOff :AxesOff:GridOff:PlotsOff
ZStandard
104→Xmax
⁻72→Ymin
ZInteger
"(X]X(:@y:@y:[[X[[ [YY ++@y(++|cross|[|cross|++(@y+++^[^+ ]:+ ^+|cross|+^ +:] |cross|+^+|cross|í℮J  x√^oY    Y^ox√  J℮í→Str2
SetUpEditor DRIFT
1→dim(˪DRIFT
˪DRIFT(1→A
Ans<0→Q
abs(A
Ans>/E/2→R
{abs(A/(8Ans+1→DRIFT
Goto M
Lbl 4
End
Lbl 3
End
End
Lbl 1
End
Lbl M
DelVar ZClrDraw
For(A,1,7
For(B,17,18
StorePic 1
Text(⁻1,2,7A+B,sub("DRIFTER",A,1
RecallPic 1
End
End
Text(10,17,"BY:ZACHARY TULLER
rand(8
getKey
23
Text(20,Ans,"1. PLAY DRIFTER
Text(28,Ans,"2. PLAY CUSTOM LEVEL
Text(36,Ans,"3. LEVEL EDITOR
Text(44,Ans,"4. EXIT
Repeat max(Ans={22,45,82,92,93,94
getKey
End
If Ans=22 or 1=abs(Ans-93
Then
For(A,20,49
Line(20,⁻A,91,⁻A,0
End
End
Spoiler For Open, then save, then open again:
Code: [Select]
:"D
StoreGDB 1
/E/9→W
Float:F:FO :AO:GO:PO
ZStandard
104→X
⁻72→Y
ZInteger
"(X]X(:::[[X[[ [YY ++(++[++(+++^[^+ ]:+ ^++^ +:] +^+J  ^Y    Y^  J→S2
DRIFT
1→(˪DRIFT
˪DRIFT(1→A
Ans<0→Q
abs(A
Ans>/E/2→R
{(A/(8A+1→DRIFT
Goto M
Lbl 4
End
Lbl 3
End
End
Lbl 1
End
Lbl M
ZCD
For(A,1,7
For(B,17,18
StorePic 1
Text(⁻1,2,7A+B,("DRIFTER",A,1
RecallPic 1
End
End
Text(10,17,"BY:ZACHARY TULLER
rand(8
getKey
23
Text(20,A,"1. PLAY DRIFTER
Text(28,A,"2. PLAY CUSTOM LEVEL
Text(36,A,"3. LEVEL EDITOR
Text(44,A,"4. EXIT
Repeat max(Ans={22,45,82,92,93,94
getKey
End
If Ans=22  1=(A-93
Then
For(A,20,49
Line(20,⁻A,91,⁻A,0
End
End

Double Posting is not allowed in those conditions, edit your post. Sigh, you should read the rule number 4:

Quote
4: The only time double-posting is allowed (posting two messages in a row in the same topic) is either if there has been 6 hours since your last post, 3 hours if your thread has been bumped out of the New Posts page, 1 hour if it's a project update or immediately if you need to upload more than 10 file attachments at once. Otherwise, update the last post using the "EDIT" button. It will be marked as unread for all members (altough it won't bump the thread).

Read all of them by the way:

http://www.omnimaga.org/index.php?action=ezportal;sa=page;p=2

Offline BlakPilar

  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 734
  • Rating: +44/-1
    • View Profile
Re: 8xp File Structure
« Reply #11 on: July 27, 2011, 04:31:01 pm »
Alright, I did, but could you clarify on the checksum? Say a program was just
Code: [Select]
9->AThe data section would be 39-04-41 (in bytes). Would the checksum be 124, or 3?
« Last Edit: July 27, 2011, 04:31:16 pm by BlakPilar »

Offline Michael_Lee

  • LV9 Veteran (Next: 1337)
  • *********
  • Posts: 1019
  • Rating: +124/-9
    • View Profile
Re: 8xp File Structure
« Reply #12 on: July 28, 2011, 01:46:07 am »
(Disclaimer: it's late at night, and I probably got something wrong.  Any corrections or modifications will be welcomed with open arms.)

The checksum is the sum of all the data from offsets starting from byte 55.

That means the sum of all the data in this region:

Code: [Select]
METADATA:

Sizes:
      Offset    Length  Description
<snip>
| Data
| |   55        2       Always has a value of $0B or $0D? (prefer $0B?)
| |   57        2       Length of Variable Data Section
| |   59        1       Variable type ID ($05 for programs)
| |   60        8       Variable name (padded with zeros)
| |   68        1       Version (??), usually $00.  (Present if byte 55 == $0D)
| |   69        1       Archive flag.  $80 archived, $00 if not. (Present if byte 55 == $0D)
| |   70        2       Length of Variable Data Section (repeat of bytes @ 57)
| |
| | Variable Data
| | | 72        2       Length of Actual Data / 2
| | | 74        n       Actual Data
| | End Variable Data
| |
| End Data
<snip>
...is the checksum.

For example, if I had the program,
Code: [Select]
3->A

the full 8xp (in hex) would be the following:

Code: [Select]
Packet
|
| Header
| | **TI83F*                   2A2A54493833462A
| | (part of signature)        1A0A00
| | Comment                    546F6B656E697A656420
| |                            7573696E67205449436F
| |                            6E766572742076302E35
| |                            00000000000000000000
| |                            0000
| | Length of Data section     1600
| End Header
|
| Data
| | Mystery value              0B00
| | Length of Variable Data    0500
| | Variable type ID           05
| | Variable name (TEST)       5445535400000000
| | Version (??)               00
| | Archive flag               00
| | Length of Variable Data    0500
| |
| | Variable Data
| | | Length of actual data    0300
| | | Actual data: 3->A        330441
| | End Variable Data
| |
| End Data
|
| Checksum                     D501
|
End Packet

The length of the Data section (present in the header) and the length of the Variable Data (present in the Data section) all are the number of bytes in their respective sections.  For example, if you were to count all the bytes in the Data section, you would get 22, which is 16 in hexadecimal, which, as you can see, is the the value recorded right after the comment.

The checksum, however, is the sum of all the bytes. 

0B + 00 + 05 + 00 + 05 + 54 + 45 + 53 + (etc, might have missed a few numbers there) + 33 + 04 + 41 = 1D5 (in hex).

The least significant byte goes first, so the checksum is D501.

(Random question: why is it that in the 8xp file, the least significant byte goes first, but looking at the actual data, specifically two-byte-tokens, the least significant byte goes last?  Wouldn't it be more consistent for the least significant byte to go last in the 8xp file format too?)
« Last Edit: July 28, 2011, 01:48:32 am by Michael_Lee »
My website: Currently boring.

Projects:
Axe Interpreter
   > Core: Done
   > Memory: Need write code to add constants.
   > Graphics: Rewritten.  Needs to integrate sprites with constants.
   > IO: GetKey done.  Need to add mostly homescreen IO stuff.
Croquette:
   > Stomping bugs
   > Internet version: On hold until I can make my website less boring/broken.

Offline BlakPilar

  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 734
  • Rating: +44/-1
    • View Profile
Re: 8xp File Structure
« Reply #13 on: July 28, 2011, 10:33:30 am »
Quote
The checksum, however, is the sum of all the bytes.

Ok, thanks!