Author Topic: OmniRPG - Coding  (Read 67526 times)

0 Members and 1 Guest are viewing this topic.

Offline Sorunome

  • Fox Fox Fox Fox Fox Fox Fox!
  • Support Staff
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 7920
  • Rating: +374/-13
  • Derpy Hooves
    • View Profile
    • My website! (You might lose the game)
Re: OmniRPG - Coding
« Reply #45 on: December 08, 2012, 07:59:26 pm »
Because they contain fast routines so you don't have to write you own (most of the time slower) ones, for in example accessing archive etc
lol, that was a retorical question stating that we could use axioms then for such stuff

THE GAME
Also, check out my website
If OmnomIRC is screwed up, blame me!
Click here to give me an internet!

Offline stevon8ter

  • LV7 Elite (Next: 700)
  • *******
  • Posts: 663
  • Rating: +10/-0
    • View Profile
Re: OmniRPG - Coding
« Reply #46 on: December 08, 2012, 08:01:34 pm »
Ooooooh ok now that i read it again it makes sense, sorry xp
None of my posts are meant offending... I you feel hurt by one of my posts, tell me ... So i can appoligise me and/or explain why i posted it


Hi there, I'm the allmighty (read as: stupid, weak...) STEVON8TER

Offline Xeda112358

  • they/them
  • Moderator
  • LV12 Extreme Poster (Next: 5000)
  • ************
  • Posts: 4704
  • Rating: +719/-6
  • Calc-u-lator, do doo doo do do do.
    • View Profile
Re: OmniRPG - Coding
« Reply #47 on: December 08, 2012, 08:20:47 pm »
Well, I figure that if we use an app, we can have more code space (16KB and an 8KB program). We can still use axioms and use an app :P

Offline TIfanx1999

  • ಠ_ಠ ( ͡° ͜ʖ ͡°)
  • CoT Emeritus
  • LV13 Extreme Addict (Next: 9001)
  • *
  • Posts: 6173
  • Rating: +191/-9
    • View Profile
Re: OmniRPG - Coding
« Reply #48 on: December 08, 2012, 09:39:57 pm »
  I think we should keep it at 6MHz for the main engine and routines so that it stays nicely compatible with the 83+
^This :D

Offline Xeda112358

  • they/them
  • Moderator
  • LV12 Extreme Poster (Next: 5000)
  • ************
  • Posts: 4704
  • Rating: +719/-6
  • Calc-u-lator, do doo doo do do do.
    • View Profile
Re: OmniRPG - Coding
« Reply #49 on: December 09, 2012, 07:43:20 am »
I started working on scrolling in the up/down direction, and it is a pain o.o I have to keep track of how far up/down it is scrolled and left/right so that I can shift in new content appropriately .__. I rewrote the right/left shifting, too and I believe it is faster, but I'm not suree (I had a better idea last night, so I don't need the IX register). Also, I have a bug somewhere in the code, apparently, because the program is doing weird stuff to my calculator every so often. On the plus side, I fixed the scrolling bug for left/right >.>

On another note, if we decided to do text compression, I have a nice algorithm for that. I was thinking this morning that I could make a Python program to read a .txt file and spit out the compressed data in assembly format.

EDIT: It is still about 60FPS at 6MHz, a few extra frames per second for shifting up/down.

Offline stevon8ter

  • LV7 Elite (Next: 700)
  • *******
  • Posts: 663
  • Rating: +10/-0
    • View Profile
Re: OmniRPG - Coding
« Reply #50 on: December 09, 2012, 07:46:07 am »
Wow Xeda, it really is faster now :D and tell me your idea...? (maybe pm? ) maybe i could help you develop something to compress text? But mine would be done in c# but whatever, if i get the idea, i might be able to help...?
None of my posts are meant offending... I you feel hurt by one of my posts, tell me ... So i can appoligise me and/or explain why i posted it


Hi there, I'm the allmighty (read as: stupid, weak...) STEVON8TER

Offline Xeda112358

  • they/them
  • Moderator
  • LV12 Extreme Poster (Next: 5000)
  • ************
  • Posts: 4704
  • Rating: +719/-6
  • Calc-u-lator, do doo doo do do do.
    • View Profile
Re: OmniRPG - Coding
« Reply #51 on: December 09, 2012, 08:21:06 am »
I'll describe it here, since I don;t think I ever released the idea, just the code. Basically, it works like this (there are two methods)

I first designed this algorithm using Celtic3+BASIC, using the idea of how common certain letters were in the English alphabet. I arranged them in order and threw in some numbers and punctuation, so I could compress my most common messages:
Code: [Select]
ETAOINSHRDLCUMFGYPBVKWXJQZ,?!.0123456789/-:'
Note that the first character is a space. Now, in the actual message, any time one of the first 14 characters appear, it will be assigned a nibble 0~D. If anything after that is used, (say the 23 character in the list), it will be a two-nibble value, starting with E or F. You can subtract 14 and add $E0 to it (essentially, add $D2 to the original value). In this way, text will take at the worst 100% the original size and at the best 50%. However, because the first 14 characters are the most common to appear in an English message, it averages around 60% compression. For example, here is the message, "HELLO!" (this works best with larger messages):
NOTE: We start counting at zero.
H is the 9th character, so it is one nibble, $8
E is the 2nd character, so it is one nibble, $1
L is the 12th character, so it is one nibble, $B
O is the 5th character, so it is one nibble, $4
! is 30th character, so it is two nibbles: $EF
Combining the nibbles, we get 81BB4EF, but we need to store bytes, and this is an odd number of nibbles, so we pad it with a 0. This is why a space is the first character, because decompressed, it would put a space at the end in cases like this.

Anyways, that is 4 bytes, compared to 6, so compression was about 67% In BatLib, to make sure the extra space at the end wasn't in the output for decompression, I compress data and store the original size as a two byte value in the beginning.

Now, why did we use 46 chars? The first 14 are used as 1-nibble values. A nibble can be 16 different values, so this leaves two extra nibbles for our "extended" chars. So 2*16=32, which means we can have 32 two-nibble values and 14 nibble values, totaling 46. Indexing at 0, we have chars numbered 0 to 45, which in hex is 00h~2Dh. If you look at what we add to find the value of 2-nibble chars ($D2), it is actually not a coincidence that it is the value 2D with its nibbles swapped :D

So what if we want to compress something other than text? What if it is arbitrary data? Basically, you need to scan the data to figure out which byte values are used, then you need to arrange them in descending popularity (I call this a "compression key"). Find your cut-off value (the number you add to get the two -nibble values) and this will also tell you your cut-off for one-nibble/two-nibble values. Because this will use a unique set of chars in some order, you will also need to include the "compression key" so that you can decompress. If the data isn't large enough, the compression key plus the compressed data will take up more space than the original.

I used this in BatLib to compress some internal data, like the main menu (the bitmap was compressed a few hundred bytes, including the compression key). You can also apply this algorithm several times in some cases to further compress the data. I got one tilemap to compress down to 12% of its original size, with each of the compression keys included!

EDIT: This is the relevant code in BatLib for most of the compression/decompression functions:
Spoiler For Spoiler:
Code: [Select]
CompressStr:
     call LoadInc
     ld hl,Codec1
     dec a
     jr nz,$+5
     ld hl,Codec2
     jr $+5
CompData:
     call CustCodec
     ld (TempWord2),hl
     call StringOrName
     ld (TempWord1),hl
     push bc
     push af
     push hl
     inc bc
     inc bc
     call MakeAnsStrX
     pop hl
     pop af
     pop bc
     ex de,hl
     ld (hl),c
     inc hl
     ld (hl),b
     inc hl
     ex de,hl
     push bc
     push de
     call ReadArc
     pop hl
     pop bc
     push hl
     push de
     ld de,(TempWord1)
     ld ix,(TempWord2)
     call TextComp
     pop de
     inc hl
;HL points to the end of where it was converted
     ex de,hl
     or a
     sbc hl,de
     ex de,hl
;DE is the size of the data to delete
;HL points to where to delete the data
     push hl
     bcall(4357h)
     pop hl
     pop de
     dec de
     dec de
     or a
     sbc hl,de
     ex de,hl
     dec hl
     ld (hl),d
     dec hl
     ld (hl),e
     ret

TextComp:
;Inputs:
;     DE points to the text to convert
;     HL points to where to convert to
;     BC is the size of the string
;     IX points to the codec
     call CodecStats
     ld (8478h),bc
     ld b,0
     ld (hl),0
TextCompLoop:
     push hl
     push bc
     ld bc,(8478h)
     ld a,b \ or c
     jr nz,$+11
       pop bc
       pop hl
       bit 0,b
       ret z
       xor a
       rld
       ret
     dec bc
     ld (8478h),bc
     ld a,(de)
     inc de
     ld c,(ix)
     push ix
     pop hl
     inc hl
     cpir

     push hl
     push af
     ld a,(ix)
     dec a
     ld h,a
     pop af
     ld a,h
     pop hl
     
     jr z,$+3
       ld c,a
     sub c
     pop bc
     pop hl
     exx
     cp c
     exx
     jr c,$+23
     exx
     sub b
     exx
     rlca \ rlca \ rlca \ rlca
     rld
     rlca \ rlca \ rlca \ rlca
     inc b
     bit 0,b
     jr nz,$+4
       inc hl
       ld (hl),0
     rld
     inc b
     bit 0,b
     jr nz,TextCompLoop
     inc hl
     jr TextCompLoop-2
Codec1:
;46 chars
; ETAOINSHRDLCUMFGYPBVKWXJQZ,?!.0123456789/-:'
 .db Codec1End-Codec1
 .db 29h,"ETAOINSHRDLCUMFGYPBVKWXJQZ",2Bh
 .db $AF,2Dh,3Ah,"0123456789",83h,71h,3Eh,$AE
Codec1End:
Codec2:
;Includes lowercase tokens
 .db Codec2End-Codec2
 .db $29,$BB,$45,$B4,$54,$C4,$41,$B0,$4F,$BF,$49,$B8,$4E,$BE,$53,$C3,$48,$B7,$52,$C2,$44,$B3,$4C,$BC
 .db $43,$B2,$55,$C5,$4D,$BD,$46,$B5,$47,$B6,$59,$C9,$50,$C0,$42,$B1,$56,$C6,$4B,$BA,$57,$C7,$58,$C8
 .db $4A,$B9,$51,$C1,$5A,$CA,$2B,$AF,$2D,$3A,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$83,$71,$3E,$AE
;need 4 more bytes
 .db 3Fh,82h,70h,$B0
Codec2End:
CustCodec:
     call StringOrName
     ld de,86ECh
     push de
     call ReadArc
     pop hl
     ret
DeCompStr:
     call LoadInc
     ld hl,Codec1
     dec a
     jr nz,$+5
     ld hl,Codec2
     jr $+5
DeCompData:
     call CustCodec
     ld (TempWord2),hl
     call StringOrName
     or a
     jp nz,ARCH
     dec bc
     dec bc
     push bc
     ld c,(hl)
     inc hl
     ld b,(hl)
     inc hl
     push hl
     inc bc
     call MakeAnsStrX
     ld (TempWord1),bc
     pop hl
     pop bc
     push de
     ld ix,(TempWord2)
     call TextDeComp
;DE is the size of the data to delete
;HL points to where to delete the data
     ex de,hl
     ld de,1
     bcall(4357h)
     pop hl
     dec hl
     dec hl
     ld a,(hl)
     dec a
     ld (hl),a
     inc a
     ret nz
     inc hl
     dec (hl)
     ret
CodecStats:
;     IX points to the codec
;        codec format is the first byte is the size, followed by the bytes allowed
;Output:
;     B' is the adjusting value for one-byte values
;     C' is the cut-off for half-byte values
;     A is the same as B'
     di
     exx
     ld a,(ix)
     rlca
     rlca
     rlca
     rlca
     and 15
     ld b,a
   ld a,(ix)
   and 15
   add a,b
   and 240
   jr z,$+3
   inc b
     ld a,16
     sub b       ;
     ld c,a      ;number of half nibbles
     ld a,b
     rlca
     rlca
     rlca
     rlca
     or c
     ld b,a
     exx
     ret
TextDeComp:
;Inputs:
;     HL points to the text to decompress
;     DE points to where to convert to
;     BC is the size of the string
;     IX points to the codec
     call CodecStats
     ld (8478h),bc
     ld b,0
TextDeCompLoop:
     call GetNextNibble
     ld a,c
     exx
     cp c
     exx
     jr c,$+15
       ld (de),a
       call GetNextNibble
       ld a,(de)
       rlca \ rlca \ rlca \ rlca
       or c
       exx
       add a,b
       exx
     push hl
     push ix
     pop hl
     inc hl
     add a,l
     jr nc,$+3
     inc h
     ld l,a
     ld a,(hl)
     pop hl
     ld (de),a
     inc de
     push bc
     ld bc,(8478h)
     ld a,b \ or c
     pop bc
     jr nz,TextDeCompLoop
     ret
GetNextNibble:
; z flag set if the last
     xor a
     inc b
     bit 0,b
     jr z,$+8
     rld
     ld c,a
     rrd
     ret
     rrd
     ld c,a
     rld
     push bc
     ld bc,(8478h)
     dec bc
     ld (8478h),bc
     pop bc
     inc hl
     ret     
GetCodec:
     call StringOrName
     or a
     ret nz
     or b
     or c
     jp z,NO_DATA
     call CountBytes
     call MakeAnsStrX
     dec bc
     ld a,c
     ld (de),a
     inc de
     ld hl,88ECh
     ldir
     ret
CountBytes:
     push hl
     push bc
     xor a
     ld hl,86ECh
     ld (hl),a
     ld d,h
     ld e,l
     inc de
     ld bc,2FFh
     ldir
     pop bc
     pop hl
GetCodecLoop:
     push bc
     ld e,(hl)
     ld d,0
     ex de,hl
     add hl,hl
     ld bc,86ECh
     add hl,bc
     inc (hl)
     jr nz,$+4
     inc hl
     inc (hl)
     ex de,hl
     pop bc
     cpi
     jp pe,GetCodecLoop
     call Order
     ld b,0
     inc bc
     ret
Order:
     di
     ld hl,86ECh     ;points to the density list (512 bytes)
     ld de,88ECh     ;points to where the output is
     ld bc,0
     ld (TempWord1),hl
     exx
     ld hl,0         ;MaxVal
     exx
OrderLoop:
     push bc
     ld c,(hl)
     inc hl
     ld b,(hl)
     inc hl
     push bc
     exx
     pop de
     or a
     sbc hl,de
     add hl,de
     jr nc,$+3
       ex de,hl
     exx
     pop bc
     djnz OrderLoop
     ld hl,(TempWord1)
     exx
     ld a,h
     or l
     exx
     ret z
     xor a
OrderLoop1:
     push bc
     ld c,(hl)
     inc hl
     ld b,(hl)
     push bc
     exx
     pop de
     or a
     sbc hl,de
     add hl,de
     exx
     pop bc
     jr nz,$+12
       ld (de),a
       inc de
       ld (hl),0
       dec hl
       ld (hl),0
       inc hl
       inc c
       ret z
     inc hl
     inc a
     djnz OrderLoop1
     ld hl,(TempWord1)
     jr OrderLoop-5

Offline aeTIos

  • Nonbinary computing specialist
  • LV12 Extreme Poster (Next: 5000)
  • ************
  • Posts: 3915
  • Rating: +184/-32
    • View Profile
    • wank.party
Re: OmniRPG - Coding
« Reply #52 on: December 09, 2012, 08:46:28 am »
Are there any other things to do? I'd like to code some stuffs.
I'm not a nerd but I pretend:

Offline stevon8ter

  • LV7 Elite (Next: 700)
  • *******
  • Posts: 663
  • Rating: +10/-0
    • View Profile
Re: OmniRPG - Coding
« Reply #53 on: December 09, 2012, 08:48:22 am »
So if i get it right, the '.' would be $F0 ? or $F1 ?
None of my posts are meant offending... I you feel hurt by one of my posts, tell me ... So i can appoligise me and/or explain why i posted it


Hi there, I'm the allmighty (read as: stupid, weak...) STEVON8TER

Offline Xeda112358

  • they/them
  • Moderator
  • LV12 Extreme Poster (Next: 5000)
  • ************
  • Posts: 4704
  • Rating: +719/-6
  • Calc-u-lator, do doo doo do do do.
    • View Profile
Re: OmniRPG - Coding
« Reply #54 on: December 09, 2012, 08:51:21 am »
@stevon8ter: yep :)
@aeTIos: I dunno, I just started experimenting with stuff. I don't think we even have a plan of what there is to do .__. We should keep the first post edited with a To Do list as well as the people working on the different parts.

Offline aeTIos

  • Nonbinary computing specialist
  • LV12 Extreme Poster (Next: 5000)
  • ************
  • Posts: 3915
  • Rating: +184/-32
    • View Profile
    • wank.party
Re: OmniRPG - Coding
« Reply #55 on: December 09, 2012, 08:52:24 am »
Just try in Axe: text(0,,241)
I'm not a nerd but I pretend:

Offline Xeda112358

  • they/them
  • Moderator
  • LV12 Extreme Poster (Next: 5000)
  • ************
  • Posts: 4704
  • Rating: +719/-6
  • Calc-u-lator, do doo doo do do do.
    • View Profile
Re: OmniRPG - Coding
« Reply #56 on: December 09, 2012, 08:53:41 am »
@aeTIos: He means using the compression algorithm I gave. '.' corresponds to the two-nibble value $F0

Offline aeTIos

  • Nonbinary computing specialist
  • LV12 Extreme Poster (Next: 5000)
  • ************
  • Posts: 3915
  • Rating: +184/-32
    • View Profile
    • wank.party
Re: OmniRPG - Coding
« Reply #57 on: December 09, 2012, 08:54:11 am »
Oh nevermind in that case :p
I'm not a nerd but I pretend:

Offline stevon8ter

  • LV7 Elite (Next: 700)
  • *******
  • Posts: 663
  • Rating: +10/-0
    • View Profile
Re: OmniRPG - Coding
« Reply #58 on: December 09, 2012, 08:59:41 am »
Yeah :p

And we should indeed keep track of a to do list, many people are asking what they can do but noone has a concrete plan. Xeda is the only one who actually coded something for this already

EDIT: xeda , you say there are 46 chars... but for now there are 45 right? with a possiblitie for a 46th char...?
« Last Edit: December 09, 2012, 09:06:23 am by stevon8ter »
None of my posts are meant offending... I you feel hurt by one of my posts, tell me ... So i can appoligise me and/or explain why i posted it


Hi there, I'm the allmighty (read as: stupid, weak...) STEVON8TER

Offline aeTIos

  • Nonbinary computing specialist
  • LV12 Extreme Poster (Next: 5000)
  • ************
  • Posts: 3915
  • Rating: +184/-32
    • View Profile
    • wank.party
Re: OmniRPG - Coding
« Reply #59 on: December 09, 2012, 09:05:32 am »
Well I can imagine a lot of things that I could already do. Just not sure if they're really needed since it's still in planning stages.
I'm not a nerd but I pretend: