91
The Axe Parser Project / Re: Axe Parser
« on: January 05, 2012, 07:02:46 am »
Announcement!
I spent several hours today thinking about the logistics, difficulty, and maintainability of creating an "Axe Shell" and then started to code my ideas. I am pleased to announce that it appears this may become a reality in the next version. I currently have about half the commands working with more to come. I will explain some of the logistic problems I faced later. This will be a beta feature to give me a chance to stabilize potential errors I might have made. It is not recommended to distribute the executable until it stabilizes.
So first of all, what am I talking about exactly? Ion, Mirage, and Doors all have a GUI application to act as a shell but Axe does not and I'm not going to add one either. What I'm referring to, is that instead of copying all the gigantic Axe routines into each program (such as DispGraph, Sprite drawing, Sound, Archive reading, and literally everything non-trivial) the program will use the Axe app itself to find and call the big routines. This of course loses platform independence in a sense, but so do all the other shells. This really isn't a shell though, its more of a framework. Which is why I'm call this the Axe Framework or AXF. With the bulk gone, this can reduce programs by up to 3000 bytes! In general, I'm seeing 10-20% reductions in my testing suite.
Now for some logistics. The way it works is that the program gets the standard ion header plus some extra code to locate the Axe app, show the message "Axe Required" if its not found, and do all the necessarily paging. Its comparable in size to the DCS or App header. This is both so that I don't have to actually create a shell and its small enough for a header anyway. But the looming problem is to map the Axe calls to physical addresses in the app. As I improve and optimize the Axe routines, the addresses will change, causing old executable to need to be recompiled to work. Normally to fix this, the other shells provide a fixed jump table that then jumps from there to whatever moving code it eventually needs to reach. But the problem with Axe is that many routines "hijack" other routines to skip parts of code rendering a fixed jump table unusable. What's worse, the axiom system of replacing absolute and native jumps and calls makes it impossible to execute over it.
To solve both of these, I have a table of subroutine "segments" that have all the code that can be skipped over and then hijack the original routine. Some routines I do have to make complete or partial copies of, but that's alright. In those cases, I leave a bit of buffer for future optimizations and changes. The only other issue I'm worried about is how Axioms will interact with the framework. Most Axioms should work, but a few really complicated ones could fail if they jump into routines past the entry points. But that is super hackish anyway.
Some other things on my mind: I could speed up some routines since the size of the routines no longer matters. Routines like multiplication are easy to unroll, but others like DispGraph are still limited by the screen driver. Also, this framework is just as useful to assembly programmers! I have doubts about how many would actually want to use it though, but nonetheless it is now possible to do that.
Thoughts and suggestions?
I spent several hours today thinking about the logistics, difficulty, and maintainability of creating an "Axe Shell" and then started to code my ideas. I am pleased to announce that it appears this may become a reality in the next version. I currently have about half the commands working with more to come. I will explain some of the logistic problems I faced later. This will be a beta feature to give me a chance to stabilize potential errors I might have made. It is not recommended to distribute the executable until it stabilizes.
So first of all, what am I talking about exactly? Ion, Mirage, and Doors all have a GUI application to act as a shell but Axe does not and I'm not going to add one either. What I'm referring to, is that instead of copying all the gigantic Axe routines into each program (such as DispGraph, Sprite drawing, Sound, Archive reading, and literally everything non-trivial) the program will use the Axe app itself to find and call the big routines. This of course loses platform independence in a sense, but so do all the other shells. This really isn't a shell though, its more of a framework. Which is why I'm call this the Axe Framework or AXF. With the bulk gone, this can reduce programs by up to 3000 bytes! In general, I'm seeing 10-20% reductions in my testing suite.
Now for some logistics. The way it works is that the program gets the standard ion header plus some extra code to locate the Axe app, show the message "Axe Required" if its not found, and do all the necessarily paging. Its comparable in size to the DCS or App header. This is both so that I don't have to actually create a shell and its small enough for a header anyway. But the looming problem is to map the Axe calls to physical addresses in the app. As I improve and optimize the Axe routines, the addresses will change, causing old executable to need to be recompiled to work. Normally to fix this, the other shells provide a fixed jump table that then jumps from there to whatever moving code it eventually needs to reach. But the problem with Axe is that many routines "hijack" other routines to skip parts of code rendering a fixed jump table unusable. What's worse, the axiom system of replacing absolute and native jumps and calls makes it impossible to execute over it.
To solve both of these, I have a table of subroutine "segments" that have all the code that can be skipped over and then hijack the original routine. Some routines I do have to make complete or partial copies of, but that's alright. In those cases, I leave a bit of buffer for future optimizations and changes. The only other issue I'm worried about is how Axioms will interact with the framework. Most Axioms should work, but a few really complicated ones could fail if they jump into routines past the entry points. But that is super hackish anyway.
Some other things on my mind: I could speed up some routines since the size of the routines no longer matters. Routines like multiplication are easy to unroll, but others like DispGraph are still limited by the screen driver. Also, this framework is just as useful to assembly programmers! I have doubts about how many would actually want to use it though, but nonetheless it is now possible to do that.
Thoughts and suggestions?