Author Topic: ELF Loader for Ndless - (now ready for non-dev use)  (Read 18338 times)

0 Members and 1 Guest are viewing this topic.

Offline tangrs

  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 229
  • Rating: +98/-0
    • View Profile
    • tangrs blog
ELF Loader for Ndless - (now ready for non-dev use)
« on: December 15, 2011, 04:41:52 am »
The ELF loader has reached a stage where most people can use it to load basic stuff.

When you run the program, it installs a hook. After that, you can load ELF files as if you'd load a normal Ndless binary.

More instructions on how to compile the loader and how to develop applications that take advantage of it are described in the README.

You can get the source code on Github.

Happy deving!

I'd also like to thank everyone who contributed to Ndless. I did borrow some of Ndless's code for use in this loader.

Original post:

I'm not sure if it's relevant here, but I'm developing an opensource ELF loader for Ndless.

I'm writing this because the current Ndless way of loading binaries doesn't work for code that relies on static initialization of pointers.

I.e. code like this doesn't work:

Code: [Select]
void foo() {
  //blah
}

int main() {
  static void (*var)() = foo; //Since it's static, the address will be inserted at link time (which is 0x8000+offset on my machine)
  var(); //The GOT based relocation code in Ndless currently does not update the static variable function pointer.
  return 0; //Crash
}

Unfortunately, there's no way to fix this because Ndless binaries are converted into a memory image before running. That means the relocation code doesn't know what to update in the .data section of the memory because it doesn't know what symbols exist and need updating and where they're located.

Loading from an ELF file works because the symbol definitions are there and the relocation code knows where to find the bits that need patching. That's why I wrote this ELF loader.

Anyway, the core code is there, just needs a lot of polishing up.

If anyone wants to help, the source code is available on Github https://github.com/tangrs/ndless-elfloader
I'm hoping eventually, it will be integrated into the program loader on Ndless.

Thanks for your time,
Apologies if this is the wrong forum to post.
« Last Edit: January 17, 2012, 12:40:47 am by tangrs »

Offline Jim Bauwens

  • Lua! Nspire! Linux!
  • Editor
  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1881
  • Rating: +206/-7
  • Linux!
    • View Profile
    • nothing...
Re: ELF Loader for Ndless
« Reply #1 on: December 15, 2011, 05:09:01 am »
Interesting :)

I don't however know much about this subject to give a good response :P
But thanks for sharing :)

(on a side note, did you get my pm of a few day's ago?)

Offline tangrs

  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 229
  • Rating: +98/-0
    • View Profile
    • tangrs blog
Re: ELF Loader for Ndless
« Reply #2 on: December 15, 2011, 05:20:45 am »
Interesting :)

I don't however know much about this subject to give a good response :P
But thanks for sharing :)

(on a side note, did you get my pm of a few day's ago?)

Yeah, I did. I couldn't respond because I'm a new user and can't send messages yet or something.

Offline Lionel Debroux

  • LV11 Super Veteran (Next: 3000)
  • ***********
  • Posts: 2135
  • Rating: +290/-45
    • View Profile
    • TI-Chess Team
Re: ELF Loader for Ndless
« Reply #3 on: December 15, 2011, 05:23:11 am »
Interesting indeed :)

The flip side of the coin of the power of ELF is its complexity, and the weight of a full-featured launcher. In the past, there were discussions about using BFLT, which is a much simpler format, easily derived from ELF, and fulfills the purpose you're describing.
Member of the TI-Chess Team.
Co-maintainer of GCC4TI (GCC4TI online documentation), TILP and TIEmu.
Co-admin of TI-Planet.

Offline tangrs

  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 229
  • Rating: +98/-0
    • View Profile
    • tangrs blog
Re: ELF Loader for Ndless
« Reply #4 on: December 15, 2011, 05:25:52 am »
BFLT does look interesting. I'll look into it.

Out of curiosity, why was BFLT never used?

Edit: You're right. ELF is pretty complex. Out of the week I spent on the project, 4-5 days were spent just reading and re-reading manuals and example code XD
« Last Edit: December 15, 2011, 05:29:34 am by tangrs »

Offline Lionel Debroux

  • LV11 Super Veteran (Next: 3000)
  • ***********
  • Posts: 2135
  • Rating: +290/-45
    • View Profile
    • TI-Chess Team
Re: ELF Loader for Ndless
« Reply #5 on: December 15, 2011, 06:49:22 am »
Quote
Out of curiosity, why was BFLT never used?
Well, I think it has been much more a matter of finding enough free time, than a matter of feasibility :)
Member of the TI-Chess Team.
Co-maintainer of GCC4TI (GCC4TI online documentation), TILP and TIEmu.
Co-admin of TI-Planet.

Offline tangrs

  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 229
  • Rating: +98/-0
    • View Profile
    • tangrs blog
Re: ELF Loader for Ndless
« Reply #6 on: December 17, 2011, 07:12:39 am »
Quote
Out of curiosity, why was BFLT never used?
Well, I think it has been much more a matter of finding enough free time, than a matter of feasibility :)

Ahhh, I see. That makes sense.

Offline ExtendeD

  • Project Author
  • LV8 Addict (Next: 1000)
  • *
  • Posts: 825
  • Rating: +167/-2
    • View Profile
Re: ELF Loader for Ndless
« Reply #7 on: December 18, 2011, 05:44:45 am »
Interesting tangrs.

But integrating to Ndless an ELF loader seems to be a bit overkilled just to support static initializers.
It's true that this would be annoying for initialization of arrays, but there's a simple workaround for this with nl_relocdata().
Ndless.me with the finest TI-Nspire programs

Offline tangrs

  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 229
  • Rating: +98/-0
    • View Profile
    • tangrs blog
Re: ELF Loader for Ndless
« Reply #8 on: December 18, 2011, 06:37:53 am »
Interesting tangrs.

But integrating to Ndless an ELF loader seems to be a bit overkilled just to support static initializers.
It's true that this would be annoying for initialization of arrays, but there's a simple workaround for this with nl_relocdata().

Oh, I know for c there are workarounds easily but I'm also working on getting C++ code working on Ndless and vtables and the lot are a lot more difficult to relocate using workarounds. That was what the ELF loader was originally for haha.

Though I do agree it might be a bit of an overkill especially considering this is an embedded platform.

Thanks for the insights guys, you guys pointed out some things I overlooked :)

Offline ExtendeD

  • Project Author
  • LV8 Addict (Next: 1000)
  • *
  • Posts: 825
  • Rating: +167/-2
    • View Profile
Re: ELF Loader for Ndless
« Reply #9 on: December 18, 2011, 10:23:31 am »
I see. But you can probably keep your ELF loader in user space until you reach a stable C++ toolchain, I then may integrate both if you want.
Ndless.me with the finest TI-Nspire programs

Offline alberthrocks

  • Moderator
  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 876
  • Rating: +103/-10
    • View Profile
Re: ELF Loader for Ndless
« Reply #10 on: December 18, 2011, 11:43:27 am »
I think an ELF loader would be amazing :D It would allow C++ support, and (possibly) dynamic libraries! :D (Although I don't think dynamic libraries would be good on the Nspire - a bundled, static package is more portable - it still will be useful for some other aspects and the thought of it being possible is nice :)

On a somewhat related note, is the Newlib issue on the wiki still relevant? I see that we are including newlib in the API building...
Withgusto Networks Founder and Administrator
Main Server Status: http://withg.org/status/
Backup Server Status: Not available
Backup 2/MC Server Status: http://mc.withg.org/status/


Proud member of ClrHome!

Miss my old signature? Here it is!
Spoiler For Signature:
Alternate "New" IRC post notification bot (Newy) down? Go here to reset it! http://withg.org/albert/cpuhero/

Withgusto Networks Founder and Administrator
Main Server Status: http://withg.org/status/
Backup Server Status: Not available
Backup 2/MC Server Status: http://mc.withg.org/status/

Activity remains limited due to busyness from school et al. Sorry! :( Feel free to PM, email, or if you know me well enough, FB me if you have a question/concern. :)

Don't expect me to be online 24/7 until summer. Contact me via FB if you feel it's urgent.


Proud member of ClrHome!

Spoiler For "My Projects! :D":
Projects:

Computer/Web/IRC Projects:
C______c: 0% done (Doing planning and trying to not forget it :P)
A_____m: 40% done (Need to develop a sophisticated process queue, and a pretty web GUI)
AtomBot v3.0: 0% done (Planning stage, may do a litmus test of developer wants in the future)
IdeaFrenzy: 0% done (Planning and trying to not forget it :P)
wxWabbitemu: 40% done (NEED MOAR FEATURES :P)

Calculator Projects:
M__ C_____ (an A____ _____ clone): 0% done (Need to figure out physics and Axe)
C2I: 0% done (planning, checking the demand for it, and dreaming :P)

Offline tangrs

  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 229
  • Rating: +98/-0
    • View Profile
    • tangrs blog
Re: ELF Loader for Ndless
« Reply #11 on: December 18, 2011, 04:27:57 pm »
I see. But you can probably keep your ELF loader in user space until you reach a stable C++ toolchain, I then may integrate both if you want.

Thanks, that would be great!

I think an ELF loader would be amazing :D It would allow C++ support, and (possibly) dynamic libraries! :D (Although I don't think dynamic libraries would be good on the Nspire - a bundled, static package is more portable - it still will be useful for some other aspects and the thought of it being possible is nice :)

On a somewhat related note, is the Newlib issue on the wiki still relevant? I see that we are including newlib in the API building...

Haha, dynamic libraries would be pretty interesting but would get pretty damn messy XD

Offline ExtendeD

  • Project Author
  • LV8 Addict (Next: 1000)
  • *
  • Posts: 825
  • Rating: +167/-2
    • View Profile
Re: ELF Loader for Ndless
« Reply #12 on: December 19, 2011, 04:18:41 pm »
On a somewhat related note, is the Newlib issue on the wiki still relevant? I see that we are including newlib in the API building...

Yes It's still relevant.
Ndless.me with the finest TI-Nspire programs

Offline ExtendeD

  • Project Author
  • LV8 Addict (Next: 1000)
  • *
  • Posts: 825
  • Rating: +167/-2
    • View Profile
Re: ELF Loader for Ndless
« Reply #13 on: December 27, 2011, 04:02:35 am »
tangrs, I see on your blog that you actually have a C++ build chain that works more or less. Do C++ programs now runs with your ELF loader?
What are the "required functions" not implemented by Ndless that exception handling needs?
Ndless.me with the finest TI-Nspire programs

Offline tangrs

  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 229
  • Rating: +98/-0
    • View Profile
    • tangrs blog
Re: ELF Loader for Ndless
« Reply #14 on: December 27, 2011, 05:27:15 am »
tangrs, I see on your blog that you actually have a C++ build chain that works more or less. Do C++ programs now runs with your ELF loader?
What are the "required functions" not implemented by Ndless that exception handling needs?

Yes, the ELF loader loads most C++ programs. There's still more testing to do though but it works for the basic test programs I wrote.

The C++ buildchain is really a hit and miss kind of thing. I can't remember what kind of things the C++ compiler needed to handle exceptions - I remember that the linker was complaining about missing symbols related to C++ exceptions. I'll look into what exactly is needed for exceptions though.

Edit: Probably something like <a href="http://wiki.osdev.org/C%2B%2B_Exception_Support">this</a> would work.
« Last Edit: December 27, 2011, 05:43:45 am by tangrs »