Author Topic: A solution to HP connectivity problems  (Read 16859 times)

0 Members and 1 Guest are viewing this topic.

Offline Adriweb

  • Editor
  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1708
  • Rating: +229/-17
    • View Profile
    • TI-Planet.org
Re: A solution to HP connectivity problems
« Reply #15 on: October 29, 2013, 05:11:46 am »
Could this be asked to HP devs directly ?
It's not as (well, so I hope) if it's a big copyrighted part of the OS nor hacking-related or whatnot ... ?
My calculator programs
TI-Planet.org co-admin.
TI-Nspire Lua programming : Tutorials  |  API Documentation

Offline timwessman

  • LV3 Member (Next: 100)
  • ***
  • Posts: 94
  • Rating: +32/-0
    • View Profile
Re: A solution to HP connectivity problems
« Reply #16 on: October 29, 2013, 08:56:53 am »
but I'm outputting verbatim the data part of the packets produced by the calculator (or so I think, at least :D).

I'm guessing then that you haven't figured out our little "hack". Basically, PNG has no definition for a 16bit per pixel (color) encoding. Since the calculator is currently running using an RGBt 5551, you have 16 bits per pixel. However, there *is* a defined 16bit per pixel greyscale in the png standard. Since PNG is lossless...

Convert your values coming out from 5551 to a 8888 or similar and I suspect it will look a bit better. :-)


As for the "8 minutes to enumerate", I've never been able to see anything like this. An empty calculator usually takes a few seconds - even over wireless which is slower. How much stuff is in your system? How long does it take an empty calc?


// CCITT CRC-16 look-up table (X^16+X^12+X^5+1)
(this looks like a good place to look at:http://www.embeddedrelated.com/groups/msp430/show/29689.php)

Initialized the crc bytes to 0 at start (yeah - dumb, caught too late to mess with). Also, looks like it might be including the first byte, but using the "length" value which does not include the first 6. (so first 6 bytes are included and last 6 are not).

Code: [Select]
unsigned short crc = 0;
while (len--)
{
   crc = crc16_tab[(crc >> 8) ^ *buffer++] ^ (crc << 8);
}
« Last Edit: October 29, 2013, 09:28:33 am by timwessman »
TW

Although I work for the HP calculator group, the comments and opinions I post here are my own.

Offline Lionel Debroux

  • LV11 Super Veteran (Next: 3000)
  • ***********
  • Posts: 2135
  • Rating: +290/-45
    • View Profile
    • TI-Chess Team
Re: A solution to HP connectivity problems
« Reply #17 on: October 29, 2013, 10:20:52 am »
Quote
// CCITT CRC-16 look-up table (X^16+X^12+X^5+1)
(this looks like a good place to look at: http://www.embeddedrelated.com/groups/msp430/show/29689.php)

Initialized the crc bytes to 0 at start (yeah - dumb, caught too late to mess with). Also, looks like it might be including the first byte, but using the "length" value which does not include the first 6. (so first 6 bytes are included and last 6 are not).
Works much better this way, thanks a lot :)
With the array and CRC computation function that you point:
Code: [Select]
static void crc16_block(const uint8_t * buffer, uint32_t len)
{
uint16_t crc = 0;
while (len--)
{
   crc = ccitt_crc16_table[(crc >> 8) ^ *buffer++] ^ (crc << 8);
}
printf("crc16 is %" PRIX16 "\n", crc);
}

the winning call is:
Code: [Select]
crc16_block(array2+1, 0x10);
crc16_block(array3+1, 0x10);
crc16_block(array4+1, 0x10);

Output:
Code: [Select]
crc16 is 9D57
crc16 is 68B
crc16 is 703F
(which shows that the CRC is written as little-endian)

I wonder why the brute-forcer didn't find it, even with 0x0000 as initial value for the CRC computation (I had already tested that instead of 0xFFFF), since it looked from 0 to 12 as start offset and 0 to 48 as end offset.

Anyhow, thanks a lot again ;)
« Last Edit: October 29, 2013, 10:24:34 am by Lionel Debroux »
Member of the TI-Chess Team.
Co-maintainer of GCC4TI (GCC4TI online documentation), TILP and TIEmu.
Co-admin of TI-Planet.

Offline Lionel Debroux

  • LV11 Super Veteran (Next: 3000)
  • ***********
  • Posts: 2135
  • Rating: +290/-45
    • View Profile
    • TI-Chess Team
Re: A solution to HP connectivity problems
« Reply #18 on: October 29, 2013, 05:02:10 pm »
So... the files seem to be nailed, but I'm still wrong for the screenshots in at least three ways...

(* as you indicated above, there's a trick for encoding the 16-bit images, so now, we know why they look like odd grayscale pictures. I didn't envision having libpng decompress the images, convert the data myself, then having libpng recompress the images, but it can be done);

* do the the colors for the paletted 4-bit color PNGs need the same kind of conversion as 16-bit grayscale PNGs ? The colors remain wrong, see
critor and others noted that this output looks like images with reversed blue and red colors. Interpreting 5551 as 565 probably wouldn't wreck the colors so much ?

* AFAICS in the test program, using the same CRC function for computing the screenshot data's CRC, and trying to match the result against the data at offset 6-7 (instead of 8-9 as in files), comes up empty, even when varying start and end (manually and programmatically). The best I could find was a match with reverse endianness, when computing between offset 8 and end of data, but that does not make sense.
In the short term, I can (and do) just ignore the CRC for screenshots, but it means that I am (and therefore, the code is) missing something.
« Last Edit: October 29, 2013, 05:07:11 pm by Lionel Debroux »
Member of the TI-Chess Team.
Co-maintainer of GCC4TI (GCC4TI online documentation), TILP and TIEmu.
Co-admin of TI-Planet.

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55943
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: A solution to HP connectivity problems
« Reply #19 on: October 30, 2013, 04:55:55 pm »
As for the "8 minutes to enumerate", I've never been able to see anything like this. An empty calculator usually takes a few seconds - even over wireless which is slower. How much stuff is in your system? How long does it take an empty calc?
I have about 110 KB of stuff on my calc. With a nearly empty calc it only takes a few seconds.

Intel Core i7 (first gen) 2.80 GHz
8 GB of RAM
1 TB hard drive (50 GB free) + two externals for a total of 6 TB
USB 2.0 ports
Windows 7 Ultimate 64 bits

Offline Lionel Debroux

  • LV11 Super Veteran (Next: 3000)
  • ***********
  • Posts: 2135
  • Rating: +290/-45
    • View Profile
    • TI-Chess Team
Re: A solution to HP connectivity problems
« Reply #20 on: October 30, 2013, 05:20:12 pm »
I'm not making progress on the screenshots' CRC, but making progress on other areas:
* backup is now fully implemented (it already was earlier) and gets the CRC right (which is new). In the test program, there are silly charset conversion issues (with e.g. the theta variable, whose UTF-16 LE name is 0xB8 0x03), but I don't plan on addressing them in the test program: this job is better done by clients of the library with proper Unicode handling (with libicu, Qt, whatever);
* the procedure for sending files to the calculator is nailed, thanks to critor's dumps of the HP Connectivity Kit <-> calc interaction. Sending a first virtual packet without payload, then another virtual packet with data payload, doesn't seem to be necessary, as creating a file through the "Content" tab of the connectivity kit does create+fill the file in one fell swoop - therefore, I plan on implementing only the combined create+fill. critor's dumps also show that for computer -> calc transfers, the very first byte in the packet is getting incremented (0x00 -> 0xFE -> 0x00 -> ..., that also indirectly confirms the fact that packets starting with 0xFF are sort of "ready" acknowledgments, I'm ignoring them). That's unlike calc -> computer transfers, where the first byte stays 0.

As for transfer slowness: during the calc -> computer backup process [EDIT: made with the HP Connectivity Kit], the timestamps in critor's USB packet dumps show intervals of between 5 and 20 ms between each 64-byte packet, which mechanically limits the transfer rate to between ~3 KB/s and ~12 KB/s... but still, 8 minutes for 110 KB of data indicates a problem somewhere, as even at 3 KB/s, it should last less than 40s.
Does the Windows HID stack (whose reliability seems to be pitiful, as shown by the horrible behaviour when printing traces to the horrendously slow Windows terminal) lose packets so easily that calc -> computer backups cannot be made unconditionally at 5 ms interval ?
BTW, 3-12 KB/s is a very far cry from the transfer rate in firmware upgrade (MSD-like) mode (EDIT: using the "official" reflashing tool; libhpcalcs does not tackle that), whose speed is good, faster than that of the Nspire family, despite having more data to transfer :(
[EDIT: I do not have any measurements of the performance of libhpcalcs wrt. the official connectivity kit. If the calculator is the one setting the pace for calc -> computer transfers, which I believe due to my understanding of how USB interrupt mode works, there shouldn't be a difference, as the speed will be limited by the calculator for not overwhelming poor little Windows.]

The libhpcalcs + test_hpcalcs source code (written in C11) now exceeds 2500 SLOC, as counted by `sloccount`. The raw line count is above 4000 lines, as all main headers now have some documentation for all APIs and data structures, there are some comments... and the license blurb repeated at the beginning of all files takes its toll as well.

[EDITs for clarification.]
« Last Edit: October 30, 2013, 05:32:33 pm by Lionel Debroux »
Member of the TI-Chess Team.
Co-maintainer of GCC4TI (GCC4TI online documentation), TILP and TIEmu.
Co-admin of TI-Planet.

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55943
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: A solution to HP connectivity problems
« Reply #21 on: October 30, 2013, 05:23:17 pm »
Nice to see that your method transfers faster. :) Now if only there was a faster way... I know the PRIZM is about 22 KB/sec but there is a small delay when killing the connection, where the PRIZM tries to convert the received data for calc use.

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55943
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: A solution to HP connectivity problems
« Reply #22 on: November 09, 2013, 07:25:24 pm »
UPDATE: The time it takes for the official software to read the calc content and display it in the nav tree seems random. Now after 5 minutes it still didn't list anything and it only just started doing so. I wonder if it has to do with the fact I am listening to music via iTunes at the same time, though. I still have 110 KB of programs.