This article explains the mechanism of finding signatures automatically with the "Signature finder" (finsig/gensig). The original text was posted on 31-09-2008 by fudgey in the CHDK forum: Hi... documentation I could find about the signature finder in the tools directory was sort of scarce, so I wrote this while fiddling with it a bit. I had my mind set on adding new features, not porting new cameras
Signature finder (finsig/gensig) usage
1) Put all firmware dumps of every camera into the CHDK source tree. Each firmware version needs to be called PRIMARY.BIN and be located in it's own subdirectory, such as platform/a570/sub/100e/PRIMARY.BIN for the a570is 1.00e dump.
Once you have done that, signatures will be automatically searched for during the build process.
You should try batch-build at this point, because it's likely that some platforms will not build due to missing stubs. This is because your dumps may not be perfect (incomplete or not starting from the correct address) or because someone changed something in CHDK source tree that breaks building with firmware dumps in place. If you have a bad firmware dump, fix it (removing garbage from the start is often enough; compare to dumps of other similar cameras and you'll see how it should start) or remove it from the source tree (to skip signature search for this platform), or just don't build for that platform (commenting it out in the Makefile works).
Tweak until the build process goes through cleanly.
2) Go to CHDK source tree tools/ directory. See readme.txt. At the time of writing this, it says:
sig_ref_vxworks_1.bin : should contain a610-100e primary sig_ref_vxworks_2.bin : should contain a710-100a primary sig_ref_vxworks_3.bin : should contain a570-100e primary sig_ref_dryos_1.bin : should contain a720-100c primary
Copy those firmware dumps to the tools/ directory with those names.
3) See sig_ref_*.txt. These are simple lists with function names and their entry points for each of these four reference firmware dumps.
Using a disassembler, find entry points to your new functions for each of these four reference firmwares and add them to the end of those files.
NOTE
You can optionally specify the length of the function in words (=instructions) after the entry point. If the function is shorter than 32 instructions, and doesn't end in a simple RET, specifying this should give better matches.
4) Build.
If all goes perfectly, your new stubs are now found as NSTUBs in platform/*/sub/*/stubs_entry.S files among all other the automatically generated stubs.
It's likely that some platforms will not build. The signatures may not be good enough to find your function from all cameras. Your function may not even exist on all cameras, or versions of it may be too different to match the signatures.
To get past an error, you can of course delete the unlucky PRIMARY.BIN or skip building for this particular firmware, but that won't fix the fact that your new signature breaks building for this platform if the firmware dump is in it's place.
What you should do is manually find the entry point for this platform and add it as an NHSTUB to stubs_entry_2.S. If it is not found, you need to add it as a null stub (a stub that has a correct name, but which points to a part of code which will just return without doing anything) to stubs_entry_2.S to keep 'make' happy.
The problem with this approach is that sometimes you may not want to bind this function to a nullstub. Instead, you may want to implement a dummy function in lib.c (only for platforms that don't have the real firmware function) that returns something more sane than a null function. But you can't add that dummy function if the name is taken by the stub!
The alternative is not to add your new function signatures to CHDK svn, but instead provide NHSTUBs in stubs_entry_2.S for all platforms, with dummy stubs or lib.c dummy functions for all unsupported platforms. This doesn't help people porting for a new camera, though (quite the contrary since there's more work to be done in porting).
NOTE
After building with the sig_ref* binaries in place, it's a good idea to compare the generated stubs_entry.S files and signatures_<OS>.h with what is in SVN. Ideally, the ones you generate should be identical to those in SVN, except for any entry points you've added. If they aren't, it may point to a problem with your dumps, or possibly that someone has checked in the generated files without updating everything else.
Adding a new reference firmware
If the function exists but isn't found, you could find it manually and add this firmware as another new reference firmware, so that this signature would be created and matched. I have never done this, but if I'm not mistaken this is how it's done:
1) Create a new signature input file (for example tools/sig_ref_vxworks_4.txt if that's not taken and if this camera was a vxworks camera) containing your function name and entry point, just like in the other txt files. There is no need to add any other entry points in this file, but for the sake of porting new cameras, it would probably be a good idea to add functions that are listed in stubs_entry_2.S, because those are the ones that weren't found automatically using the old signatures.
2) Copy the firmware dump where gensig can find it (in our example, tools/sig_ref_vxworks_4.bin).
3) Modify tools/Makefile in the numerous occasions where additions are required ('clean:' and 'signatures_vxworks.h:' need to be modified in our example, and 'sig_ref_vxworks_4.bin:' must be added).
4) Update tools/readme.txt.
And to continue the already lengthy rant, I tested this with EnterToCompensationEVF and ExitFromCompensationEVF. I had already manually determined entry points for 35 firmwares from disassemblies. Two of those failed to build with firmwares in place in ways I didn't know how to fix quickly.
Out of the remaining 33, EnterToCompensationEVF was correctly found in 23 dumps and ExitFromCompensationEVF 16 dumps. Four of the successful ones were reference firmwares. So, we got success ratios of 19/29 and 12/29 (66 % and 41 %). It _is_ possible that some of the incorrectly found addresses would work, since the same function can be entered in more than one way. But it is also possible that they wouldn't work properly (see explanation below (1)). I'm also assuming that the entry points I found manually are correct (and the ones that one could expect the signature finder to be able to find).
A candidate for ExitFromCompensationEVF was found in all firmwares, but 17 of 29 (59 %) were incorrect. For two of the incorrect ones, finsig gave a correct alternative (commented out in stubs_entry.S). EnterToCompensationEVF was not found in 6 firmwares, meaning only 4 out of 29 (14 % were) incorrectly found.
Some of this could be handled by selecting more reference firmwares (to better represent all incarnations of the functions and their calling mechanisms). This may still not be enough (unless finsig is really smart and follows branches?).
—————
(1) The way these functions are laid out in the firmware can be divided into groups that look similar (this is one reason why we need more than one reference firmware). When finding functions manually using strings, some have strings located just before the function (e.g. a570is), and a good entry point is right after the string, on an instruction that loads a pointer to that string for debug purposes.
On many other firmwares, there are blocks of strings with no code in between. Near those blocks there are lists that map these strings to code.
But these lists often point to a very short code wrapper, which just branches to the real entry point of the function. So - unless finsig is smarter than I think it is - it's not a good idea to use the listed address as an entry point for a reference firmware, because that will result in a bad signature. It can be used as an entry point for a manually found function, altough it will run slower by a few instruction cycles. S3IS is an exemple for this category.
For some firmwares this short code pointed from the list actually does something. It may be the actual function, but often it is just some simple input or output variable manipulation that only takes a few instructions before or after a branch to where most of the function code resides. This short code snippet may still be something that is necessary in order to make the function work correctly (it's possible that two functions share code this way, for instance?).
These kinds of layouts are more difficult to find automatically (unless finsig is smart and follows branches), since the actual code may be only a few instructions long and the real functionality happens elsewhere. So, using this kind of a firmware as a reference is not going to work (a very short signature would create lots and lots of matches, and a signature extending beyond a function is just plain wrong), but not using one may result in finsig creating a stub that points to the real code without the (possibly cruicial) wrapper.