diff --git a/content/2014/03/ctf-armeb-debugging.html b/content/2014/03/ctf-armeb-debugging.html new file mode 100644 index 0000000..a59630b --- /dev/null +++ b/content/2014/03/ctf-armeb-debugging.html @@ -0,0 +1,118 @@ +--- +title: "CTF + ARMeb + debugging" +description: > + CTF + ARMeb + debugging +created: !!timestamp '2014-03-05' +time: 7:21 PM +tags: + - ctf + - debugging +--- + +I've been working on making the AVILA board work again with FreeBSD. +Thanks to Jim from Netgate for sending me a board to do this work. + +I still have a pending patch waiting to go through bde to fix an +unaligned off_t store which gets things farther, but with the patch I'm +getting a: `panic: vm_page_alloc: page 0xc0805db0 is wired` shortly after +the machine launches the daemons. + +I did work to get cross gdb working for armeb (committed in r261787 and +r261788), but that didn't help as there is no kernel gdb support on +armeb. As I'm doing this debugging over the network, I can't dump a +core. + +I didn't feel like hand decoding a struct vm_page, so I thought of other +methods, and one way is to use CTF to parse the data type and decode the +data. I know python and ctypes, so I decided to wrap libctf and see +what I could do. + +Getting the initial python wrapper working was easy, but my initial test +data was the kernel on my amd64 box that I am developing on. Now I +needed to use real armeb CTF data. I point it to my kernel, and I get: +"`File uses more recent ELF version than libctf`". Ok, extract the CTF +data from the kernel (ctf data is stored in a section named `.SUNW_ctf`) +and work on that directly: +``` +$ objcopy -O binary --set-section-flags optfiles=load,alloc -j .SUNW_ctf /tftpboot/kernel.avila.avila /dev/null +objcopy: /tftpboot/kernel.avila.avila: File format not recognized +``` + +Well, ok, that's not too surprising since it's an ARMEB binary, lets try: +``` +$ /usr/obj/arm.armeb/usr/src.avila/tmp/usr/bin/objcopy -O binary --set-section-flags optfiles=load,alloc -j .SUNW_ctf /tftpboot/kernel.avila.avila /tmp/test.avila.ctf +$ ls -l /tmp/test.avila.ctf +-rwxr-xr-x 1 jmg wheel 0 Mar 5 17:59 /tmp/test.avila.ctf +``` + +Hmm, that didn't work too well, ok, lets just use dd to extract the data +using info from `objdump -x`. + +Ok, now that I've done that, I get: +``` +ValueError: '/tmp/avila.ctf': File is not in CTF or ELF format +``` + +Hmm, why is that? Well, it turns out that the endian of the CTF data +is wrong. The magic is `cf f1`, but the magic on amd64 is f1 cf, it's +endian swapped. That's annoying. After spending some time trying to +build an cross shared version of libctf, I find that it has the same +issue. + +After a bit of looking around, I discover the CTF can only ever read +native endianness, but `ctfmerge` has a magic option that will write out +endian swapped data if necessary depending upon the ELF file it's +putting in. This means that the CTF data in an armeb object file will +be different depending upon the endian you compiled it on, so the object +file isn't cross compatible. But, this does mean that the data in the +object files will be readable by libctf, just not the data written into +the kernel. + +So, I create a sacrificial amd64 binary: +``` +$ echo 'int main() {}' | cc -o /tmp/avila2.ctf -x c - +``` + +And use `ctfmerge` to put the data in it: +``` +$ ctfmerge -L fldkj -o /tmp/avila2.ctf /usr/obj/arm.armeb/usr/src.avila/sys/AVILA/*.o +``` + +and again use `dd` to extract the `.SUNW_ctf` section into a separate file. + +With all this work, I finally have the CTF data in a format that libctf +can parse, so, I try to parse some data. Now the interesting thing is +that the CTF data does encode sizes of integers, but it uses the native +arch's pointer sizes for `CTF_K_POINTER` types, which means that pointers +appear to be 8 bytes in size instead of the correct 4 bytes. A little +more hacking on the ctf.py script to force all pointers to be 4 bytes, +and a little help to convert ddb output to a string and finally, I have +a dump of the struct vm_page that I was trying to get all along: +``` +{'act_count': '\x00', + 'aflags': '\x00', + 'busy_lock': 1, + 'dirty': '\xff', + 'flags': 0, + 'hold_count': 0, + 'listq': {'tqe_next': 0xc0805e00, 'tqe_prev': 0xc06d18a0}, + 'md': {'pv_kva': 3235856384, + 'pv_list': {'tqh_first': 0x0, 'tqh_last': 0xc0805de0}, + 'pv_memattr': '\x00', + 'pvh_attrs': 0}, + 'object': 0xc06d1878, + 'oflags': '\x04', + 'order': '\t', + 'phys_addr': 17776640, + 'pindex': 3572, + 'plinks': {'memguard': {'p': 0, 'v': 3228376932}, + 'q': {'tqe_next': 0x0, 'tqe_prev': 0xc06d1f64}, + 's': {'pv': 0xc06d1f64, 'ss': {'sle_next': 0x0}}}, + 'pool': '\x00', + 'queue': '\xff', + 'segind': '\x01', + 'valid': '\xff', + 'wire_count': 1} +``` + +So, the above was produced w/ the final [ctf.py](https://www.funkthat.com/~jmg/ctf.py) script.