IRIX Binary Compatibility, Part 2by Emmanuel Dreyfus
Unix Program Startup
Now that our kernel is able to distinguish the difference between IRIX binaries and other programs, we need to arrange the program environment so that the IRIX binary is able to start up. (See Part 1 in this series for more on this.)
Generally speaking, Unix kernels have to communicate a few things to user programs in order to start them up. This includes the program's arguments and environment, and for dynamic binaries, the ELF auxiliary table, which is used by the dynamic linker to learn how to link the program. All this information is transmitted to the user program through the CPU registers and the stack.
If this information is corrupted, static binaries are still likely to work, but they will lose their arguments and environment. On the other hand, dynamic executables will not start at all if the ELF auxiliary table is screwed, because the dynamic linker will not be able to link them.
Therefore, it is a good idea to start with a simple static binary. We will use
sed(1) here. When we run it using the NetBSD native function to set up
the stack and CPU register, it is able to start up. Then it gets a SIGSYS
signal and it dumps core on the first system call, because our system call
table for IRIX binaries is still empty. It is possible to check what the
missing system call is with the
ktrace(1) command on NetBSD:
$ ktrace -di /emul/irix/bin/sed /etc/passwd Bad system call (core dumped) $ kdump 1209 ktrace EMUL "netbsd" 1209 ktrace CALL execve(0x7fffea5f,0x7fffe99c,0x7fffe9a4) 1209 ktrace NAMI "/emul/irix/bin/sed" 1209 sed EMUL "irix o32" 1209 sed RET execve 0 1209 sed CALL #4 (unimplemented write) 1209 sed PSIG SIGSYS
Most of the system calls first used by
/bin/sed are plain SVR4, so it
was easy to emulate them: just copy the system call definition from
sys/irix/syscall.master, issue a make to refresh
the files generated from
syscall.master, rebuild a kernel, reboot, and retry.
Within a few minutes, it was easy to get IRIX's
/bin/sed nearly working. The
next problem was to have it take its arguments correctly.
Setting Up the Stack for Program Startup
For static binaries, the stack is used to transmit arguments and the environment to the user program. The way it should be done is documented in the SVR4 ABI MIPS processor supplement.
In This Series
IRIX Binary Compatibility, Part 6
IRIX Binary Compatibility, Part 5
IRIX Binary Compatibility, Part 4
IRIX Binary Compatibility, Part 3
IRIX Binary Compatibility, Part 1
The NetBSD kernel uses a function pointed to by the
es_copyargs field of
execsw to set up the program stack on startup. Because
ports conform to the SVR4 ABI, we could have expected the NetBSD
version of this function (
to just work with IRIX binaries. Unfortunately, this is not true. Using the
elf32_copyargs function with static o32 IRIX binary such as
showed weird behavior with the way argument read: sometimes
was reading the arguments correctly, sometimes it was not. The behavior
was dependent upon the argument length. This suggested that something in
the stack had to be aligned on a particular boundary; with some argument
lengths it was being aligned, and with others it was not.
I already had to face this kind of situation when working on Linux/PowerPC
binary compatibility on NetBSD (read the whole story). However, the situation is different here: IRIX is a closed source proprietary OS; therefore it is not possible to grab kernel sources and look at the way the IRIX kernel sets up the stack. Worse,
because I was not able to build static binaries, it was impossible to make
a static test program that dumped the stack and displayed
envp to check what was wrong in the way the stack was set up.
The solution was to use
gdb we can run
/bin/sed on IRIX, set a breakpoint at the beginning of the program and then examine the stack.
The first thing to know is the program startup address. This information
can be obtained using objdump on IRIX's sed:
$ objdump -f /bin/sed /bin/sed: file format elf32-bigmips architecture: mips:3000, flags 0x00000102: EXEC_P, D_PAGED start address 0x100000c0
Then we can start
gdb and set our breakpoint. It seems that it is not
possible to break on the program's first instruction, but we can break on
the second instruction. On the MIPS, all instructions are four bytes long,
hence the second instruction is four bytes away from the program's start
0x100000c0 + 0x4 = 0x100000c4:
$ gdb /bin/sed (gdb) b *0x100000c4 Breakpoint 1 at 0x100000c4 (gdb) run aa aaa Starting program: ./sed aa aaa Breakpoint 1, 0x100000c4 in ?? () (gdb) x/16wx $sp 0x7fff2fa0: 0x00000003 0x7fff3000 0x7fff3027 0x7fff302a 0x7fff2fb0: 0x00000000 0x7fff302e 0x7fff3057 0x7fff306a 0x7fff2fc0: 0x7fff30a7 0x7fff30b4 0x7fff30be 0x7fff30d4 0x7fff2fd0: 0x7fff30e1 0x7fff30f6 0x7fff3109 0x7fff3113
In this dump, we recognize a standard startup stack layout--the
argc value (three arguments: '
aa', and '
aaa'), followed by the
argv array (a NULL terminated array of pointers to the argument strings), and then the
envp array (a NULL terminated array of pointers to the environment
strings). There are a lot of environment strings here, hence we do not see
the trailing NULL here, it is a bit farther in the stack dump.
It is possible to dig up the value of an argument with its address:
(gdb) x/s 0x7fff3000 0x7fffea60: "/bin/sed" (gdb) x/s 0x7fff3027 0x7fffea72: "aa"
Dumping the stack with various arguments to
/bin/sed, it is possible to
discover that, for an IRIX binary, the
argv must be aligned on a 16-byte boundary. The IRIX kernel sets the stack that way, and IRIX binaries depend
on this particular layout.
It was not possible to modify
elf32_copyargs() to implement this particular
behavior because it is also used by native NetBSD binaries. Hence, the
solution was to duplicate what was done in
elf32_copyargs() in an
irix_copyargs() function, which can be found in
irix_copyargs() function just does
elf32_copyargs() job and it enforces the 16-byte alignment of
irix_copyargs() function must be used in the
of the struct
execsw for IRIX in
With this adjustment, static IRIX binaries were able to read their arguments and environment correctly.
Pages: 1, 2