Linux Compatibility on BSD for the PPC Platform: Part 2
Pages: 1, 2, 3, 4

By looking at Linux kernel source file linux/fs/binfmt_elf.h, in the create_elf_tables() function, we can learn how the table should be laid out so Linux's ld.so works. The job is nearly the same on the PowerPC and Alpha platforms, so we can use the NetBSD/Alpha version again. The PowerPC platform just has a special trick: The ELF auxiliary table must also be aligned on a 16-byte boundary. This is a bit difficult to understand in the Linux kernel sources, but we can see comments about this in linux/fs/binfmt_elf.h, and also in the shove_aux_table() function, which is in linux/arch/ppc/kernel/process.c.

We therefore have to add another LINUX_SHIFT conditional before writing the ELF auxiliary table:

#ifdef LINUX_SHIFT
/*
 * From Linux's arch/ppc/kernel/process.c:shove_aux_table().
 * GNU ld.so expects the ELF auxiliary table to start on a 
 * 16 bytes boundary on the PowerPC.
 */
   (unsigned long) stack = 
       ((unsigned long) stack + LINUX_SHIFT) & ~LINUX_SHIFT;
#endif

Finding out where ld.so really expects the table was fairly difficult: When dynamic linking does not work, it is impossible to even output a string from the program, so stack-dumping a dynamically linked program is not an option. I had to blindly try a few different alignments and test the result before I managed to get it to work.

When the ELF Auxiliary table is correctly set up onto the stack, dynamically linked Linux binaries should link and run. Using GNU ld-1.7.0.so, everything was fine: ld.so got its argument, and the program was able to run (and then crash, but this was actually caused by another bug we will study in the next section). However, when upgrading to GNU ld-2.1.3.so, we discovered a new problem: Dynamically linked executables did not get their arguments anymore. This problem will be studied in a later section. In the next section, we will focus on other bug-crashing Linux binaries dynamically linked with GNU ld-1.7.0.so.

A mmap() fix

At this point, it is obvious that ld.so was successfully launched: The kernel trace did show attempts to open() and mmap() files such as /emul/lib/libc.so.6. But the mmap() call failed.

mmap() is used to remap physical memory and files into a process's virtual address space. It is widely used when linking shared libraries because the library code doesn't have to be loaded into the process memory. mmap() is used to map the shared library file from the disk to the virtual address space of several user processes. When a process uses the library, it is loaded into physical memory by the virtual memory subsystem, but it will never be loaded twice, because other processes share the library through their virtual memory mappings. The library is loaded once and used several times. If you need more information about the mmap() system call, take a look at the mmap (2) man page.

To debug this kind of problem, it is useful to make a small test program that uses the bogus system call. Here is a simple mmap() tester:

/*
 * mmap.c -- mmap() tester
 */
#include <sys/types.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <stdlib.h>
#include <stdio.h>

int main (int argc, char **argv) {
  int fd;
  char* ptr;

  fd = open ("/etc/passwd", O_RDONLY, 0);
  if (fd < 0) {
    printf ("open failed\n");
    exit(-1);
  }

  (void*)ptr = mmap (NULL, 512, PROT_READ,MAP_PRIVATE|MAP_FILE, fd, 0);

  if (ptr == NULL) {
    perror ("mmap failed");
    exit(-1);
  }

  printf ("%c-%c-%c-%c\n", ptr[0], ptr[1], ptr[2], ptr[3]);

  return 0;
}

Using this program, it is clear the problem is caused by our mmap() emulation, and nothing else. After some investigation, we found the problem was caused by the size of the offset argument to mmap(). This argument is 32 bits long on a PowerPC Linux system, and it is 64 bits long on a PowerPC NetBSD system. The result is that when a Linux executable made a mmap() system call, NetBSD used for offset the actual argument given by the Linux executable, plus the next 32 bits of data on the stack.

Adding a wrapper function that correctly handles the offset argument and transfers control to linux_sys_mmap() fixes the problem. This wrapper function is defined in sys/compat/linux/arch/powerpc/linux_mmap_powerpc.c. Obviously, this is not very clean design, and it would be better to define a linux_off_t in architecture-dependent linux_mmap.h files, and then use them in the architecture-independent linux_sys_mmap() function.

After this mmap() fix, we are able to run dynamically linked programs such as the stack dumper or the argument printer. Everything is fine with ld-1.7.0.so (which is available with Linux's glibc-1), but upgrading to ld-2.1.3.so, which comes with glibc-2, breaks argument passing for dynamic executables.

Pages: 1, 2, 3, 4

Next Page