Cross Compile Madness.

So I pushed that pax-utils.

  • Improved Makefile handling. (vapier)
  • QA_TEXTREL/QA_EXECSTACK/QA_WX_LOAD exempt filtering flags. (kevquinn)
  • Handle versioned symbols correctly on unstripped ELF files when using the -s flag.
  • Do not assume it is ok to read from stdin if the -l -p flags were given. (solar)

Now it’s time to get back to some cross compiling goodness. Well one thing I know about myself is anything I do I tend todo in excess. So I figure why build 1 cross compiler when you can build them all. Why merge into 1 place when you can reasonable test cross compiling for all arches.. Well that’s just what I did and am doing.

Little script I’m using for just this task..

#!/bin/sh
# Copyright 2006 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: <solar@gentoo.org>

CFLAGS="-Os -pipe"
CBUILD=$(portageq envvar CHOST)
CXXFLAGS="${CFLAGS}"

export PYTHON_DONTCOMPILE=1

USE_SAVE="${USE}"
if [ ! -x /usr/bin/qlist ]; then
        echo "please emerge portage-utils"
        exit 1
fi

gcc_executables=$(qlist -oeCI gcc| sort -u | grep ^cross- | sed s/cross-//g | tr / -)

opts="$@"
for x in "$@"; do
        if [[ $x == "--libc" ]]; then
                opts="${opts/${x}/}"
                DO_LIBC=1
                do_libc=1
        fi
        if [[ "${lastopt}" == "--target" ]]; then
                ctarget_only="${x}"
                opts="${opts/${x}/}"
                opts="${opts/${lastopt}/}"
        fi
        lastopt=${x}
done

function cross_env_update() {
        mkdir -p ${ROOT}/etc
        for lpath in /usr/local/lib /usr/${CTARGET}/lib /usr/lib/gcc/${CTARGET}/$(${CTARGET}-gcc -dumpversion) /usr/lib/libstdc++-v3/; do
                echo "${lpath}" >> ${ROOT}/etc/ld.so.conf
        done
}

function cmerge() {
        CHOST=${1}
        shift

        export CTARGET=${CHOST}
        ARCH=$(echo ${CTARGET} | cut -d '-' -f 1)
        CC=${CTARGET}-gcc
        CXX=${CTARGET}-g++
        PKGDIR=${HOME}/packages/${CHOST}
        ROOT="${HOME}/ROOT/${CHOST}"
        LDFLAGS="-L${ROOT}/lib -L${ROOT}/usr/lib"

        do_libc=${DO_LIBC}
        unset ELIBC
        case ${CTARGET} in
                *-linux-gnu)    ELIBC=glibc;;
                *-linux-uclibc) ELIBC=uclibc;;
                avr) ELIBC=avr-libc;;
        esac
        export ELIBC
        [[ $ELIBC == "" ]] && do_libc=0

        USE="${ARCH} ${USE_SAVE}"
        case ${ARCH} in
                armeb) ARCH=arm;;
                s390x) ARCH=s390;;
                mips*) ARCH=mips;;
                powerpc64) ARCH=ppc64;;
                sparc*) ARCH=sparc;;
                powerpc) ARCH=ppc;;
                hppa*) ARCH=hppa;;
                sh4*|sheb) ARCH=sh;;
                x86_64) ARCH=amd64;;
                avr|ee|iop|cris) ARCH="x86";;
                i?86) ARCH=x86;;
        esac
        ACCEPT_KEYWORDS="${ARCH}"
        [[ "${unstable}" != "" ]] && ACCEPT_KEYWORDS="${ACCEPT_KEYWORDS}  ~${ARCH}"

        USE="-* ${ARCH} ${USE} elibc_${ELIBC} multicall make-symlinks"

        export ARCH ACCEPT_KEYWORDS CFLAGS CXXFLAGS LDFLAGS ROOT CBUILD CHOST CTARGET CC CXX PKGDIR USE

        mkdir -p ${ROOT}/etc
        if [[ $ELIBC != "" ]]; then
                if [ -e ${ROOT}/etc/ld.so.conf ]; then
                        if [ "$(md5sum ${ROOT}/etc/ld.so.conf|awk '{print $1}')" == "b18efa8c9f95b6aecc0974c0f54d8bb9" ]; then
                                touch ${ROOT}/etc/ld.so.conf
                                cross_env_update
                        fi
                else
                        cross_env_update
                fi
        fi

        if [[ "$do_libc" == 1 ]]; then
                emerge -b cross-${CHOST}/${ELIBC} ${opts}
        else
                emerge -b ${opts}
        fi

        unset ARCH ACCEPT_KEYWORDS LDFLAGS ROOT CHOST CTARGET CC CXX PKGDIR USE ELIBC
}

if [[ "$1" == "" ]]; then
        echo "$0: <opts> <emerge opts>"
        echo "   --libc"
        echo "   --target <chost>"
        echo "   --libc  --target i386-gentoo-linux-uclibc busybox -pv"
        exit 1
fi

for gnugcc in ${gcc_executables}; do
        [ -e /usr/bin/${gnugcc} ] || { echo bummer $gnugcc ; continue; }
        # include=$($gnugcc -print-file-name=)
        [[ $ctarget_only != "" ]] && [[ ${gnugcc/-gcc/} != $ctarget_only ]] && continue
        cmerge "${gnugcc/-gcc/}" "$@"
done

A blind run at building busybox and libc for just about every chost combo yields the following results. Anything marked N is something that failed. R means it installed of course.

tinderbox ~ # ./cmerge -qpv busybox
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/alpha-gentoo-linux-uclibc/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/alpha-unknown-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/arm-gentoo-linux-uclibc/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/arm-softfloat-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/arm-softfloat-linux-uclibc/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/arm-unknown-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/armeb-gentoo-linux-uclibc/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/armeb-softfloat-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/armeb-softfloat-linux-uclibc/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/armeb-unknown-linux-gnu/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/avr/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/cris-gentoo-linux-uclibc/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/ee/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/hppa-unknown-linux-gnu/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/hppa1.1-unknown-linux-gnu/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/hppa2.0-unknown-linux-gnu/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/hppa64-unknown-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/i386-gentoo-linux-uclibc/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/i386-pc-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/i686-gentoo-linux-uclibc/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/ia64-unknown-linux-gnu/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/iop/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/m68k-gentoo-linux-uclibc/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/m68k-unknown-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/mips-gentoo-linux-uclibc/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/mips-unknown-linux-gnu/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/mips64-unknown-linux-gnu/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/mips64el-unknown-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/mipsel-gentoo-linux-uclibc/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/mipsel-unknown-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/powerpc-gentoo-linux-uclibc/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/powerpc-softfloat-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/powerpc-softfloat-linux-uclibc/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/powerpc-unknown-linux-gnu/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/powerpc64-unknown-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/s390-ibm-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/s390x-ibm-linux-gnu/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/sh-gentoo-linux-uclibc/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/sh-unknown-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/sh4-gentoo-linux-uclibc/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/sh4-unknown-linux-gnu/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/sh4eb-gentoo-linux-uclibc/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/sh4eb-unknown-linux-gnu/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/sheb-gentoo-linux-uclibc/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/sheb-unknown-linux-gnu/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/sparc-gentoo-linux-uclibc/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/sparc-unknown-linux-gnu/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/sparc64-unknown-linux-gnu/ 
[ebuild  N    ] sys-apps/busybox-1.1.0 to /root/ROOT/x86_64-gentoo-linux-uclibc/ 
[ebuild   R   ] sys-apps/busybox-1.1.0 to /root/ROOT/x86_64-pc-linux-gnu/ 

Well that’s cool and all. But how to test stuff? Thats where qemu comes to the rescue.. Few mins of hacking and poof we come up with a little something something like so.

#!/bin/sh

pkg_genext2fs() {

        if ! type -p genext2fs > /dev/null; then
                echo "no genext2fs"
                return 1
        fi

        cd $1 || return 1

rom=/rom

        mkdir -p {usr/,}{s,}bin etc/init.d proc/self dev/shm root tmp ./${rom}

        chmod 700 root
        chmod 1777 tmp

################
cat <<EOF > etc/init.d/rcS 
#!/bin/sh
rom=$rom
mount -a
mount tmpfs \$rom -t tmpfs
mkdir -p \$rom/proc \$rom/dev \$rom/ro \$rom/tmp
mount -o bind /dev \$rom/dev
mount -o bind /proc \$rom/proc
busybox cp -a bin etc lib root sbin usr \$rom/
chmod 1777 \$rom/tmp
pivot_root \$rom/ \$rom/ro
EOF
################

        chmod +x etc/init.d/rcS

        if [ ! -L proc/self/exe ]; then
                cd proc/self
                ln -s /bin/busybox exe

                if [ -e /proc/self/fd ] ; then
                        ln -fs /proc/self/fd fd 2> /dev/null
                        ln -fs fd/0 stdin
                        ln -fs fd/1 stdout
                        ln -fs fd/2 stderr
                fi
                cd ../../
        fi
        cd etc
        ln -sf ../proc/mounts mtab
        grep -v ^# /etc/protocols > protocols

        cd ..

        cd dev
        ln -sf /proc/kcore core

        # ttys
        for i in `seq 0 9`; do
                [ ! -e tty$i ] && mknod tty$i c 4 $i
        done
        [ ! -e tty ] && (mknod -m 666 tty c 5 0 2> /dev/null || echo cant mknod for tty)

        [ ! -e null ] &&  (mknod -m 666 null c 1 3 2> /dev/null || echo cant mknod for null)
        [ ! -e zero ] && (mknod -m 666 zero c 1 5 2> /dev/null || echo cant mknod for zero)
        [ ! -e urandom ] && (mknod -m 644 urandom c 1 9 2> /dev/null || echo cant mknod for urandom)
        [ ! -e random ] && mknod random c 1 8
        [ ! -e console ] && mknod console c 5 1
        [ ! -e full ] && mknod full c 1 7
        [ ! -e kmem ] && mknod kmem c 1 2
        [ ! -e mem ] && mknod mem c 1 1
        [ ! -e port ] && mknod port c 1 4

        # IDE devs
                [ ! -e hda ] && mknod hda b 3 0
        [ ! -e hdb ] && mknod hdb b 3 64
        [ ! -e hdc ] && mknod hdc b 22 0
        [ ! -e hdd ] && mknod hdd b 22 64

        # loop devs
        for i in `seq 0 7`; do [ ! -e loop$i ] && mknod loop$i b 7 $i; done

        # ram devs
        for i in `seq 0 9`; do [ ! -e  ram$i ] && mknod ram$i b 1 $i ; done
        [ ! -L ram ] && ln -s ram1 ram

        # virtual console screen devs
        for i in `seq 0 9`; do [ ! -e vcs$i ] && mknod vcs$i b 7 $i; done

        chown root:tty tty 2> /dev/null
        chown root:sys null zero 2> /dev/null
        cd -

################
## /etc/fstab ##
################
cat <<EOF > etc/fstab
#/dev/root      /               ro,defaults  defaults 0 0
proc            /proc           proc    defaults 0 0
#none           /dev            devfs   defaults 0 0
devpts          /dev/pts        devpts  defaults,gid=5,mode=620  0 0
tmpfs           /tmp            tmpfs   defaults 0 0
tmpfs           $rom            tmpfs   defaults 0 0
tmpfs           /dev/shm        tmpfs   defaults 0 0
EOF
################

        cd ${TOPDIR} || return 1
        cd $1 || return 1

        scanelf -qnBR . | grep libgcc
        REALSIZE=$(LANG=C du ./ -s -c -k | grep total | awk '{print $1}')
        ADDTOROOTSIZE=$([ $REALSIZE -ge 20000 ] && echo 16384 || echo 16)
        SIZE=$(( $REALSIZE + $ADDTOROOTSIZE ))
        INODES=$(($(find ${DESTDIR} | wc -l) + 400))
        genext2fs -i ${INODES} -b ${SIZE} -q -d . $2
        cd $TOPDIR
}

TOPDIR=$PWD

tarballs="$@"
[[ $1 == "" ]] && tarballs="$(ls *.tar.bz2)"

for x in ${tarballs} ; do
        d=$(basename $x .tar.bz2)
        cd $TOPDIR
        pkg_genext2fs ${d} $TOPDIR/${d}.ext2
        [ $? != 0 ] && echo WTF $?
done

End result..
arm-gentoo-linux-uclibc.ext2 armeb-softfloat-linux-uclibc.ext2 mips-gentoo-linux-uclibc.ext2 powerpc-softfloat-linux-uclibc.ext2
arm-softfloat-linux-uclibc.ext2 i386-gentoo-linux-uclibc.ext2 mipsel-gentoo-linux-uclibc.ext2 sh4-gentoo-linux-uclibc.ext2
armeb-gentoo-linux-uclibc.ext2 i686-gentoo-linux-uclibc.ext2 powerpc-gentoo-linux-uclibc.ext2

As we can see uClibc is clearly better at everything. In this case being cross compiled and producing all the parts needed in order to be booted..

wget http://tinderbox.x86.dev.gentoo.org/portage/local/misc/i386-gentoo-linux-uclibc.ext2

qemu -kernel /boot/bzImage -append “root=/dev/hda init=/sbin/init rootfstype=ext2 ro” i386-gentoo-linux-uclibc.ext2

Sweet worked like a charm..
Next task is to mass x-compile some kernels for other arches and see what qemu can do for us..

Other people to the rescue

Well thanks to Corey Shields of OSU OSL our tinderbox now has 167 GB more space to play with. I’ve already moved all the web/ftp content over to that drive.
We also enabled rsync as a service on the box so that anybody using the binrepo as the PORTAGE_BINHOST= for new installs can have the same tree used to build stuff. Any customizations (rare but needed here and there) are in $PORTDIR/local dir but that of course excluded by default when doing an emerge –sync.

Kevin Quinn hacked on pax-utils the other day and managed to come up with a patch that shifts all the QA_TEXTREL/QA_EXECSTACK filtering logic into the scanelf utility directly. This is good news to us because it keeps ebuild.sh clean and all the logic in a single place between the stable and unstable portage trees.

portage-2.0.54-r2 is starting to be marked stable by the arches. This is also good news as it fixes a silly long standing emerge -e portage looping bug with minor revisions.

htmlifying a binrepo

I did a cvs -Q up yesterday and noticed I had forgotten to commit a netkit-telnetd update from a while ago. Our netkit-telnetd package inherits patches from debian. While I was checking to see what the latest debian patch level was I became inspired to do something similar to their pkg listing page. debian netkit-telnet

So I started hacking on a little bash script last night before I went to sleep,when I woke up today I started hacking on it a bit more and it’s starting to look ok now. I still want to gather and display a bit more information than I am now, but I’m starting to become inspired to eat lunch and do nothing for a few hours. So here’s what i’ve got now.. Gentoo netkit-telnetd

The index pages

Hopefully somebody else will come along and do it the right way. IE full .xml/.rss feeds and linking it to http://packages.gentoo.org etc..

Random follow up

Our tinderbox now has 5259 binpkgs between all of it’s $CHOST repos. (not bad.. glep19 stuff almost)

Managed to get portage-2.0.54-r2 out the door yesterday. Which is cool because it fixes an annoying looping bug with -e system that’s been present for a long time (but overlooked in 2.0.x). SpanKY and Kevin Quinn are trying to perfect a small snippet of code for handling text relocations and executable stacks. I expect that we will see a portage-2.0.54-r3 after those two guys work it out, which hopefully will make it a candidate for stable marking.

TclPython is neat but giving me a few problems with stdout handling that it really should not be giving me.

Example:

if {[file exists /usr/lib/tclpython/tclpython.so ]} {
catch {load /usr/lib/tclpython/tclpython.so}
}

package require tclpython 4

set interp [python::interp new]

$interp exec {import site, sys}
$interp exec {sys.path.append(“/var/irc/jeeves”)}
$interp exec {import metadata}
$interp exec {sys.argv=[‘e’,’mtd’]}

catch { [$interp exec {metadata.main()}] } result

python::interp delete $interp

The metadata.py can be found at http://people.gentoo.org/solar/irc/metadata.py

Anybody that’s worked with Tcl knows the catch {} should grab the stdout/stderr and shove the output in $result. However it’s printing directly to the console and never getting saved in $result. major bummer..

Tcl+Python

I just discovered tclpython yesterday. I really did not have time to play with it then, but I’m trying to squeeze in playing with it while sitting here at work and updating/renewing a few hundred domains.
If all goes well I hope to speed up the bot known as jeeves when doing !herd and !metadata lookups on irc by keeping the python interpreter loaded in memory. Oh and it’s an unmaintained package in portage in need of a bump to 4.1.
Guess I’ll do that.