A new Koji instance for a new arch

So we’ve fired up a new Koji setup to facilitate our future builds of ARMv6hl capable packages. The new site can be reached and tracked here ( http://japan.proximity.on.ca/koji/ ). Right now, it is attempting to catch up by “cleanly” and natively building Fedora Linux 17 for ARMv6hl for the first time. In the future, we may run the koji-shadow build script which matches the build process of the secondary && primary arch’s and employs a smarter que’ing technique.

Note:

For a new arch, you'll have to patch Koji as well. For example, you can find any references to armhfp/armv7hl and so on and replace them with armv6hl using this command:

rpm -qa | grep -Ei '(rpm|yum|koji)' | while read p ; do echo "$p" ; rpm -ql "$p" | xargs grep -i "armhfp" 2> /dev/null ; done

This (brute-force) build script can be run concurrently with other copies of itself at the same time.

que.sh

#!/bin/bash
while true
do
	x=1
	last="a"
	pkgn="b"
	tskl=`curl -sL 'http://japan.proximity.on.ca/koji/tasks?view=flat&state=active&method=build&order=-id' | grep -i 'through' | sed -e 's/^.*through [0-9]* of //g' -e 's/<.*$//g' | head -n 1`
	if [ "$tskl" == "" ]
	then
		tskl=0
	fi
	while [ $tskl -lt $1 ]
	do
		pkgn=`head -n "$x" pkgs.txt | tail -n 1`
		if [ "$pkgn" == "$last" ]
		then
			exit
		fi
		#echo "$pkgn"
		if [ ! -f "$pkgn.src.rpm" ]
		then
			echo "    building [$pkgn]...."
			koji -s 'http://arm.koji.fedoraproject.org/kojihub' download-build --arch=src --topurl='http://arm.koji.fedoraproject.org/' "$pkgn"
			koji -s 'http://japan.proximity.on.ca/kojihub' build f17 "$pkgn.src.rpm" --nowait
			let tskl="$tskl + 1"
		fi
		last="$pkgn"
		let x="$x + 1"
	done
	sleep 30
done

This script below will reset the failed build attempts after each run is completed.

rst.sh

#!/bin/bash
rm -fv *.src.rpm
for tagd in `koji -s 'http://japan.proximity.on.ca/kojihub' list-tagged f17 --quiet --latest --inherit | awk '{ print $1 }'`
do
	touch "${tagd}.src.rpm"
done

And since Koji seems to suck (no offence) at managing its sessions table, this script will clear out any expired sessions which are more than 10 minutes old so the table doesn’t become huge and slow.

exp.sh

#!/bin/bash
while true
do
        psql koji -c "select update_time from sessions where expired = 't';" | cat | grep -i '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]' | while read date
        do
                a=`date --date="$date" "+%s"`
                b=`date "+%s"`
                let c="($b - $a) / (60 * 10)"
                if [ $c -gt 0 ]
                then
                        echo "deleting [$date] exp [$c] from [`date`]"
                        psql koji -c "delete from sessions where expired = 't' and update_time = '$date';"
                fi
        done
        sleep 300
done
A new Koji instance for a new arch

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s