Quick Seneca Summary

Projects:

Scripts:

Quick Seneca Summary

Some Koji Scripts (The Push For Primary)

The push for ARM to become primary arch is not as easy as one might think. Everyone in the community is working hard to prove how ARM is as capable in most areas to that of the machines in primary architecture. Anyway, here are a couple of scripts one could use to help determine some basic stats about ones current build process.

To find out the current error counts and types (and unbuilt packages) of a given tag:

(could help to show how far off one is in keeping up with primary and where the errors are)

#!/bin/bash

tagname="f17"
tagnumb=`echo "${tagname}" | sed -e 's/[^0-9]//g'`

if [ ! -f pripkgs.txt -o "$1" == "f" ]
then
	koji --server=http://koji.fedoraproject.org/kojihub list-tagged ${tagname} --inherit --latest --quiet | awk '{ print $1 }' | sed -e 's/-[^-]*-[^-]*$//g' > pripkgs.txt
fi

if [ ! -f armpkgs.txt -o "$1" == "f" ]
then
	koji --server=http://arm.koji.fedoraproject.org/kojihub list-tagged ${tagname} --inherit --latest --quiet | awk '{ print $1 }' | sed -e 's/-[^-]*-[^-]*$//g' > armpkgs.txt
fi

builtnum=0; builtstr=""
archnumb=0; archstri=""
depsnumb=0; depsstri=""
reqsnumb=0; reqsstri=""
confnumb=0; confstri=""
buildnum=0; buildstr=""
makenumb=0; makestri=""
unknnumb=0; unknstri=""
ubuilnum=0; ubuilstr=""

outpstri=""

while read pkgname
do
	pkgtagged=`cat armpkgs.txt | grep "${pkgname}"`
	
	if [ "${pkgtagged}" == "" ]
	then
		echo "[p] ${pkgname}" 1>&2
		
		pkgstat=`curl -Ls "http://arm.koji.fedoraproject.org/koji/search?match=glob&type=package&terms=${pkgname}" | tr -d '\t\r\n' | sed -e 's/<tr/\t<tr/g' | tr '\t' '\n' | grep -i "^<tr.*buildinfo.buildID" | sed -e 's/<a[^>]*buildinfo.buildID=\([0-9]*\)[^>]*>/ \1 /g' -e 's/<img[^>]*alt=.\([a-zA-Z]*\).[^>]*>/ \1 /g' -e 's/<[^>]*>/ /g' | grep -i "fc${tagnumb}" | sed -e 's/^[ \t]*//g' -e 's/[ \t]*$//g'`
		builtstat=`echo "${pkgstat}" | grep -i "complete$" | head -n 1`
		failstat=`echo "${pkgstat}" | grep -i "failed$" | head -n 1`
		
		if [ "${builtstat}" != "" ]
		then
			if [ "$2" == "p" ]
			then
				let builtnum="${builtnum} + 1"
				builtstr="${builtstr}${pkgname} http://arm.koji.fedoraproject.org/koji/search?match=glob&type=package&terms=${pkgname}\n"
			
			else
				outpstri="${outpstri}\n\"${pkgname}\":[\"complete\", \"\", 0]"
			fi
		
		elif [ "${failstat}" != "" ]
		then
			pkgnumb=`echo "${failstat}" | awk '{ print $1 }'`
			tasknumb=`curl -Ls "http://arm.koji.fedoraproject.org/koji/buildinfo?buildID=${pkgnumb}" | tr -d '\t\r\n' | sed -e 's/<a/\t<a/g' | tr '\t' '\n' | grep -i "^<a.*taskinfo.taskID" | sed -e 's/<a[^>]*taskinfo.taskID=\([0-9]*\)[^>]*>/ \1 /g' | awk '{ print $1 }' | head -n 1`
			failnumb=`curl -Ls "http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=${tasknumb}" | tr -d '\t\r\n' | sed -e 's/<a/\t<a/g' | tr '\t' '\n' | grep -i "^<a.*taskinfo.taskID.*failed" | sed -e 's/<a[^>]*taskinfo.taskID=\([0-9]*\)[^>]*>/ \1 /g' | awk '{ print $1 }' | head -n 1`
			logsoutp=`koji --server=http://arm.koji.fedoraproject.org/kojihub watch-logs "${failnumb}"`
			
			archerro=`echo "${logsoutp}" | grep -i 'error: architecture is excluded' | head -n 1`
			depserro=`echo "${logsoutp}" | grep -i 'error: no package found for' | head -n 1`
			reqserro=`echo "${logsoutp}" | grep -i 'error: package' -A 1 | head -n 2 | tr -d '\t\r\n' | tr -s ' '`
			conferro=`echo "${logsoutp}" | grep -i 'error: bad exit status from' | head -n 1`
			builderr=`echo "${logsoutp}" | grep -i 'error: file not found' | head -n 1`
			makeerro=`echo "${logsoutp}" | grep -i 'make: .* error' | head -n 1`
			
			if [ "${archerro}" != "" ]
			then
				if [ "$2" == "p" ]
				then
					let archnumb="${archnumb} + 1"
					archstri="${archstri}${pkgname} http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=${failnumb}\n"
				
				else
					outpstri="${outpstri}\"${pkgname}\":[\"arch-error\", \"${archerro}\", ${failnumb}]\n"
				fi
			
			elif [ "${depserro}" != "" ]
			then
				if [ "$2" == "p" ]
				then
					let depsnumb="${depsnumb} + 1"
					depsstri="${depsstri}${pkgname} http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=${failnumb}\n"
				
				else
					outpstri="${outpstri}\"${pkgname}\":[\"dep-error\", \"${depserro}\", ${failnumb}]\n"
				fi
			
			elif [ "${reqserro}" != "" ]
			then
				if [ "$2" == "p" ]
				then
					let reqsnumb="${reqsnumb} + 1"
					reqsstri="${reqsstri}${pkgname} http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=${failnumb}\n"
				
				else
					outpstri="${outpstri}\"${pkgname}\":[\"req-error\", \"${reqserro}\", ${failnumb}]\n"
				fi
			
			elif [ "${builderr}" != "" ]
			then
				if [ "$2" == "p" ]
				then
					let buildnum="${buildnum} + 1"
					buildstr="${buildstr}${pkgname} http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=${failnumb}\n"
				
				else
					outpstri="${outpstri}\"${pkgname}\":[\"build-error\", \"${builderr}\", ${failnumb}]\n"
				fi
			
			elif [ "${makeerro}" != "" ]
			then
				if [ "$2" == "p" ]
				then
					let makenumb="${makenumb} + 1"
					makestri="${makestri}${pkgname} http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=${failnumb}\n"
				
				else
					outpstri="${outpstri}\"${pkgname}\":[\"make-error\", \"${makeerro}\", ${failnumb}]\n"
				fi
			
			elif [ "${conferro}" != "" ]
			then
				if [ "$2" == "p" ]
				then
					let confnumb="${confnumb} + 1"
					confstri="${confstri}${pkgname} http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=${failnumb}\n"
				
				else
					outpstri="${outpstri}\"${pkgname}\":[\"config-error\", \"${conferro}\", ${failnumb}]\n"
				fi
			
			else
				if [ "$2" == "p" ]
				then
					let unknnumb="${unknnumb} + 1"
					unknstri="${unknstri}${pkgname} http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=${failnumb}\n"
				
				else
					outpstri="${outpstri}\"${pkgname}\":[\"unknown-error\", \"Error could not be interpreted.\", ${failnumb}]\n"
				fi
			fi
		
		else
			if [ "$2" == "p" ]
			then
				let ubuilnum="${ubuilnum} + 1"
				ubuilstr="${ubuilstr}${pkgname}\n"
			
			else
				outpstri="${outpstri}\"${pkgname}\":[\"unbuilt\", \"\", 0]\n"
			fi
		fi
	fi
done < pripkgs.txt

extranum=0; extrastr=""

while read pkgname
do
	pkgtagged=`cat pripkgs.txt | grep "${pkgname}"`
	
	if [ "${pkgtagged}" == "" ]
	then
		echo "[s] ${pkgname}" 1>&2
		
		if [ "$2" == "p" ]
		then
			let extranum="${extranum} + 1"
			extrastr="${extrastr}${pkgname}\n"
		
		else
			outpstri="${outpstri}\"${pkgname}\":[\"extra\", \"\", 0]\n"
		fi
	fi
done < armpkgs.txt

if [ "$2" == "p" ]
then
	printf '%10s|%10s|%10s|%10s|%10s|%10s|%10s\n' "Arch" "Deps" "Reqs" "Config" "Build" "Make" "Unknown"
	for i in `seq 1 80` ; do printf '-' ; done ; printf '\n'
	printf '%10s|%10s|%10s|%10s|%10s|%10s|%10s\n' "${archnumb}" "${depsnumb}" "${reqsnumb}" "${confnumb}" "${buildnum}" "${makenumb}" "${unknnumb}"
	echo
	if [ "${archstri}" != "" ] ; then echo -e "Arch-Errors:" ; echo -e "${archstri}" ; fi
	if [ "${depsstri}" != "" ] ; then echo -e "Dep-Errors:" ; echo -e "${depsstri}" ; fi
	if [ "${reqsstri}" != "" ] ; then echo -e "Require-Errors:" ; echo -e "${reqsstri}" ; fi
	if [ "${confstri}" != "" ] ; then echo -e "Config-Errors:" ; echo -e "${confstri}" ; fi
	if [ "${buildstr}" != "" ] ; then echo -e "Build-Errors:" ; echo -e "${buildstr}" ; fi
	if [ "${makestri}" != "" ] ; then echo -e "Make-Errors:" ; echo -e "${makestri}" ; fi
	if [ "${unknstri}" != "" ] ; then echo -e "Unknown-Errors:" ; echo -e "${unknstri}" ; fi
	if [ "${builtstr}" != "" ] ; then echo -e "Built but un-tagged/blocked pkgs = ${builtnum}" ; echo -e "${builtstr}" ; fi
	echo -e "Number of unbuilt/cancelled/building pkgs = ${ubuilnum}\n"
	if [ "${extrastr}" != "" ] ; then echo -e "Pkgs tagged in arm and not primary = ${extranum}" ; echo -e "${extrastr}" ; fi
	echo

else
	echo -e "${outpstri}"
	echo
fi

This script can help to compare build times between your various machines and primary:

(could help to compare how quickly ones build farm could keep up with primaries)

#!/bin/bash

function taskinfo
{
	roottask=`koji --server="$1/kojihub" buildinfo "$2" | grep -i '^task:' | awk '{ print $2 }'`
	curl -Ls "$1/koji/taskinfo?taskID=${roottask}" | tr -d '\t\r\n' | sed -e 's/<a/\t<a/g' | tr '\t' '\n' | grep -i 'taskinfo.taskID=.*buildArch' | sed -e 's/^.*taskinfo.taskID=\([0-9]*\).*$/ \1 /g' > /tmp/koji.task.$4.txt
	
	while read tasknumb
	do
		taskinfo=`curl -Ls "$1/koji/taskinfo?taskID=${tasknumb}" | tr -d '\t\r\n' | sed -e 's/<tr/\t<tr/g' | tr '\t' '\n'`
		begtime=`echo "${taskinfo}" | grep -i '^<tr>.*<th>.*Started.*</th>' | sed -e 's/^.*<td>//g' -e 's/<[^>]*>/ /g' -e 's/^[ \t]*//g' -e 's/[ \t]*$//g'`
		endtime=`echo "${taskinfo}" | grep -i '^<tr>.*<th>.*Completed.*</th>' | sed -e 's/^.*<td>//g' -e 's/<[^>]*>/ /g' -e 's/^[ \t]*//g' -e 's/[ \t]*$//g'`
		archname=`echo "${taskinfo}" | grep -i "^<tr>.*<th>.*Arch.*</th>" | sed -e 's/^.*<td>//g' -e 's/<[^>]*>/ /g' -e 's/^[ \t]*//g' -e 's/[ \t]*$//g'`
		hostname=`echo "${taskinfo}" | grep -i "^<tr>.*<th>.*Host.*</th>.*<td>.*$3.*</td>" | sed -e 's/^.*<td>//g' -e 's/<[^>]*>/ /g' -e 's/^[ \t]*//g' -e 's/[ \t]*$//g'`
		
		if [ "${begtime}" != "" -a "${endtime}" != "" -a "${archname}" != "" -a "${hostname}" != "" ]
		then
			begepoch=`date --date="${begtime}" '+%s'`
			endepoch=`date --date="${endtime}" '+%s'`
			
			hourdiff=`echo "(${endepoch} - ${begepoch}) / 3600" | bc`
			if [ ${hourdiff} -lt 10 ] ; then hourdiff="0${hourdiff}" ; fi
			
			mindiff=`echo "((${endepoch} - ${begepoch}) / 60) % 60" | bc`
			if [ ${mindiff} -lt 10 ] ; then mindiff="0${mindiff}" ; fi
			
			echo "${archname} ${hourdiff}:${mindiff}"
		fi
	done < /tmp/koji.task.$4.txt
}

tagname="f17"
tagnumb=`echo "${tagname}" | sed -e 's/[^0-9]//g'`

if [ ! -f primary.built.txt ]
then
	koji --server=http://koji.fedoraproject.org/kojihub list-tagged ${tagname} --inherit --latest --quiet | awk '{ print $1 }' > primary.built.txt
fi

tempfile="$1"
uniqname=`head -c 1024 /dev/urandom | md5sum | awk '{ print $1 }'`

if [ "${tempfile}" == "" ]
then
	koji --server=http://arm.koji.fedoraproject.org/kojihub list-tagged ${tagname} --inherit --latest --quiet | awk '{ print $1 }' > arm.built.txt
	tempfile="arm.built.txt"
fi

builder="$2"

if [ "${builder}" == "" ]
then
	builder="cdot.trimslice"
fi

printf '%-40s | %10s | %10s | %10s | %10s | %20s\n' "Package Name" "x86" "x64" "ARMvSFP" "ARMvHFP" "(Time = HH:MM)"
for i in `seq 1 95` ; do printf '-' ; done ; printf '\n'

while read srpmname
do
	pkgname=`echo "${srpmname}" | sed -e 's/-[^-]*-[^-]*$//g'`
	primpkg=`cat primary.built.txt | grep "^${pkgname}-[0-9\.][0-9\.]*-" | head -n 1`
	
	arminfo=`taskinfo "http://arm.koji.fedoraproject.org" "${srpmname}" "${builder}" "${uniqname}"`
	
	if [ "${arminfo}" != "" ]
	then
		priminfo=`taskinfo "http://koji.fedoraproject.org" "${primpkg}" ".*" "${uniqname}"`
		
		tttime=`echo "${priminfo}" | grep -i '^i386 ' | awk '{ print $2 }' | head -n 1`
		sftime=`echo "${priminfo}" | grep -i '^x86_64 ' | awk '{ print $2 }' | head -n 1`
		astime=`echo "${arminfo}" | grep -i '^arm ' | awk '{ print $2 }' | head -n 1`
		ahtime=`echo "${arminfo}" | grep -i '^armhfp ' | awk '{ print $2 }' | head -n 1`
		
		printf '%-40s | %10s | %10s | %10s | %10s |\n' "${pkgname}" "${tttime}" "${sftime}" "${astime}" "${ahtime}"
	fi
done < ${tempfile}

Some Koji Scripts (The Push For Primary)

[really off topic] I also really like colorful circles moving around randomly!

If my server is still up -> Liquid Particles

liquid-particles.js

/**
*   Liquid particles canvas experiment
*   ©2010 spielzeugz.de 
*/

var PI_2        = Math.PI * 2;

var canvasW     = 8;
var canvasH     = 8;
var numMovers   = 1024;
var friction    = 0.96;
var movers      = [];

var canvas = null;
var ctx = null;

var mouseX;
var mouseY;
var mouseVX;
var mouseVY;
var prevMouseX;
var prevMouseY;
var isMouseDown;

function init(){
	canvas = document.getElementById("mainCanvas");
	
	canvasW = canvas.width;
	canvasH = canvas.height;
	numMovers = (canvasW + canvasH);
	
	if ( canvas.getContext ){
		setup();
		setInterval( run , 33 );
		trace("interact with the mouse, occasionally click or hold down the mousebutton");
	}
	else{
		trace("Sorry, needs a recent version of Chrome, Firefox, Opera, Safari, or Internet Explorer 9.");
	}
}
   
function setup(){
	ctx       = canvas.getContext("2d");
	
	var i = numMovers;
	while ( i-- ){
		var m = new Mover();
		m.x   = canvasW * 0.5;
		m.y   = canvasH * 0.5;
		m.vX  = Math.cos(i) * Math.random() * 34;
		m.vY  = Math.sin(i) * Math.random() * 34;
		movers[i] = m;
	}
	
	mouseX = prevMouseX = canvasW * 0.5;
	mouseY = prevMouseY = canvasH * 0.5;
}

function run(){
	ctx.globalCompositeOperation = "source-over";
	ctx.fillStyle = "rgba(8,8,12,0.65)";
	ctx.fillRect( 0 , 0 , canvasW , canvasH );
	ctx.globalCompositeOperation = "lighter";
	
	mouseVX    = mouseX - prevMouseX;
	mouseVY    = mouseY - prevMouseY;
	prevMouseX = mouseX;
	prevMouseY = mouseY;
	
	var toDist   = canvasW * 0.86;
	var stirDist = canvasW * 0.125;
	var blowDist = canvasW * 0.5;
	
	var Mrnd = Math.random;
	var Mabs = Math.abs;
	
	var i = numMovers;
	while ( i-- ){
		var m  = movers[i];
		var x  = m.x;
		var y  = m.y;
		var vX = m.vX;
		var vY = m.vY;
		
		var dX = x - mouseX;
		var dY = y - mouseY; 
		var d  = Math.sqrt( dX * dX + dY * dY ) || 0.001;
		dX /= d;
		dY /= d;
		
		if ( isMouseDown ){
			if ( d < blowDist ){
				var blowAcc = ( 1 - ( d / blowDist ) ) * 14;
				vX += dX * blowAcc + 0.5 - Mrnd();
				vY += dY * blowAcc + 0.5 - Mrnd();
			}
		}
		
		if ( d < toDist ){
			var toAcc = ( 1 - ( d / toDist ) ) * canvasW * 0.0014;
			vX -= dX * toAcc;
			vY -= dY * toAcc;			
		}
		
		if ( d < stirDist ){
			var mAcc = ( 1 - ( d / stirDist ) ) * canvasW * 0.00026;
			vX += mouseVX * mAcc;
			vY += mouseVY * mAcc;			
		}
		
		vX *= friction;
		vY *= friction;
		
		var avgVX = Mabs( vX );
		var avgVY = Mabs( vY );
		var avgV  = ( avgVX + avgVY ) * 0.5;
		
		if( avgVX < .1 ) vX *= Mrnd() * 3;
		if( avgVY < .1 ) vY *= Mrnd() * 3;
		
		var sc = avgV * 0.45;
		sc = Math.max( Math.min( sc , 3.5 ) , 0.4 );
		
		var nextX = x + vX;
		var nextY = y + vY;
		
		if ( nextX > canvasW ){
			nextX = canvasW;
			vX *= -1;
		}
		else if ( nextX < 0 ){
			nextX = 0;
			vX *= -1;
		}
		
		if ( nextY > canvasH ){
			nextY = canvasH;
			vY *= -1;
		}
		else if ( nextY < 0 ){
			nextY = 0;
			vY *= -1;
		}
		
		m.vX = vX;
		m.vY = vY;
		m.x  = nextX;
		m.y  = nextY;
		
		ctx.fillStyle = m.color;
		ctx.beginPath();
		ctx.arc( nextX , nextY , sc , 0 , PI_2 , true );
		ctx.closePath();
		ctx.fill();		
	}
}

function onDocMouseMove( e ){
	var ev = e ? e : window.event;
	mouseX = ev.clientX - canvas.offsetLeft;
	mouseY = ev.clientY - canvas.offsetTop;
}

function onDocMouseDown( e ){
	isMouseDown = true;
	return false;
}

function onDocMouseUp( e ){
	isMouseDown = false;
	return false;
}

function Mover(){
	this.color = "rgb(" + Math.floor( Math.random()*255 ) + "," + Math.floor( Math.random()*255 ) + "," + Math.floor( Math.random()*255 ) + ")";
	this.y     = 0;
	this.x     = 0;
	this.vX    = 0;
	this.vY    = 0;
	this.size  = 1; 
}

function rect( context , x , y , w , h ){
	context.beginPath();
	context.rect( x , y , w , h );
	context.closePath();
	context.fill();
}

function trace( str ){
	console.log(str);
}

lp.html

<html>
	<head>
		<title>Automatic Liquid Particles</title>
		
		<style>
			body
			{
				background-color: black;
			}
			
			canvas
			{
				border: 1px dotted white;
			}
		</style>
		
		<script src="liquid-particles.js"></script>
		<script>
			var mousepos = [0, 0, 1, 1];
			var lasttime = 0;
			
			function fakemove()
			{
				var dateobjc = new Date();
				var prestime = (dateobjc.getTime() / 1000);
				
				if ((canvasW > 8) && (canvasH > 8) && (canvas != null))
				{
					onDocMouseDown(null);
					
					if ((prestime - lasttime) >= 1)
					{
						mousepos[2] = parseInt(Math.random() * 2);
						mousepos[3] = parseInt(Math.random() * 2);
						
						if (mousepos[2] == 0) { mousepos[2] = -1; }
						if (mousepos[3] == 0) { mousepos[3] = -1; }
						
						lasttime = prestime;
					}
					
					mousepos[0] = Math.min(canvasW, Math.max(0, mousepos[0] + (16 * mousepos[2])));
					mousepos[1] = Math.min(canvasH, Math.max(0, mousepos[1] + (16 * mousepos[3])));
					
					var mouseeve = {"clientX":(mousepos[0] + canvas.offsetLeft), "clientY":(mousepos[1] + canvas.offsetTop)};
					
					onDocMouseMove(mouseeve);
				}
				
				setTimeout("fakemove();", 32);
			}
			
			function prepinit()
			{
				document.body.innerHTML = ("<center><canvas id='mainCanvas' width='" + (document.body.offsetWidth - 32) + "' height='" + (document.body.offsetHeight - 32) + "'></canvas></center>");
				init();
				fakemove();
			}
		</script>
	</head>
	
	<body onload="prepinit();"></body>
</html>
[really off topic] I also really like colorful circles moving around randomly!

[off topic] I like the horoscopes from thestar…

Ok so I was bored again and I was randomly reading my scopes from the star and I liked them so here’s an overly complicated bash command to get them for you!

u=`curl -s "http://www.thestar.com/horoscopes" | tr -d '\t\r\n' | sed -e 's/<a /\t<a /g' | tr '\t' '\n' | grep -i '/horoscopes/article/.*--' | sed -e 's/^.*<a href="//g' -e 's/">.*$//g' | head -n 1` ; h=`curl -s "http://www.thestar.com/${u}" | tr -d '\t\r\n' | sed -e 's/<b>/\t/g' | tr '\t' '\n' | sed -e 's/<[^>]*>/ /g' | tr -s ' ' | grep -i "^aqua"` ; echo "$u" ; echo ; echo "$h" ; echo

Just change the grep -i "^aqua" part near the end of the command to the start of your own horoscope name. Also note, if the star decides to change the structure of their html web pages even just a little, this script is likely to break!

johnny stoops

[off topic] I like the horoscopes from thestar…

Fedora ARM Image Installer (FAII)

So my third "git-worthy" project is called the Fedora ARM Image Installer. It will basically allow one to "easily" convert an SD-Card into a bootable ARM media device. It is written in Python (2 & 3 for Linux & Windows) and it is based on the PyQt4 GUI framework giving it a more native feel for each platform. Below are some screenshots and links!

Source Code (Git Hub)
Binary Program (Fedora People)
Fedora Wiki

Fedora ARM Image Installer (FAII)

Fedora Losing My WiFi Inet…

So Fedora has been disconnecting me a lot lately from my home wifi and I think it’s because I have disabled a lot of background services that I don’t need. Anyway, I’m trying something new:

yum remove NetworkManager
mkdir /etc/wifi
wpa_passphrase "your_ssid_here" > /etc/wifi/home.conf

Run this script as root when wanting to connect to wifi (assumes the wlan0 interface):

#!/bin/bash
pkil=`which killall`
intf=`which ifconfig`
scan=`which iwlist`
wpas=`which wpa_supplicant`
dhcp=`which dhclient`
ifce="wlan0"
while true
do
	ping -c 1 google.ca > /dev/null 2>&1
	if [ $? -eq 0 ]
	then
		sleep 10
		continue
	fi
	for file in /etc/wifi/*
	do
		$intf "$ifce" up
		ssid=`cat $file | grep -i 'ssid' | sed -e 's/^[ \t]*ssid="//g' -e 's/"[ \t]*$//g'`
		exis=`$scan "$ifce" scan | grep -i "$ssid"`
		if [ "$exis" != "" ]
		then
			echo "[`date`] connecting to [$ssid]..."
			$pkil -9 wpa_supplicant > /dev/null 2>&1
			$pkil -9 dhclient > /dev/null 2>&1
			$intf "$ifce" up
			$wpas -Dwext -i "$ifce" -c "$file" > /dev/null 2>&1 &
			$dhcp "$ifce" > /dev/null 2>&1 &
			break
		fi
	done
	sleep 10
done
Fedora Losing My WiFi Inet…

Bored And Off Topic!

Just wanted to see how many lines of Python it took me to make a generalized password bruteforce script…

import time

slst = "abcdefghijklmnopqrstuvwxyz"
slen = (len(slst) - 1)
dwrd = [0]
dlen = 1

numb = 0
last = time.time()
wait = 2

while (1):
	# process password
	i = 0
	s = ""
	while (i < dlen):
		s = (s + slst[dwrd[i]])
		i = (i + 1)
	# print stats
	numb = (numb + 1)
	pres = time.time()
	if ((pres - last) >= wait):
		print("pw=[%s] @ [%d p/s]" % (s, numb / wait))
		numb = 0
		last = pres
	# overflow increase
	dwrd[dlen - 1] = (dwrd[dlen - 1] + 1)
	i = (dlen - 1)
	u = 1
	while (i > -1):
		if (dwrd[i] > slen):
			dwrd[i] = 0
			if ((i - 1) > -1):
				dwrd[i - 1] = (dwrd[i - 1] + 1)
		else:
			u = 0
		i = (i - 1)
	# length expansion
	if (u == 1):
		i = 0
		while (i < dlen):
			dwrd[i] = 0
			i = (i + 1)
		dwrd.append(0)
		dlen = (dlen + 1)
Bored And Off Topic!

Formatting Flat Files As Block Devices

Here’s a small shell script based on dd’s output which helps to partition a flat file, split it up into its respective partitions so they can be formatted and put them all back together again.

#!/bin/bash

if [ "${1}" == "c" ]
then
	if [ "${3}" == "" ]
	then
		echo "Usage: $0 $1 <filename> <filesize>"
		exit 1
	fi
	echo "Creating new file [${2}]..."
	dd if=/dev/zero of=${2} bs=${3}G count=1 > /dev/null 2>&1
	echo "Partition the new flat file ( fdisk ${2} )"
	echo
fi

if [ "${1}" == "s" ]
then
	if [ "${2}" == "" ]
	then
		echo "Usage: $0 $1 <filename>"
		exit 1
	fi
	unit=`fdisk -l "${2}" | grep -i '^Units' | sed -e 's/^.* \([0-9][0-9]*\) bytes.*$/\1/g'`
	lase="-1"
	numb=0
	fdisk -l "${2}" | sed -e 's/^[ \t]*//g' -e 's/\*//g' | grep -i "^${2}[0-9][0-9]*" | while read line
	do
		curb=`echo "${line}" | awk '{ print $2 }'`
		cure=`echo "${line}" | awk '{ print $3 }'`
		let size="${cure} - ${curb} + 1"
		let diff="${curb} - ${lase} - 1"
		if [ ${diff} -gt 0 ]
		then
			let endi="${curb} - 1"
			let skip="${lase} + 1"
			echo "[${numb}] Unused space [${skip} - ${endi}] [${diff} blocks]..."
			dd if=${2} bs=${unit} skip=${skip} count=${diff} of=${2}${numb} > /dev/null 2>&1
			let numb="${numb} + 1"
		fi
		echo "[${numb}] Carving partition [${curb} - ${cure}] [${size} blocks]..."
		dd if=${2} bs=${unit} skip=${curb} count=${size} of=${2}${numb} > /dev/null 2>&1
		let numb="${numb} + 1"
		lase="${cure}"
	done
	fdisk -l "${2}" | sed -e 's/^[ \t]*//g' -e 's/\*//g' | grep -i "^${2}[0-9][0-9]*" | tail -n 1 | while read line
	do
		endi=`echo "${line}" | awk '{ print $3 }'`
		let rest="${endi} + 1"
		numb="9"
		echo "[${numb}] Copying ending [${rest} - <end>]..."
		dd if=${2} bs=${unit} skip=${rest} of=${2}${numb} > /dev/null 2>&1
	done
	echo
	echo "Create any filesystem needed ( mkfs.* ${2}[0-9] )"
	echo "Mount any filesystem needed ( mount -o loop ${2}[0-9] /mnt/tmp[0-9] )"
	echo "Copy any files needed ( cp -r source/* /mnt/tmp[0-9]/ )"
	echo "Unmount any filesystem needed ( umount /mnt/tmp[0-9] )"
	echo
fi

if [ "${1}" == "j" ]
then
	if [ "${2}" == "" ]
	then
		echo "Usage: $0 $1 <filename>"
		exit 1
	fi
	echo "Writing to file [${2}.join]..."
	cat ${2}[0-9]* > ${2}.join
	echo
fi

Formatting Flat Files As Block Devices

[offtopic] iTunes Library Scripts

So I was trying to get my pc laptop setup for fudcon which meant a triple boot setup between windows, ubuntu and fedora. I had some troubles importing various important information about my music within iTunes.

Here is a script to remove duplicate songs in an iTunes folder:

import hashlib
import os
import re
import sys

def hash(fnam):
	fobj = open(fnam, "r")
	hobj = hashlib.sha256()
	
	while (1):
		data = fobj.read(2**20)
		
		if (not data):
			break
		
		hobj.update(data)
	
	return hobj.hexdigest()

fdic = {}

while (1):
	file = sys.stdin.readline()
	
	if (not file):
		break
	
	file = file.strip()
	uniq = hash(file)
	
	if (uniq in fdic.keys()):
		print("removing:",file)
		os.unlink(file)
	
	else:
		print("found:",file)
		fdic[uniq] = file

This script attempts to merge an old iTunes library xml file with a newly imported one. The result is a merged iTunes library xml file that can then be re-imported into iTunes thus restoring some various meta-data about your music. Note: Before you re-import the merged xml playlist file, make sure you delete the current one and disable the check-box stating to copy songs into the media folder.

import os
import re
import sys

if (len(sys.argv) < 3):
	print("Usage: %s <source> <merge>" % (sys.argv[0]))
	sys.exit(0)

def outp(ordr, dict, line):
	for item in ordr:
		sys.stdout.write(dict[item][1])
	
	sys.stdout.write(line)
	sys.stdout.flush()

def xmlr(fnam, sdic={}):
	alen = len(sys.argv)
	slen = len(sdic.keys())
	xord = []; xdic = {}
	xobj = open(fnam, "r")
	
	xdat = {}
	
	while (1):
		xlin = xobj.readline()
		
		if (not xlin):
			break
		
		slin = xlin.strip("\0\t\r\n ")
		xreg = re.match("^<key>([^<]+)</key>(<[^>]+>[^<]+<[^>]+>)$", slin)
		
		try:
			xkey = (xdic["Artist"][0] + xdic["Album"][0] + xdic["Name"][0])
		
		except:
			xkey = ""
		
		if (slin == "<dict>"):
			if (slen > 0):
				outp(xord, xdic, xlin)
			
			xord = []; xdic = {}
		
		elif (xreg):
			if (xreg.group(1) not in xord):
				xord.append(xreg.group(1))
			
			xdic[xreg.group(1)] = [xreg.group(2), xlin]
		
		elif (slin == "</dict>"):
			if (slen < 1):
				xdat[xkey] = xdic
			
			else:
				for x in range(3, alen):
					try:
						xdic[sys.argv[x]] = sdic[xkey][sys.argv[x]]
						
						if (sys.argv[x] not in xord):
							xord.append(sys.argv[x])
					
					except:
						#print("error:",sys.exc_info())
						pass
				
				outp(xord, xdic, xlin)
			
			xord = []; xdic = {}
		
		else:
			if (slen > 0):
				outp(xord, xdic, xlin)
			
			xord = []; xdic = {}
	
	return xdat

sdat = xmlr(sys.argv[1])
ddat = xmlr(sys.argv[2], sdic=sdat)

Here’s an example of how to use the command:

python.exe /cygdrive/c/Users/jon/Desktop/itml.py /cygdrive/g/tmp/itunes/iTunes\ Music\ Library0.xml /cygdrive/g/tmp/itunes/iTunes\ Music\ Library.xml 'Date Added' 'Play Count' | unix2dos.exe | tee /cygdrive/g/tmp/itunes/iTunes\ Music\ Library2.xml && cp /cygdrive/g/tmp/itunes/iTunes\ Music\ Library2.xml /cygdrive/g/tmp/itunes/iTunes\ Music\ Library3.xml
[offtopic] iTunes Library Scripts

[updated] F15 Koji Que Script

Here it is, hopefully it works a bit better!

#!/bin/bash
export x=1
export l=`find ./f15srpms -type f | grep -i '\.src\.rpm' | wc -l`
koji list-tagged dist-f15 > /tmp/pkoji.txt
find ./f15srpms -type f | sort -R | grep -i '\.src\.rpm' > /tmp/pkgsl.txt
while true
do
	while true
	do
		ak list-tagged dist-f15 > /tmp/akoji.txt
		ak list-tasks --mine --quiet | grep '^[0-9]' | grep -Ei ' (open|free) .* build' > /tmp/tasks.txt
		#echo "number of our tasks == `cat /tmp/tasks.txt | wc -l`"
		n=`cat /tmp/tasks.txt | wc -l`
		if [ $n -ge 10 ]
		then
			break
		fi
		p=`cat /tmp/pkgsl.txt | head -n "$x" | tail -n 1`
		q=`basename "$p" | sed -e 's/-[^-]*-[^-]*$//'`
		let x="($x % $l) + 1"
		#echo "checking pkg [$p] name [$q] not built on (akoji) and built on (pkoji)"
		c=`cat /tmp/akoji.txt /tmp/tasks.txt | grep -i "$q"`
		if [ "$c" != "" ]
		then
			continue
		fi
		c=`cat /tmp/pkoji.txt | grep -i "$q"`
		if [ "$c" == "" ]
		then
			continue
		fi
		echo "queing [$p] position [$x]"
		ak build dist-f15 "$p" --nowait --background
	done
	sleep 60
done

[updated] F15 Koji Que Script

[off-topic] Fighting Governments — DNS Re-Implemented

The U.S. government is taking down websites using DNS poisoning/hijacking methods on pages that are claimed to be infringing on copyright law. I have re-written a *really* basic DNS server which re-implements DNS forwarding/proxying, DNS caching and DNS un-blacklisting of A/IN/IPv4 records. Basically, if one were to run their own DNS server (like the one below) and maintain a simple list of government-poisoned IP addresses for it, the DNS server would be able to provide (previously expired but still good) IPv4 addresses for the blocked site. This could essentially allow one to re-access a now blacklisted website.

Here is a screen shot of my computer using this basic DNS server to visit slashdot:

$ host slashdot.org
slashdot.org has address 216.34.181.45
slashdot.org mail is handled by 10 mx.sourceforge.net.

And here is what it would look like if it were taken down via DNS poisoning:

$ host slashdot.org
slashdot.org has address 74.81.170.110
slashdot.org has address 216.34.181.45
slashdot.org mail is handled by 10 mx.sourceforge.net.

Once this happens, all one has to do now is append the newly offending IP address to the DNS server’s black list text file like so:

# cat dnsb.txt 
74.81.170.110

And if you perform another DNS request to the same (custom) DNS server, you get the last cached answers bringing you back to slashdot:

$ host slashdot.org
slashdot.org has address 216.34.181.45
slashdot.org mail is handled by 10 mx.sourceforge.net.

The code below is simply proof-of-concept software and should *NOT* be used for production as I have implemented everything from scratch here and know very little about how DNS really works (i.e. message compression).

import os
import re
import socket
import sqlite3
import sys
import time

sqlo = sqlite3.connect("dnsd.db")
sqlc = sqlo.cursor()

def fixs(inpt):
	outp = ""
	
	for letr in inpt:
		if (ord(letr) < 32):
			outp += "."
		
		else:
			outp += letr
	
	return outp

def rslv():
	nlst = []
	fobj = open("/etc/resolv.conf", "r")
	
	dnsl = fobj.readlines()
	
	for dnse in dnsl:
		item = dnse.strip("\0\t\r\n ")
		regx = re.match("^nameserver[ \t]*(.*)$", item)
		
		if (regx):
			addr = regx.group(1)
			nlst.append(addr)
	
	return nlst

def forw(addr, data):
	resp = ""
	serv = (addr, 53)
	nobj = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
	
	try:
		nobj.settimeout(5)
		nobj.sendto(data, serv)
		resp = nobj.recv(2**20)
	
	except:
		pass
	
	try:
		nobj.close()
	
	except:
		pass
	
	return resp

def answ(data, deet=False):
	# http://www.ietf.org/rfc/rfc1035.txt
	
	requ = 13
	null = (requ + data[requ:].find("\0"))
	dtyp = (null + 5)
	dlen = (dtyp + 10)
	
	size = len(data)
	name = fixs(data[requ:null])
	begi = -1
	endi = -1
	ansl = []
	
	while ((dlen + 1) < size):
		dsiz = ((ord(data[dlen]) << 8) + ord(data[dlen + 1]))
		dsig = (dtyp + 4)
		
		if (((dlen + 5) < size) and (data[dtyp:dsig] == "\xc0\x0c\x00\x01")):
			if (deet == True):
				if (begi == -1):
					begi = dtyp
				
				else:
					endi = (dlen + 1 + dsiz + 1)
			
			addr = ""
			
			for x in range(2, 2 + 4):
				addr += ("." + str(ord(data[dlen + x])))
			
			ansl.append([name, addr[1:]])
		
		dtyp = (dlen + 1 + dsiz + 1)
		dlen = (dtyp + 10)
	
	if (begi == -1):
		begi = (null + 5)
		endi = begi
	
	if (deet == False):
		return ansl
	
	else:
		return [begi, endi]

def sqle(comd, outp=False):
	global sqlo
	global sqlc
	
	try:
		sqlc.execute(comd)
	
	except:
		pass
	
	if (outp == False):
		sqlo.commit()
	
	else:
		return sqlc.fetchall()

def filt():
	try:
		fobj = open("dnsb.txt", "r")
		
		for line in fobj.readlines():
			addr = line.strip("\0\t\r\n ")
			sqle("DELETE FROM data WHERE addr = '%s';" % (addr))
		
		fobj.close()
	
	except:
		return 0
	
	return 1

def cche(name):
	dbdl = sqle("SELECT * FROM data WHERE name = '%s';" % (name), outp=True)
	outp = []
	
	for dbdi in dbdl:
		adrl = dbdi[1].split(".")
		adrs = ""
		
		for adri in adrl:
			adrs += chr(int(adri))
		
		outp.append([int(dbdi[2]), adrs])
	
	outp.sort()
	outp.reverse()
	
	return outp

def shim(data, adrl):
	hedn = [6, 8]
	payn = answ(data, deet=True)
	
	lead = (payn[0] + 12)
	size = len(adrl)
	
	dlen = (chr((size >> 8) & 0xff) + chr((size >> 0) & 0xff))
	head = (data[:hedn[0]] + dlen + data[hedn[1]:payn[0]])
	
	payl = ""
	
	for adri in adrl:
		payl += (data[payn[0]:lead] + adri[1])
	
	payl += data[payn[1]:]
	
	outp = (head + payl)
	
	# beg: compression offset bypass
	
	outp = (outp[:8] + chr(0) + chr(0) + outp[10:])
	outp = (outp[:10] + chr(0) + chr(0) + outp[12:])
	
	# end: compression offset bypass
	
	# beg: compression offset fix
	'''
	x = 0
	l = len(outp)
	s = ((ord(data[6]) << 8) + ord(data[7]))
	d = ((size - s) * 16)
	
	if (d > 0):
		while ((x + 1) < l):
			o = ((ord(outp[x]) << 8) + ord(outp[x + 1]))
			
			if ((o & 0xff00) == 0xc000):
				o = (o & 0x00ff)
				
				if (o > 0x0c):
					o = (o + d)
					outp = (outp[:x] + chr((o >> 8) & 0xff) + chr((o >> 0) & 0xff) + outp[x + 2:])
			
			x = (x + 2)
	'''
	# end: compression offset fix
	
	return outp

def debg(addr, requ, answ):
	adrl = addr.split(".")
	adrl[3] = "x"
	prio = ""
	
	for letr in answ:
		prio += ("." + str(ord(letr)))
	
	print("[%s] requ [%s] answ [%s, ...]" % (".".join(adrl), requ, prio[1:]))

def serv():
	snam = ("", 53)
	sobj = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
	
	sobj.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
	sobj.bind(snam)
	
	while (1):
		(data, addr) = sobj.recvfrom(2**20)
		rlvl = rslv()
		
		for rlvi in rlvl:
			dnsr = forw(rlvi, data)
			
			if (dnsr):
				resl = answ(dnsr)
				
				if (len(resl) > 0):
					for resi in resl:
						dnsq = re.sub("[^0-9A-Za-z\.\-]", "", resi[0])
						dnsa = re.sub("[^0-9\.]", "", resi[1])
						epch = str(int(time.time()))
						
						dbdl = sqle("SELECT * FROM data WHERE name = '%s' AND addr = '%s';" % (dnsq, dnsa), outp=True)
						
						if (len(dbdl) < 1):
							sqle("INSERT INTO data VALUES ('%s', '%s', '0');" % (dnsq, dnsa))
						
						sqle("UPDATE data SET time = '%s' WHERE name = '%s' AND addr = '%s';" % (epch, dnsq, dnsa))
					
					filt()
					dbdl = cche(dnsq)
					
					if (len(dbdl) > 0):
						debg(addr[0], resl[0][0], dbdl[0][1])
						#a=open("a","w");a.write(dnsr);a.close();
						dnsr = shim(dnsr, dbdl)
						#b=open("b","w");b.write(dnsr);b.close();
				
				sobj.sendto(dnsr, addr)
				break

def priv():
	# get username & os.setgid() & os.setuid()
	pass

if (__name__ == "__main__"):
	sqle("CREATE TABLE data (name VARCHAR(256), addr VARCHAR(256), time VARCHAR(256));")
	priv()
	serv()

[off-topic] Fighting Governments — DNS Re-Implemented

Attempting To Finally Contribute

So I haven’t had much to talk about lately due the amount of work given for the last few weeks remaining in school and the building of Fedora 15 for ARMv5 and ARMv7. However, in my spare time, I’ve been trying to contribute positively to the Koji Build System project by submitting mailing list items containing source code unified diff patches. Most of my posts/changes contain feature enhancements requested by other people that I found to be somewhat interesting to code. If you’re interested in viewing the mailing list items, the links can be found below:

Hosts-Page Colour Highlight: [ Ticket ] [ Patch ]
Hosts-Page “Life-Line” Links: [ Ticket ] [ Patch ]
Host-Info Latest-Builds Links: [ Ticket ] [ Patch ]
Koji-Client watch-logs “tail”: [ Ticket ] [ Patch ]
Koji-Hub RPMTAG_SIGSHA2 Hashing: [ Ticket ] [ Patch ]

Attempting To Finally Contribute