Pidora 18 (Raspberry Pi Fedora Remix) Release

Pidora 18 (Raspberry Pi Fedora Remix) Release

We're excited to announce the release of Pidora 18 -- 
an optimized Fedora Remix for the Raspberry Pi. 
It is based on a brand new build of Fedora for the ARMv6 
architecture with greater speed and includes packages 
from the Fedora 18 package set.

* * *

There are some interesting new features we'd like to highlight: 
* Almost all of the Fedora 18 package set available via yum 
    (thousands of packages were built from the official Fedora 
    repository and made available online)
* Compiled specifically to take advantage of the hardware already 
    built into the Raspberry Pi
* Graphical firstboot configuration (with additional modules 
    specifically made for the Raspberry Pi)
* Compact initial image size (for fast downloads) and auto-resize 
    (for maximum storage afterwards)
* Auto swap creation available to allow for larger memory usage
* C, Python, & Perl programming languages available & included 
    in the SD card image
* Initial release of headless mode can be used with setups 
    lacking a monitor or display
* IP address information can be read over the speakers and 
    flashed with the LED light
* For graphical operation, Gedit text editor can be used with 
    plugins (python console, file manager, syntax highlighting) 
    to serve as a mini-graphical IDE
* For console operation, easy-to-use text editors are included 
    (nled, nano, vi) plus Midnight Commander for file management
* Includes libraries capable of supporting external hardware 
    such as motors and robotics (via GPIO, I2C, SPI)

* * *

For further documentation, downloads, faq's, read-me's, 
    how-to's, tutorials, or videos: 
        http://pidora.ca/

* * *

Pidora 18 is a Fedora Remix -- a combination of software packages 
from the Fedora Project with other software.

The Fedora Project is a global community of contributors working 
to advance open source software. For more information or to join 
the Fedora Project, see http://fedoraproject.org

Pidora is a project of the Seneca Centre for Development of Open 
Technology (CDOT). To connect with CDOT, please visit 
http://cdot.senecacollege.ca

The Raspberry Pi is a small, inexpensive computer board designed 
to provoke curiosity and experimentation in programming and 
computer electronics. For more information, see the Raspberry 
Pi Foundation website at http://raspberrypi.org

* * *


- - -
The CDOT team at Seneca College
Pidora 18 (Raspberry Pi Fedora Remix) Release

Convert a really big hex string into a decimal string using bash & bc

x=1 ; m=1 ; n=0 ; while true ; do h=`cat /tmp/prime.txt | tr -d ' \t\r\n' | rev | cut -c $x | sed -e 's/A/10/' -e 's/B/11/' -e 's/C/12/' -e 's/D/13/' -e 's/E/14/' -e 's/F/15/'` ; if [ "$h" == "" ] ; then break ; fi ; n=`echo "$n + ($h * $m)" | bc` ; m=`echo "$m * 16" | bc` ; x=`echo "$x + 1" | bc` ; done ; n=`echo "$n" | tr -d ' \t\r\n' | sed -e 's/[^0-9]//g'` ; echo "$n"

Edit: Random self plug – http://fedoraproject.org/wiki/Fedora_ARM_Installer

Convert a really big hex string into a decimal string using bash & bc

Generating A GUI Windows EXE From A Python Script

# exes: python-2.7-x32, pyqt4-2.7-x32, py2exe-2.7-x32
# run: cd C:\Users\admin\Desktop\fedora-arm-installer
# files: setup.py, MSVCP90.dll
# run: C:\Python27\python.exe setup.py py2exe

import sys,os
from distutils.core import setup
import py2exe

os.system("rd /s /q .\\build")
os.system("rd /s /q .\\dist")
os.system("rd /s /q .\\fedora-arm-installer-0.0.0-0.x32")

a=open("fedora-arm-installer","r")
b=open("fedora-arm-installer-2","w")
for l in a.readlines():
 m = l
 m = m.replace("sys.stderr","#sys.stderr")
 m = m.replace("proglist.append(progcomd)","proglist.append(re.sub('fedora-arm-installer-2.exe$','helper.exe',progcomd))")
 b.write(m)
b.close()
a.close()

os.system("copy fedora-arm-installer-2 helper /y")

setup(console=['helper'], options={"py2exe":{"includes":["sip"]}})
setup(windows=['fedora-arm-installer-2'], options={"py2exe":{"includes":["sip"]}})

os.system("md .\\dist\\data")
os.system("xcopy .\\data .\\dist\\data")
os.system("rename .\\dist fedora-arm-installer-0.0.0-0.x32")

Edit: Updated the setup.py script above to remove the command line window

# exes: python-2.7-x32, pyqt4-2.7-x32, py2exe-2.7-x32
# run: cd C:\Users\admin\Desktop\fedora-arm-installer
# files: setup.py, MSVCP90.dll
# run: C:\Python27\python.exe setup.py py2exe

import sys,os
from distutils.core import setup
import py2exe

os.system("rd /s /q .\\build")
os.system("rd /s /q .\\dist")

a=open("fedora-arm-installer","r")
b=open("fedora-arm-installer-2","w")
for l in a.readlines():
 m = l.replace("sys.stderr","#sys.stderr")
 b.write(m)
b.close()
a.close()

setup(console=['fedora-arm-installer'], options={"py2exe":{"includes":["sip"]}})
#setup(windows=['fedora-arm-installer-2'], options={"py2exe":{"includes":["sip"]}})

os.system("md .\\dist\\data")
os.system("xcopy .\\data .\\dist\\data")
Generating A GUI Windows EXE From A Python Script

A Really Basic Koji-Shadow Script

This script will follow the general procedure:
* List the latest tagged (non-inherited) packages from the arch being followed
* Download and import any of the non-arch detected packages
* Sort all missing builds by their creation timestamp and order them to be qued
* Loop thru the missing package list and download and que each build

A simple server-side script to help reduce the size of the Koji sessions table:

#!/bin/bash
pd=$(echo $(date "+%s") - 1800 | bc) ; echo "$pd" ; psql koji -c "delete from sessions where extract(epoch from update_time) < $pd and master != 0;"
while true
do
	pd=$(echo $(date "+%s") - 1800 | bc) ; echo "$pd" ; psql koji -c "delete from sessions where expired = 't' and extract(epoch from update_time) < $pd;"
	sleep 2400
done

A simple script to re-que the latest failed build tasks in case of a build-root breakage:

#!/bin/bash
let x="0"
let l="50 * $1"
while [ $x -lt $l ]
do
	echo "[$x]"
	for t in `curl -sL "$1/koji/tasks?start=${x}&state=failed&view=flat&method=build&order=-completion_time" | grep -Eiv "pcre|libssh|glib2|glibc" | grep -i 'taskinfo.taskID=[0-9]*' | sed -e 's/^.*taskinfo.taskID=\([0-9]*\).*$/\1/g'`
	do
		echo "    [$t]"
		koji -s "$1/kojihub" resubmit --nowait "$t"
	done
	let x="$x + 50"
done

Source Code (Git Hub)
Source Code (Fedora People)

A Really Basic Koji-Shadow Script

Syncing iTunes Playlists From Windows 7 To A Nexus 4

* Connect both devices to a common WiFi AP and determine their IP addresses (Windows/Nexus)
* Download a capable SSH/SFTP/SCP server app that also allows you to setup a capable login user (Nexus)
* Download and run “dokan-sshfs” and then connect to and mount the phone as a “U:\” drive (Windows)
* Run the “syncplay.js” wscript below to sync a smart playlist called “recent” to the phone (Windows)

var iTunesApp = WScript.CreateObject("iTunes.Application");
var mainLibrary = iTunesApp.LibrarySource;
var playlists = mainLibrary.Playlists;
var numPlaylists = playlists.Count;
var fileSys = WScript.CreateObject("Scripting.FileSystemObject");

while (numPlaylists != 0)
{
	var currPlaylist = playlists.Item(numPlaylists);
	
	if (currPlaylist.Name == "recent")
	{
		var playlistTracks = currPlaylist.Tracks;
		var numTracks = playlistTracks.Count;
		
		WScript.Echo("[" + currPlaylist.Name + ":" + numTracks + "]");
		
		while (numTracks != 0)
		{
			var sourceTrack = playlistTracks.Item(numTracks);
			var targetTrack = ("U:\\sdcard\\Music\\" + fileSys.GetBaseName(sourceTrack.Location) + "." + fileSys.GetExtensionName(sourceTrack.Location));
			var sourceSize = fileSys.GetFile(sourceTrack.Location).Size;
			var targetSize = 0;
			
			if (fileSys.FileExists(targetTrack))
			{
				targetSize = fileSys.GetFile(targetTrack).Size;
			}
			
			//WScript.Echo("[" + sourceTrack.Name + ":" + sourceTrack.Location + ":" + targetTrack + "]");
			
			if (sourceSize != targetSize)
			{
				fileSys.CopyFile(sourceTrack.Location, targetTrack);
			}
			
			numTracks--;
		}
		
		//add check to remove uneeded songs
		//create a playlist xml file
		//force device refresh
		WScript.Echo("Done");
	}
	
	numPlaylists--;
}

Syncing iTunes Playlists From Windows 7 To A Nexus 4

A new Koji instance for a new arch

So we’ve fired up a new Koji setup to facilitate our future builds of ARMv6hl capable packages. The new site can be reached and tracked here ( http://japan.proximity.on.ca/koji/ ). Right now, it is attempting to catch up by “cleanly” and natively building Fedora Linux 17 for ARMv6hl for the first time. In the future, we may run the koji-shadow build script which matches the build process of the secondary && primary arch’s and employs a smarter que’ing technique.

Note:

For a new arch, you'll have to patch Koji as well. For example, you can find any references to armhfp/armv7hl and so on and replace them with armv6hl using this command:

rpm -qa | grep -Ei '(rpm|yum|koji)' | while read p ; do echo "$p" ; rpm -ql "$p" | xargs grep -i "armhfp" 2> /dev/null ; done

This (brute-force) build script can be run concurrently with other copies of itself at the same time.

que.sh

#!/bin/bash
while true
do
	x=1
	last="a"
	pkgn="b"
	tskl=`curl -sL 'http://japan.proximity.on.ca/koji/tasks?view=flat&state=active&method=build&order=-id' | grep -i 'through' | sed -e 's/^.*through [0-9]* of //g' -e 's/<.*$//g' | head -n 1`
	if [ "$tskl" == "" ]
	then
		tskl=0
	fi
	while [ $tskl -lt $1 ]
	do
		pkgn=`head -n "$x" pkgs.txt | tail -n 1`
		if [ "$pkgn" == "$last" ]
		then
			exit
		fi
		#echo "$pkgn"
		if [ ! -f "$pkgn.src.rpm" ]
		then
			echo "    building [$pkgn]...."
			koji -s 'http://arm.koji.fedoraproject.org/kojihub' download-build --arch=src --topurl='http://arm.koji.fedoraproject.org/' "$pkgn"
			koji -s 'http://japan.proximity.on.ca/kojihub' build f17 "$pkgn.src.rpm" --nowait
			let tskl="$tskl + 1"
		fi
		last="$pkgn"
		let x="$x + 1"
	done
	sleep 30
done

This script below will reset the failed build attempts after each run is completed.

rst.sh

#!/bin/bash
rm -fv *.src.rpm
for tagd in `koji -s 'http://japan.proximity.on.ca/kojihub' list-tagged f17 --quiet --latest --inherit | awk '{ print $1 }'`
do
	touch "${tagd}.src.rpm"
done

And since Koji seems to suck (no offence) at managing its sessions table, this script will clear out any expired sessions which are more than 10 minutes old so the table doesn’t become huge and slow.

exp.sh

#!/bin/bash
while true
do
        psql koji -c "select update_time from sessions where expired = 't';" | cat | grep -i '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]' | while read date
        do
                a=`date --date="$date" "+%s"`
                b=`date "+%s"`
                let c="($b - $a) / (60 * 10)"
                if [ $c -gt 0 ]
                then
                        echo "deleting [$date] exp [$c] from [`date`]"
                        psql koji -c "delete from sessions where expired = 't' and update_time = '$date';"
                fi
        done
        sleep 300
done
A new Koji instance for a new arch

A lesson in how / why companies hire & fire

Was doing some late-night Slashdot reading and came across an article submission which had a blog post about a facebook employee who had been fired. Although it seems like personal rant, there are some interesting points which give insight on how companies think about employees.

http://okdork.com/2012/09/29/why-i-got-fired-from-facebook-a-100-million-dollar-lesson/

Quoting:

“When you hire people there are three types of employees:

1- Grower. Someone who starts when the company is small and improves / adapts their skills as the company scales.

2- Show-er. Someone who can be good for the company where they are now but NOT where they are going.

3- Veteran. They’ve done it before and it’s second nature for them to teach you how to do it in your company.”

“Firing:

1- It stings the person WAY more than the company. I thought every day that the company missed me but I’ve learned they just keep going on with business. AND (UN)FORTUNATELY most businesses get better. So be stern when letting someone go but be reasonable and thoughtful to how it must feel. I encourage everyone to get fired once so they know that feeling. It’s unbelievable and something to definitely learn from.

2- EVERYONE is replaceable. You are NOT special and there is guaranteed someone better than you on this planet. So be the opposite, find the way to be invaluable where you work. This doesn’t mean locking things into you but opening things up so you are trusted and subsequently valued more.

3- Most people when they get let go, they know it’s time. They may not want to accept what their subconscious tells them but they know it’s right and it opens them up to something better. Instead of throwing them away, help guide / work with them to see what is their true calling and better suited for them.”

edit

I also came across another article which highlighted a video about motivation which I had posted before on this blog. It tries to explain why we do what we do in this open source world: http://www.youtube.com/watch?v=u6XAPnuFjJc ( http://www.ted.com/talks/dan_pink_on_motivation.html )

A lesson in how / why companies hire & fire

Bootstrapping Fedora ARM For A New Arch (v6hl)

Note: These steps assume you have a gcc compiler capable of producing the correct assembly instructions for the intended new arch

Step 1: Add the necessary architecture definitions

/usr/lib/rpm/platform/armv6hl-linux/macros

...
%optflags               -O2 -g -march=armv6 -mfloat-abi=hard -mfpu=vfp
%__isa_name             armv6hl
...

/usr/lib/rpm/{,redhat}/{rpmrc,macros}

...
optflags: armv6hl -O2 -g -march=armv6 -mfloat-abi=hard -mfpu=vfp
arch_canon:     armv6hl: armv6hl        12
buildarchtranslate: armv6hl: armv6hl
arch_compat: armv6hl: armv7hnl
buildarch_compat: armv6hl: armv7hnl
...
%arm    armv3l armv4b armv4l armv4tl armv5tel armv5tejl armv6l armv6hl armv7l armv7hl armv7hnl
...
optflags: armv6hl %{__global_cflags} -march=armv6 -mfloat-abi=hard -mfpu=vfp
buildarchtranslate: armv6hl: armv7hnl
...

/usr/lib/python2.7/site-packages/rpmUtils/arch.py

...
"armv6hl": "armv7hnl",
...

Step 2: Compile the latest version of gcc for your related arch using rpmbuild (--target armv6hl)

Step 3: Modify mock to hijack && inject these new files into all future build roots

/usr/sbin/mock

...
def do_rebuild(config_opts, chroot, srpms):
...
    chroot.init()

    ldst = (chroot.basedir+"/root"+"/usr/lib/rpm/platform")
    import shutil
    shutil.copytree("/usr/lib/rpm/platform/armv6hl-linux",ldst+"/armv6hl-linux")
    os.system("for f in /usr/lib/rpm/{,redhat}/{rpmrc,macros} ; do cp -fv \"$f\" \""+chroot.basedir+"/root/$f\" ; done")
    lcmd = ("/bin/rpm --force --ignorearch --nodeps --root "+chroot.basedir+"/root/ -i /repo/gcc-4.7.0-5.fc17/*.rpm")
    os.system(lcmd)

    chroot.build(srpm, timeout=config_opts['rpmbuild_timeout'])
...

Step 4: Build glibc and all other needed packages using mock (use a helper shell script loop && make sure to patch rpm/yum/redhat-rpm-config later)

/etc/mock/f17v6.cfg

...
config_opts['target_arch'] = 'armv6hl'
...

Step 4.5.0: rpm C code (unified diff patch)

rpm-x.x/lib/rpmrc.c

...
+#      if defined(__linux__) && defined(__arm__)
+       {
+           if (strcmp(un.machine, "armv7l") == 0 ) {
+               if (has_neon() && has_hfp())
+                    strcpy(un.machine, "armv7hnl");
+                else if (has_hfp())
+                    strcpy(un.machine, "armv7hl");
+           }
+           if (strcmp(un.machine, "armv6l") == 0 ) {
+               if (has_neon() && has_hfp())
+                    strcpy(un.machine, "armv6hnl");
+                else if (has_hfp())
+                    strcpy(un.machine, "armv6hl");
+               strcpy(un.machine, "armv6hl");
+           }
+       }
+#      endif   /* arm*-linux */
+
...

Step 4.5.1: yum Python script (mod)

yum-x.x/{yum/misc.py,rpmUtils/arch.py}

...
os.uname().replace("armv6l","armv6hl").replace("armv7l","armv6hl").replace("armv7hl","armv6hl")
...

Step 5: Install a core set of packages needed for a minimal boot (use a capable host/arch)

compose.sh

#!/bin/bash

if [ "$repo" == "" -o "$root" == "" ] ; then exit 1 ; fi

setenforce 0

for pkgn in 'acl' 'attr' 'audit' 'authconfig' 'basesystem' 'bash' 'binutil' 'bzip' 'ca-certificate' 'chkconfig' 'cpio' 'coreutil' 'cracklib' 'cryptsetup' 'curl' '.*curses' 'cyrus-sasl' 'db4' 'dbus' 'dbus-glib' 'diffutil' 'dracut' 'e2fsprog' 'elfutil' 'expat' 'fedora-release' '/file' 'findutil' 'fipscheck' 'gamin' 'gawk' 'gcc' 'gdbm' 'generic-release' 'glib' 'gmp' 'gnupg2' 'gpgme' 'grep' 'gzip' 'hardlink' 'hostname' 'hwdata' 'initscript' 'iproute' 'iptable' 'iputil' '.*kernel' 'keyutil' 'kmod' 'krb5' 'libassuan' 'libcap.*' 'libdb' 'libedit' 'libffi' 'libgcrypt' 'libgpg-error' 'libidn' 'libpwquality' 'libse' 'libssh2' 'libus' 'libut' 'libxml2' 'linux-atm' 'logrotate' 'lua' 'lvm2' 'mingetty' 'net-tool' 'newt' 'nspr' '/nss.*' '/openss' 'openldap' 'pam' 'passwd' 'pcre' 'pinentry' 'pkgconfig' 'popt' 'procps' 'psmisc' 'pth' 'pygpgme' 'python' 'python-chardet' 'python-iniparse' 'python-kitchen' 'python-pycurl' 'python-urlgrabber' 'readline' '/rpm' 'sed' '/setup' 'shadow-util' 'shared-mime-info' 'slang' 'sqlite' 'syslog' 'systemd' 'sysvinit' 'tcp_wrapper' 'texinfo' 'tzdata' 'udev' 'ustr' 'util-linux' 'xz' 'yum' 'yum-metadata-parser' 'yum-util' 'zlib' ; do fill=`find $repo -type f | grep -i "/[^/-]*$pkgn[^/-]*-[^/-]*-[^/-]*/[^/]*.rpm$" | grep -Eiv '(-debuginfo-|.src.rpm)'` ; if [ "$fill" == "" ] ; then echo "FOF $pkgn" 1>&2 ; continue ; fi ; echo "$fill" ; done | sort | uniq | grep -Eiv '/(audispd-plugins|audit-libs-devel|audit-libs-static|cmirror|cpp|cryptsetup-devel|cryptsetup-python|cyrus-sasl-sql|dbus-glib-devel|dracut-fips|emacs-libidn|gcc|glib2-devel|glib2-static|glibc-devel|glibc-headers|glibc-static|glibc-utils|gnupg2-smime|krb5-server|libcap-ng-devel|libgcrypt-devel|libgcj|libgfortran|libgudev1-devel|libitm|libmudflap|libselinux-python|libsemanage-python|libuser-devel|linux-atm-libs-devel|lvm2-cluster|newt-devel|newt-static|openldap-devel|openldap-servers|openldap-servers-sql|openssh-askpass|openssh-ldap|openssl-devel|openssl-perl|openssl-static|pam_|passwdqc|policycoreutils-python|policycoreutils-sandbox|python[0-9]*-debug|python[0-9]*-devel|python[0-9]*-test|python[0-9]*-tkinter|python[0-9]*-tools|rpm-build-[0-9]|rpm-cron|rpm-devel|rpm-sign|rpmdevtools|rpmlint|rsyslog-gnutls|rsyslog-libdbi|rsyslog-mysql|rsyslog-pgsql|rsyslog-relp|rsyslog-snmp|rsyslog-udpspoof|setuptool|slang-slsh|sqlite-tcl|systemd-analyze|texinfo-tex|tkinter|yum-cron|yum-plugin-changelog|yum-plugin-local|yum-plugin-puppetverify|yum-plugin-refresh-updatesd|yum-plugin-tmprepo)[^/-]*-[^/]*$' > pkglist.txt

#for p in $(rpm --force --ignorearch --root $root/ -i $(cat pkglist.txt | grep -Eiv '(gtk|gui|qt|x11)') 2>&1 | awk '{ print $1","$NF }' | sort | uniq) ; do a=$(echo "$p" | cut -d ',' -f 1) ; b=$(echo "$p" | cut -d ',' -f 2) ; q=$(repoquery --whatprovides "$a" | head -n 1) ; r=$(repoquery -qi "$q" | grep -i '^source') ; echo "[$b] -> [$a] -> [$q] -> [$r]" ; done
rpm --force --ignorearch --nodeps --root $root/ -i $(cat pkglist.txt | grep -Eiv '(gtk|gui|qt|x11)') x2

rm -frv $root/usr/share/*doc*

cat << EOF > $root/etc/fstab
LABEL="rootfs"          /                       ext4    defaults,noatime                 1 1
LABEL="boot"            /boot                   vfat    noauto,comment=systemd.automount 1 2
EOF

cat << EOF > $root/etc/sysconfig/network
HOSTNAME=raspiv6hl.local
EOF

cat << EOF > $root/etc/resolv.conf
nameserver 4.2.2.1
EOF

echo 'utmp:x:22:' >> $root/etc/group
echo 'PS1="[\u@\h: \w]# "' > $root/root/.bashrc
chmod 755 $root/root/.bashrc
ln -s .bashrc $root/root/.profile

#raspberrypi
sed -i 's|root:.*:|root:$6$KW0GGbE5$zlEB9.PbHVh8kmXj1WMFGLJGwwthhU4oXn2oNxHZllbUSzTsVhTZ9jts8RC7uicuUCWyrsZ1e2yEj4ErDLOHQ/:15525:0:99999:7:::|' $root/etc/shadow

rm -fv $root/etc/systemd/system/default.target && ln -s /usr/lib/systemd/system/multi-user.target $root/etc/systemd/system/default.target

Step 6: Boot the new Fedora ARM Linux system

Bootstrapping Fedora ARM For A New Arch (v6hl)