So I went to an interview recently and I was asked on the spot how I would implement an entire agile development platform, end-to-end, with specific details, (no requirements listed) – and it had to be cloud friendly/scalable! I was running different ideas through my head because it was a fairly open ended question and I felt like it deserved more thought than I could provide in just the 30 minutes given to me…

There are a dozen different ways this can be achieved but I wanted to post an example proof-of-concept of the essential steps needed to accomplish something like this. There are basically 5 different pieces to this puzzle that need to be solved:

1. The developers need a central repository with local branching capabilities so they can test different versions of their changes on demand
2. They need to be able to take an automatic snapshot of their code base that they are currently working on so they can specify which change they want to test
3. They need a simple mechanism, which when triggered, takes that code-change snapshot and sends it to a build server for compilation
4. The build server, when finished, should deploy and run their compiled test code, usually in a temporary, clean, isolated environment like a vm
5. The build server returns a simple and unique URL pointing to their new test instance so the developer can easily see and test their desired changes

Example Tech Env: GIT, SSH, NetCat, Bash Scripts, Java

Components Workflow:

• Developer (java)
• Central Repo (git/ssh)
• Automatic Snapshot Transfer (netcat)
• Automatic Building (javac)
• Automatic Deployment (bash script)
• Temporary Test Instance (netcat website)

 

[devops engineer setup]

git init
git add project
git commit -a

# setup initial supporting scripts

 

[developer]

# get the code
git clone ssh://jon@buildsrv:/home/git/projects

# make a change
cd project/
vim src/program.java
# @@ -11,5 +11,6 @@ public class program {
#  			System.out.println(x+": "+resList.get(x));
#  		}
# +		System.out.println("Changed!");
#  	}
#  }

./make.sh # magic!
# this script tar's up the current code and connects to the server to run an init.sh bootstrap script
# the init.sh script chooses an initial random port and calls the deploy.sh script
#   to perform the snapshot code transfer
# the deploy.sh script opens a netcat port to receive the snapshot'd code
#   and then runs build.sh on the result (port is closed after)
# the build.sh script opens a netcat port to communicate the build results
#   and then runs a http netcat instance of the executed code for the developer to test/visit

 

[automated build, deploy, test instance]

Extracting build...
./src/
./src/dep.java
./src/program.java
Extraction completed!

Building started...
Build completed!


Visit the build URL below:
http://192.168.161.2:29792/testing

[example web change test output]

Fri Nov 11 05:22:10 UTC 2016

0: [A, D]
1: [B, E]
2: [C, F]
Changed!

Completed!

 

[project file tree layout]

project/
|-- make.sh
|-- init.sh
|-- deploy.sh
|-- build.sh
`-- src
    |-- dep.java
    |-- lib
    `-- program.java

 

[script source code]

[make.sh]

#!/bin/bash
tar -czf snapshot.tgz ./src
USRN=`git config --get remote.origin.url | sed -e 's/@.*$//' -e 's@^.*/@@'`
REPO=`git config --get remote.origin.url | sed -e 's/^.*://'`
PROJ="project"
PORT=`ssh "${USRN}@buildsrv" "${REPO}/${PROJ}/init.sh"`
false
while [ $? -ne 0 ] ; do
	sleep 1
	nc buildsrv "$PORT" < snapshot.tgz
done
rm snapshot.tgz
let PORT="$PORT + 1"
false
while [ $? -ne 0 ] ; do
	sleep 1
	nc buildsrv "$PORT"
done

[init.sh]

#!/bin/bash

DIR=$(dirname "$0")
let PORT="1028 + ( $RANDOM % 64000 )"

nohup "$DIR/deploy.sh" "$PORT" > /dev/null 2> /dev/null < /dev/null &
echo "$PORT"

[deploy.sh]

#!/bin/bash

DIR=$(dirname "$0")
PORT="$1"

TMP="/tmp/build.`date '+%s'`"

mkdir -p "$TMP/"
nc -l -p "$PORT" -w 30 > "$TMP/src.tgz"

if [ -s "$TMP/src.tgz" ] ; then
	let PORT="$PORT + 1"
	"$DIR/build.sh" "$PORT" "$TMP" | nc -l -p "$PORT" -w 30 -q 1
fi

[build.sh]

#!/bin/bash

export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

let PORT="$1 + 1"
TMP="$2"

echo
echo "Extracting build..."
cd "$TMP/"
tar -xzvf "src.tgz"
cd src/
echo "Extraction completed!"
echo
echo "Building started..."
javac -d . dep.java 2>&1
javac -cp . program.java 2>&1
echo "Build completed!"
echo

GW=`netstat -nr | grep -i '^0.0.0.0' | awk '{ print $2 }' | cut -c 1-4`
IP=`ifconfig | grep -i "inet addr:$GW" | awk '{ print $2 }' | sed -e 's/^[^:]*://'`

echo
echo "Visit the build URL below:"
echo "http://${IP}:${PORT}/testing"
echo

( echo -e 'HTTP/1.1 200 OK\r\n\r\n' ; date ; echo ; java program ; echo ; echo 'Completed!' ) | nohup nc -l -p "$PORT" -w 60 -q 1 > /dev/null 2> /dev/null &

[src/dep.java]

package lib;
import java.util.*;
public class dep {
	public dep() {
	}
	public static ArrayList merge(ArrayList listA, ArrayList listB) {
		ArrayList outList = new ArrayList();
		for (int x = 0; x < listA.size(); ++x) {
			ArrayList tmpList = new ArrayList();
			tmpList.add(listA.get(x));
			tmpList.add(listB.get(x));
			outList.add(tmpList);
		}
		return outList;
	}
}

[src/program.java]

import java.util.*;
import lib.dep;
public class program {
	public static void main(String[] args) {
		ArrayList inA = new ArrayList();
		ArrayList inB = new ArrayList();
		inA.add("A"); inA.add("B"); inA.add("C");
		inB.add("D"); inB.add("E"); inB.add("F");
		dep help = new dep();
		ArrayList resList = help.merge(inA, inB);
		for (int x = 0; x < resList.size(); ++x) {
			System.out.println(x+": "+resList.get(x));
		}
	}
}