Jekyll2020-07-24T10:07:56+00:00https://andybotting.com/feed.xmlAndy BottingMostly tech notesAndy Bottingandy@andybotting.comBuilding a new kernel for the Nexus Player2015-12-19T20:11:41+00:002015-12-19T20:11:41+00:00https://andybotting.com/building-a-new-kernel-for-the-nexus-player<p>I bought a cheap Micro USB (OTG) to USB hub with built-in Ethernet from Ebay for my Nexus Player. It seemed like the perfect way to make use of the single USB port available.</p>
<p><a href="/assets/images/micro-usb-ethernet-usb.jpg"><img class="size-medium wp-image-1209 alignnone" style="border: 0px;" alt="micro-usb-ethernet-usb" src="/assets/images/micro-usb-ethernet-usb-300x300.jpg" width="300" height="300" srcset="/assets/images/micro-usb-ethernet-usb-300x300.jpg 300w, /assets/images/micro-usb-ethernet-usb-150x150.jpg 150w, /assets/images/micro-usb-ethernet-usb.jpg 500w" sizes="(max-width: 300px) 100vw, 300px" /></a></p>
<p>Once I got it, I realised the Ethernet used a Davicom chipset and while it did have good Linux support with the dm9601 module, it wasn’t enabled in Android kernels.</p>
<p>As I started looking at guides for compiling kernels for Android, I found that they didn’t quite work properly for the Nexus Player.</p>
<p>Most Android devices are ARM based, but as the arch of the Nexus Player is x86, there’s some minor differences in some of the steps.</p>
<p>Here’s a quick run-down of the steps I did to simply add a new module to the Nexus Player kernel. I’m assuming that you’ve read a few of the more detailed guides or you’ve done some kernel building before.</p>
<p><strong>NOTE</strong>: You’ll need to go into OEM unlock on the device. You probably don’t need to be rooted, but I was. YMMV.</p>
<p>First step is to find the kernel version you’re currently running. Connect to the device with adb and run this in the shell:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>shell@fugu:/ $ cat /proc/version
</code></pre></div></div>
<p>We’ll need to get the git commit for the kernel.</p>
<p>In this case:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Linux version 3.10.20-g912890c
</code></pre></div></div>
<p>The kernel commit is the part after the ‘g’, so ‘<em>912890c</em>‘.</p>
<p>Depending on your Linux distro, you may want to pull Google’s toolchain. I’m using Arch Linux, which has GCC 5.3 and I had a build error, so instead, I just pulled the same toolchain that Google used for their production builds. If you want to match it, just look for the GCC version from the output above. E.g. ‘<em>gcc version 4.8</em>‘</p>
<p>In my case, I cloned the repository:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git clone https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.8
</code></pre></div></div>
<p>Add this to your path (substitute the $HOME/android part for your path)</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>export PATH="$PATH:$HOME/android/x86_64-linux-android-4.8/bin"
</code></pre></div></div>
<p>Add this CROSS_COMPILE variable to instruct the build to use this new toolchain. This is the prefix of the GCC binaries in the bin directory</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>export CROSS_COMPILE="x86_64-linux-android-"
</code></pre></div></div>
<p>Now we’ll clone the Kernel repository. For the Nexus Player (fugu) we’re using:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ git clone https://android.googlesource.com/kernel/x86_64.git fugu-kernel
</code></pre></div></div>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ cd fugu-kernel
</code></pre></div></div>
<p>We’ll create our own branch to work on, based on the last commit of our current kernel</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git checkout -b my-fugu-kernel 912890c
</code></pre></div></div>
<p>Let’s modify the kernel now</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ export ARCH=x86
$ make fugu_defconfig
$ make menuconfig
</code></pre></div></div>
<p>We can now make changes to our config.</p>
<p>Once you’re happy, we’ll build it using all our cores:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ make -j$(nproc)
</code></pre></div></div>
<p>Once you’re done, you’ll have a kernel image at <em>arch/x86_64/boot/bzImage</em></p>
<p>Now we’ll update the boot image to include our new kernel.</p>
<p>You’ll want to install the abootimg tool, and get a copy of the boot.img. Best to find it from the Nexus Player factory image, then tar and unzip.</p>
<p>We’ll update our factory boot.img and include our new kernel only:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ abootimg -u boot.img -k (kernel path)/arch/x86_64/boot/bzImage
reading kernel from (kernel path)/arch/x86_64/boot/bzImage
Writing Boot Image boot.img
</code></pre></div></div>
<p>Boot the Nexus Player into fastboot mode, and we’ll test our new kernel (before flashing)</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ fastboot boot boot.img
</code></pre></div></div>
<p>If you’re happy with it, then don’t forget to flash it:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ fastboot flash boot boot.img
</code></pre></div></div>
<p>Success!</p>
<p>For more information, I found this page really useful:</p>
<p><a href="http://softwarebakery.com/building-the-android-kernel-on-linux">http://softwarebakery.com/building-the-android-kernel-on-linux</a></p>
<p> </p>Andy BottingI bought a cheap Micro USB (OTG) to USB hub with built-in Ethernet from Ebay for my Nexus Player. It seemed like the perfect way to make use of the single USB port available.P2127 code on a Focus XR5/Focus ST2014-03-28T12:19:19+00:002014-03-28T12:19:19+00:00https://andybotting.com/p2127-code-on-a-focus-xr5focus-st<p>I had an issue with my 2007 Ford Focus XR5 (aka Focus ST) recently.</p>
<p>I started getting a ‘<strong>Steering Assist Failure</strong>‘ message appear occasionally when starting my car. Usually I could just turn if off, leave it for a little while then start again and it would be OK, and the car would be behave normally.</p>
<p>Shortly after I got an ‘<strong>Engine Systems Fault</strong>‘ message and the car entered ‘<strong>Low Acceleration Mode</strong>‘ to protect itself. Using my OBD-II adapter and the ‘Torque’ Android app, I found the specific error to be:</p>
<p style="padding-left: 30px;">
P2127 – Throttle/Pedal Position Sensor/Switch ‘E’ Circuit Low
</p>
<p>Using the magic powers of Google, I found <a href="http://www.fordstownerssa.co.za/forums/viewtopic.php?f=20&t=25953">this</a>:</p>
<p style="padding-left: 30px;">
“Finally the P2127 error code was sorted.
</p>
<p style="padding-left: 30px;">
When the car is started in the morning the voltage drops below 9v.
</p>
<p style="padding-left: 30px;">
9v is the threshold voltage for the all the ECU sensors, first sensor on my car that then schemes there’s k@k in the land is the throttle position sensor.”
</p>
<p>Using my very cheap multimeter, I tested the load across the battery. When I turned the key, sure enough, the voltage dropped from 12.43V to 8.3V, sending the throttle position sensor into an error state. Testing the same thing on Bek’s car showed it dropped to about 10.5V which is much healthier.</p>
<p>Once I installed a new battery, everything worked perfectly.</p>
<p>Hopefully this might help someone else, although I’m sure a different type of car would exhibit different symptoms.</p>Andy BottingI had an issue with my 2007 Ford Focus XR5 (aka Focus ST) recently.ZFS on Linux2013-10-07T13:16:47+00:002013-10-07T13:16:47+00:00https://andybotting.com/zfs-on-linux<p>ZFS is a fantastic filesystem developed by Sun. Compared to other filesystems, it’s quite interesting as it combines both a filesystem and a logical volume manager. This allows you to get great flexibility, features and performance. It supports things like integrated snapshots, native NFSv4 ACL support and clever data integrity checking.</p>
<p>I’m now running a HP ProLiant MicroServer N36L which is a small NAS unit containing a 4-bay SATA enclosure. It has a low-performance AMD CPU, and comes with 1GB RAM and a 250GB harddisk. I’ve upgraded mine to 4GB of RAM and 4 x 2TB Seagate Barracuda drives.</p>
<p>The benefit of these units are that they’re a standard x86 machine allowing you to easily install any OS you like. They’re also really cheap and often have cash-back promotions.</p>
<p>I bought mine when I was in the UK and I brought it back with me to Australia. I waited until I got back to upgrade it so save me the trouble of shipping the extra harddisks on the ship.</p>
<p>In this post, I’ll document how to easily install ZFS on Debian Wheezy and some basic ZFS commands you’ll need to get started.</p>
<h2 id="installation">Installation</h2>
<p><strong>UPDATE:</strong> ZFS on Linux now has their own Debian Wheezy repository! <a href="http://zfsonlinux.org/debian.html">http://zfsonlinux.org/debian.html</a></p>
<p>Install the ZFS packages</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># apt-get install debian-zfs
</code></pre></div></div>
<p>This should use DKMS to build some new modules specific to your running kernel and install all the required packages.</p>
<p>Pull the new module into the kernel</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># modprobe zfs
</code></pre></div></div>
<p>If all went well, you should see that spl and zfs have been loaded into the kernel.</p>
<p> </p>
<h2 id="prepare-disks">Prepare disks</h2>
<p>ZFS works best if you give it full access to your disks. I’m not going to run ZFS on my root filesystem, so this makes things much simpler.</p>
<p>Find our ZFS disks. We use the disk ID’s instead of the standard <em>/dev/sdX</em> naming because it’s more stable.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># ls /dev/disk/by-id/ata-*
lrwxrwxrwx 1 root root 9 Jan 21 19:18 /dev/disk/by-id/ata-ST2000DM001-1CH164_Z1E1GYH5 -> ../../sdd
lrwxrwxrwx 1 root root 9 Jan 21 08:55 /dev/disk/by-id/ata-ST2000DM001-9YN164_Z1E2ACRM -> ../../sda
lrwxrwxrwx 1 root root 9 Jan 21 08:55 /dev/disk/by-id/ata-ST2000DM001-9YN164_Z1F1SHN4 -> ../../sdb
</code></pre></div></div>
<p>Create partition tables on the disks so we can use them in a zpool:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># parted /dev/disk/by-id/ata-ST2000DM001-9YN164_Z1E2ACRM mklabel gpt
# parted /dev/disk/by-id/ata-ST2000DM001-9YN164_Z1F1SHN4 mklabel gpt
# parted /dev/disk/by-id/ata-ST2000DM001-1CH164_Z1E1GYH5 mklabel gpt
</code></pre></div></div>
<p> </p>
<h2 id="create-a-new-pool">Create a new pool</h2>
<p>ZFS uses the concept of pools in a similar way to how LVM would handle volume groups.</p>
<p>Create a pool called <strong>mypool</strong>, with the initial member being a RAIDZ composed of the remaining three drives.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># zpool create -m none -o ashift=12 mypool raidz /dev/disk/by-id/ata-ST2000DM001-1CH164_Z1E1GYH5/dev/disk/by-id/ata-ST2000DM001-9YN164_Z1E2ACRM/dev/disk/by-id/ata-ST2000DM001-9YN164_Z1F1SHN4
</code></pre></div></div>
<p>RAIDZ is a little like RAID-5. I’m using RAID-Z1, meaning that from a 3-disk pool, I can lose one disk while maintaining the data access.</p>
<p>NOTE: Unlike RAID, once you build your RAIDZ, you cannot add new individual disks.</p>
<p>The <strong>-m none</strong> means that we don’t want to specify a mount point for this pool yet.</p>
<p>The <strong>-o ashift=12</strong> forces ZFS to use 4K sectors instead of 512 byte sectors. Many new drives use 4K sectors, but lie to the OS about it for ‘compatability’ reasons. My first ZFS filesystem used the 512-byte sectors in the beginning, and I had shocking performance (~10Mb/s write).</p>
<p>See <a href="http://zfsonlinux.org/faq.html#HowDoesZFSonLinuxHandlesAdvacedFormatDrives">http://zfsonlinux.org/faq.html#HowDoesZFSonLinuxHandlesAdvacedFormatDrives</a> for more information about it.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
mypool 5.44T 1.26T 4.18T 23% 1.00x ONLINE -
</code></pre></div></div>
<p>Disable atime for a small I/O boost</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># zfs set atime=off mypool
</code></pre></div></div>
<p>Deduplication is probably not worth the CPU overheard on my NAS.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># zfs set dedup=off mypool
</code></pre></div></div>
<p>Our pool is now ready for use.</p>
<p> </p>
<h2 id="create-some-filesystems">Create some filesystems</h2>
<p>Create our documents filesystem, mount and share it by NFS</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># zfs create mypool/documents
# zfs set mountpoint=/mnt/documents mypool/documents
# zfs set sharenfs=on mypool/documents
</code></pre></div></div>
<p>Create our photos filesystem, mount and share it by NFS</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># zfs create mypool/photos
# zfs set mountpoint=/mnt/photos mypool/photos
# zfs set sharenfs=on mypool/photos
</code></pre></div></div>
<p>Photos are important, so keep two copies of them around</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># zfs set copies=2 mypool/photos
</code></pre></div></div>
<p>Documents are really important, so we’ll keep three copies of them on disk</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># zfs set copies=3 mypool/documents
</code></pre></div></div>
<p>Documents are mostly text, so we’‘ll compress them.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># zfs set compression=on mypool/documents
</code></pre></div></div>
<h2 id="scrub">Scrub</h2>
<p>ZFS pools should be scrubbed at least once a week. It helps balance the data across the disks in your pool and to fix up any data integrity errors it might find.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># zpool scrub <pool>
</code></pre></div></div>
<p>To do automatic scrubbing once a week, set the following line in your root crontab</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># crontab -e
...
30 19 * * 5 zpool scrub <pool>
...
</code></pre></div></div>
<p><strong>Coming soon</strong> is a follow-up to this post with some disk fail/recovery steps.</p>Andy BottingZFS is a fantastic filesystem developed by Sun. Compared to other filesystems, it’s quite interesting as it combines both a filesystem and a logical volume manager. This allows you to get great flexibility, features and performance. It supports things like integrated snapshots, native NFSv4 ACL support and clever data integrity checking.Puppet filebucketing fail with NFS2012-09-27T14:09:28+00:002012-09-27T14:09:28+00:00https://andybotting.com/puppet-filebucketing-fail-with-nfs<p>I’ve got back to Australia and I’m continuing my UK job from home.</p>
<p>So yesterday, I was doing some cleaning up and needed to unmount an NFS share and clean up its mount point directory.</p>
<p>You can see from the Puppet code below that I marked both resources as ‘absent’ to clean them up.</p>
<pre>file { '/tmp/install':
ensure => 'absent'
}
mount { '/tmp/install':
ensure => 'absent',
device => nfs-server:/install,
fstype => nfs
}</pre>
<p>This triggered Puppet to start filebucketing everything it could from the NFS share and subsequently filling up the root filesystem. I managed to revert my commit fairly quickly, but a large number of hosts in our infrastructure had already picked this up. This included both development and production systems.</p>
<p>There is an existing Puppet bug report about the issue at <a href="http://projects.puppetlabs.com/issues/2019">http://projects.puppetlabs.com/issues/2019</a></p>
<p>Apart from the obvious mistake that I should have just run this in a test environment first, this was totally unexpected behaviour.</p>
<p>A couple of things you could do to prevent this happening to you:</p>
<ol>
<li>Disable filebucketing either globally, or just for this file resource</li>
<li>Don’t try to remove the NFS mount and directory at the same time</li>
</ol>
<p>Hope this helps.</p>Andy BottingI’ve got back to Australia and I’m continuing my UK job from home.AFL plugins for XBMC2012-04-08T07:31:07+00:002012-04-08T07:31:07+00:00https://andybotting.com/afl-plugins-for-xbmc<p><strong>UPDATE: </strong>Plugins now updated and working for the 2013 season!</p>
<p>Whilst being in London, it’s been hard to get my AFL fix. So to keep up with what’s going on, I’ve created two new XBMC plugins: <strong>AFL Video</strong> and <strong>AFL Radio</strong>.</p>
<h2 id="afl-video">AFL Video</h2>
<p>You can browse all the latest videos from the AFL web site, including match replays, interviews and highlights.</p>
<div id="attachment_442" style="width: 310px" class="wp-caption alignnone">
<a href="/assets/images/xbmc-afl-video-plugin.png"><img class="size-medium wp-image-442 " title="xbmc-afl-video-plugin" alt="" src="/assets/images/xbmc-afl-video-plugin-300x182.png" width="300" height="182" srcset="/assets/images/xbmc-afl-video-plugin-300x182.png 300w, /assets/images/xbmc-afl-video-plugin.png 697w" sizes="(max-width: 300px) 100vw, 300px" /></a>
<p class="wp-caption-text">
AFL Video plugin
</p>
</div>
<p>You have a bunch of channels to choose from, including a team channel. The team channel will list the videos specific to your club.</p>
<div id="attachment_441" style="width: 310px" class="wp-caption alignnone">
<a href="/assets/images/xbmc-afl-video-list.png"><img class="size-medium wp-image-441 " title="xbmc-afl-video-list" alt="" src="/assets/images/xbmc-afl-video-list-300x182.png" width="300" height="182" srcset="/assets/images/xbmc-afl-video-list-300x182.png 300w, /assets/images/xbmc-afl-video-list.png 699w" sizes="(max-width: 300px) 100vw, 300px" /></a>
<p class="wp-caption-text">
Cats TV
</p>
</div>
<p>Match replays are usually available 12-24 hours after the match has been played.</p>
<h2 id="afl-radio">AFL Radio</h2>
<p>Unfortunately, you can’t watch the games live without some sort of paid subscription – so the radio streams are the next best thing.</p>
<div id="attachment_440" style="width: 310px" class="wp-caption alignnone">
<a href="/assets/images/xbmc-afl-radio-list.png"><img class="size-medium wp-image-440 " title="xbmc-afl-radio-list" alt="" src="/assets/images/xbmc-afl-radio-list-300x182.png" width="300" height="182" srcset="/assets/images/xbmc-afl-radio-list-300x182.png 300w, /assets/images/xbmc-afl-radio-list.png 696w" sizes="(max-width: 300px) 100vw, 300px" /></a>
<p class="wp-caption-text">
AFL Radio
</p>
</div>
<p>Just choose the stream you want and away it goes. I’m not sure how stable this will be long term due to how the stream works, but so far so good.</p>
<p>Interestingly, most of the streams work outside of the AFL game calls, but most of the streams are only 64k WMA, so the bit rate is a little low.</p>
<p> </p>
<h2 id="installation">Installation</h2>
<p>You can grab the latest ZIP files from the Github project download pages for <a href="http://code.google.com/p/xbmc-catchuptv-au/downloads/list?q=label:Plugin-AFL_Video">AFL Video</a> and <a href="http://code.google.com/p/xbmc-catchuptv-au/downloads/list?q=label:Plugin-AFL_Radio">AFL Radio</a>. You can then choose the ZIP files from the XBMC Addon <em>install from Zip file</em> menu option.</p>
<p>These will also be included in the <a href="https://code.google.com/p/xbmc-catchuptv-au/">AU CatchUp TV XBMC repository</a> too.</p>
<p> </p>
<h2 id="issues">Issues</h2>
<p>For any issues, please file a bug at <a href="https://github.com/andybotting/xbmc-addon-afl-video/issues">issue tracker</a> and please include a copy of your XBMC log file.</p>Andy BottingUPDATE: Plugins now updated and working for the 2013 season!AFL streaming radio from Linux2011-04-02T21:45:52+00:002011-04-02T21:45:52+00:00https://andybotting.com/afl-streaming-radio-from-linux<p>This is a big sarcastic thanks to AFL and Telstra for building the AFL web site in such a way that it only really works properly in Windows.</p>
<p>Being in London, I want to listen to the Geelong games over the streaming radio, but in Linux (and probably Mac), Silverlight just won’t cut it – and the radio fails to load with an error.</p>
<p>I did some digging around, and worked out the URL for the the streaming radio, which you can then plug into MPlayer to obtain the ASX stream:</p>
<p><code class="language-plaintext highlighter-rouge">mplayer -user-agent "NSPlayer/11.08.0005.0000" http://lon-cdn220-is-1.se.bptvlive.ngcdn.telstra.com/online-radio-afl_12</code></p>
<p>The code on the end is the stream ID. These are the station codes I’ve managed to work out:</p>
<ul>
<li>ABC774: 2</li>
<li>5AA Adelaide: 3</li>
<li>6PR Perth: 4</li>
<li>3AW Melbourne: 5</li>
<li>National Indigenous Radio Service: 6</li>
<li>Gold FM Gold Coast: 7</li>
<li>Triple M Sydney: 11</li>
<li>Triple M Melbourne: 12</li>
<li>Triple M Brisbane: 13</li>
<li>Triple M Adelaide: 14</li>
<li>K-Rock Geelong: 15</li>
</ul>
<p>I hope this proves useful to someone else.</p>
<p><strong>UPDATE: </strong>This has now been changed for the 2013 season. If you’re interested in listening to AFL radio on Linux/Mac/Windows, then try my XBMC <a href="http://www.andybotting.com/afl-plugins-for-xbmc" title="AFL plugins for XBMC">AFL Radio plugin</a>.</p>Andy BottingThis is a big sarcastic thanks to AFL and Telstra for building the AFL web site in such a way that it only really works properly in Windows.Using pkgutil on Solaris with Puppet for easy package management2010-12-10T03:18:10+00:002010-12-10T03:18:10+00:00https://andybotting.com/using-pkgutil-on-solaris-with-puppet-for-easy-package-management<p>I’ve been using <a href="http://www.puppetlabs.com">Puppet</a> on Linux systems for some time now, but I’ve only just started using it in a Solaris environment.</p>
<p>I think one of the killer functions of Puppet is being able to easily install packages and manage services on a system. Most Linux distros these days have tools for working with repositories of packages, like Yum on Fedora/RedHat/CentOS and Apt on Debian and Ubuntu. These work really well with Puppet, because you can easily script a class which requires a specific package, and Puppet will just call the package tool and it’ll install the right package and all of the required dependencies.</p>
<p>Using Solaris feels like a step back from Linux, not having an official repository tool like Yum and Apt. Its package system seems quite primitive which can suffer from the dependency hell that we used to have with RPM before it was wrapped up with Yum. Enter: <strong>pkgutil</strong>.</p>
<p><a href="http://pkgutil.wikidot.com/">Pkgutil</a> is like Yum for Solaris, written in Perl by Peter Bonivart. It was designed for <a href="http://www.opencsw.org/">OpenCSW</a>, which is a repository for Open Source packages on Solaris – and also the best place to install Puppet from. With a few simple steps, you can actually build an OpenCSW compatible repository of Solaris packages and tell pkgutil to use it, rather than the standard OpenCSW one.</p>
<p>Puppet has almost gained a proper package provider for Pkgutil (See <a href="http://projects.puppetlabs.com/issues/4258">Puppet issue #4258: Add pkgutil provider)</a>, which should be available in Puppet 2.6.4 maybe. In the mean time, we can just install it into our Ruby path to make use of it right now.</p>
<p>Steps involved are:</p>
<ul>
<li>Install pkgutil</li>
<li>Install Puppet on Solaris</li>
<li>Install the pkgutil provider</li>
<li>Build an OpenCSW-compatible repository of your own packages</li>
<li>Define pkgutil as a provder in your Puppet configuration</li>
<li>Install some packages!</li>
</ul>
<h2 id="install-pkgutil">Install pkgutil</h2>
<p>Before we do anything, we should install pkgutil. This handy one-liner will install it for Solaris 10 and OpenSolaris.</p>
<p><code class="language-plaintext highlighter-rouge"># pkgadd -d http://mirror.opencsw.org/opencsw/pkgutil-`uname -p`.pkg</code></p>
<p>For Solaris 8 and 9, take a look at the <a href="http://www.opencsw.org/get-it/pkgutil/">pkgutil installation page</a> for more details.</p>
<h2 id="install-puppet">Install Puppet</h2>
<p>Now that pkgutil is installed, installing Puppet is a breeze!</p>
<p><code class="language-plaintext highlighter-rouge"># /opt/csw/bin/pkgutil --install puppet</code></p>
<p>This will resolve all the dependencies and install everything just like the Linux package management tools do.</p>
<h2 id="install-the-pkgutil-provider">Install the pkgutil provider</h2>
<p>I’m using a version of pkgutil from Dominic Cleal’s git repository.</p>
<p><code class="language-plaintext highlighter-rouge"># wget --no-check-certificate https://github.com/domcleal/puppet/raw/143fc744a839affd328234fca26246d49d15d3d8/lib/puppet/provider/package/pkgutil.rb -O /opt/csw/lib/ruby/site_ruby/1.8/puppet/provider/package/pkgutil.rb</code></p>
<p>This wget will download it, and copy into the right place in the filesystem for Puppet to pick it up.</p>
<h2 id="build-an-opencsw-compatible-repository">Build an OpenCSW-compatible repository</h2>
<p>As part of OpenCSW, Peter Bonivart has released a tool for creating OpenCSW repositories, called <a href="http://pkgutil.wikidot.com/bldcat">bldcat</a>. You can find it as part of the pkgutilplus package from OpenCSW.</p>
<p>Create yourself a new directory for your packages on your webserver. For me, I needed OpenSolaris 2009.06 and Solaris 10 support, so:</p>
<p>`# mkdir -p repo/solaris/i386/5.11/<br /></p>
<h1 id="mkdir--p-reposolarisi386510">mkdir -p repo/solaris/i386/5.10/`</h1>
<p>Then just put all your packages into that directory, and run bldcat:</p>
<p><code class="language-plaintext highlighter-rouge"># bldcat .</code></p>
<p>This will generate the catalog, and descriptions file needed for pkgutil. Once you make this directory available by HTTP, you can add the URL into the pkgutil.conf file.</p>
<p>One thing to remember is that you’ll need to do this on a Solaris machine. Although bldcat will work on Linux, it requires some of the Solaris package tools, which won’t be available. For me, I just did it NFS mounted from a Linux server.</p>
<p>Now, set the mirror and noncsw entries like this:</p>
<p><code class="language-plaintext highlighter-rouge">mirror=http://repo.mydomain/repo/solaris<br />
noncsw=true</code></p>
<p>For my situation, I had to include a few packages that we provided as our standard environment, and the package names weren’t prefixed with CSW, to the ‘noncsw’ option needs to be set.</p>
<p>Because I wanted a mix of OpenCSW packages and our corporate standard packages, I copied in the OpenCSW packages (and dependencies) along with the corporate ones into the one repository. You can put Puppet in there also.</p>
<p>NOTE: All your packages need to be *.pkg.gz format, so make sure you compress any packages that aren’t already gzipped!</p>
<h2 id="define-pkgutil-as-a-provider-in-your-puppet-configuration">Define pkgutil as a provider in your Puppet configuration</h2>
<p>In the site.pp file on my Puppet Master, I include this definition:</p>
<pre>Package {
provider => $operatingsystem ? {
redhat => yum,
centos => yum,
sles => zypper,
solaris => pkgutil,
}
}
</pre>
<p>To see this in action, I’ve used Nagios’s NRPE as an example.</p>
<pre>package { nrpe_package:
name => $operatingsystem ? {
Solaris => 'CSWnrpe'
CentOS => 'nrpe',
SLES => 'nagios-nrpe',
Debian => 'nagios-nrpe-server',
},
ensure => installed,
}
</pre>
<p>So with pkgutil, installing packages on Solaris can be as easy as Linux with Puppet.</p>Andy BottingI’ve been using Puppet on Linux systems for some time now, but I’ve only just started using it in a Solaris environment.New Tram Hunter web site2010-10-09T21:12:08+00:002010-10-09T21:12:08+00:00https://andybotting.com/new-tram-hunter-web-site<p>I’ve been slowly doing some bits and pieces for a new Tram Hunter web site. I would now like to announce the new site at <a href="http://tramhunter.andybotting.com/">http://tramhunter.andybotting.com/</a>.</p>
<p>Since v0.5 of Tram Hunter, we’ve included an option to send anonymous usage statistics to a server I have running on Google’s App Engine. My main aim was to generate some heat maps, based on the location of tram stop requests.</p>
<p>You can now see the final version of the <a href="http://tramhunter.andybotting.com/stats/map/">heap map</a>, which is generated nightly, from the latest 1000 requests. It turns out to be quite interesting to look at.</p>
<p>I’m also using the Google Chart API to generate some nice <a href="http://tramhunter.andybotting.com/stats/chart/">pie charts</a> showing some other info like <a href="http://tramhunter.andybotting.com/stats/chart/device_model">handset model</a>, <a href="http://tramhunter.andybotting.com/stats/chart/device_version">Android version</a> and <a href="http://tramhunter.andybotting.com/stats/chart/mobile_network_number">mobile networks</a>.</p>
<p>In other Tram Hunter news, the latest stats from the Android Market show <span><strong>5687</strong></span> total installs, with <span><strong>4293</strong></span> active installs (<span>75%</span>). We also have a <strong>4.85</strong> rating out of 5, with <strong>255</strong> comments. The comments are all really positive, so it definitely makes development worthwhile.</p>
<p>I’ve created a new Twitter account for Tram Hunter, so for the latest updates, follow <a href="http://twitter.com/tram_hunter">@tram_hunter</a>.</p>Andy BottingI’ve been slowly doing some bits and pieces for a new Tram Hunter web site. I would now like to announce the new site at http://tramhunter.andybotting.com/.Tram Hunter: the blog post2010-09-14T19:02:00+00:002010-09-14T19:02:00+00:00https://andybotting.com/tram-hunter-the-blog-post<p>I think this post has been a <strong>long</strong> time in the making, but I thought it might be time to share this little story.</p>
<p><strong>Tram Hunter</strong> is a project I started nearly 2 years ago. It’s an Android client to the <a href="http://www.yarratrams.com.au">Yarra Trams</a> <strong>TramTracker</strong> web service, which their iPhone client leaverages to provide real-time tram arrival information to users of trams in Melbourne.</p>
<p>I’m not sure what it is about Trams, but I’m almost enchanted by them. They’re slow, many are really old and usually it’s a pretty rough ride, but they also have much more character than buses and trains.</p>
<p>A friend and I made a mashup of Google Maps with tram stops once, and using timetable information, we plotted approximated locations of trams along a line. The trams even moved along the line, although it wasn’t really realistic, it was fun to watch. I spoke to Yarra Trams about what we had done, and I was invited to come and see the Operations Centre in South Melbourne, which was quite interesting. They offered me a job working with their development team on some .NET/Windows web services stuff (which turned out to be the TramTracker service), but I just couldn’t leave <a href="http://www.vpac.org">VPAC</a> at the time.</p>
<div id="attachment_402" style="width: 210px" class="wp-caption alignnone">
<a href="/assets/images/tramhunter-stopdetails.png"><img class="size-medium wp-image-402" title="Tram Hunter Stop Details" src="/assets/images/tramhunter-stopdetails-200x300.png" alt="Tram Hunter Stop Details" width="200" height="300" srcset="/assets/images/tramhunter-stopdetails-200x300.png 200w, /assets/images/tramhunter-stopdetails.png 320w" sizes="(max-width: 200px) 100vw, 200px" /></a>
<p class="wp-caption-text">
Real Time Departures
</p>
</div>
<div id="attachment_401" style="width: 210px" class="wp-caption alignnone">
<a href="/assets/images/tramhunter-menu.png"><img class="size-medium wp-image-401" title="Tram Hunter Menu" src="/assets/images/tramhunter-menu-200x300.png" alt="Tram Hunter Menu" width="200" height="300" srcset="/assets/images/tramhunter-menu-200x300.png 200w, /assets/images/tramhunter-menu.png 320w" sizes="(max-width: 200px) 100vw, 200px" /></a>
<p class="wp-caption-text">
Application Menu
</p>
</div>
<p>So once Android was finally released, I bought their ADP1 development phone as quickly as I could. It cost a fortune, as the Australian dollar was quite weak at the time, but was pretty exciting. The idea of an Open Source phone to finally kick start some innovation in the mobile industry really appealed to me. I started messing with the Android API soon after.</p>
<p>I started working on Tram Hunter but got a bit stuck. I ended up shelving the project because I couldn’t figure out a problem I had, and moved on to other projects. It wasn’t until later (and I had moved to <em>London</em>), I was speaking to a friend of mine who was doing some Android development and he offered to help with the project. I proceeded to clean up the code, so it was in a compile-able state for someone else to look at. Somehow I managed to solve the issue and get something working. Everything seemed to just fall into place, and I had a working first version done.</p>
<p>I came across another project by accident by a couple of guys looking to do the same thing. I emailed them, and suddenly we had three developers and another joined soon after. I opened a <a href="http://tramhunter.googlecode.com">Google Code project</a>, put all our stuff into SVN and released version 0.1 to the Android Market. I later started a <a href="http://groups.google.com/group/tramhunter">Google Groups mailing list</a> for the project also.</p>
<p>The Tram Tracker iPhone application is slow and takes many taps to get to the information you want. Their interface has been designed to mimic the information screens at tram stops which is a nice idea, but actually provides an irritating user experience.</p>
<p>In comparison, the goal of Tram Hunter is to bring as many useful features as we can, without compromising the interface. I wanted to provide users the ability to get the information they want, with the least amount of clicks.</p>
<p>By using all the standard Android UI features, we gain a lot without needing to write a lot of code. Google Maps, location information by GPS, Network and Wifi, UI and search are all provided in the API so we don’t need to write this stuff ourselves. It also means it’s fast and simple.</p>
<p>Since the first version, we’ve introduced a few new features and have been fixing bugs. We’re on version 0.5 right now, and there’ll be a new one just around the corner.</p>
<p>The latest stats from the Android Market show <span><strong>4325</strong></span> total installs, with <span><strong>3128</strong></span> active installs (<span>72%</span>). Not bad considering the slow uptake of Android in Melbourne, and the limited number of tram users in Melbourne.</p>
<p>In version 0.4 of Tram Hunter, I introduced some code which (when only specifically enabled by the user) would send some usage information to a Google App Engine site I have set up. Tram Hunter will provide information about the user’s handset and Tram Hunter settings (e.g. What device is being used, what version of Tram Hunter is installed, which mobile network are we using, etc). It will also send information about which stops a user is requesting, and their location when they make the request.</p>
<p><a href="http://tramhunter.andybotting.com/stats/map/"><img class="size-medium wp-image-403" title="Melbourne Heat Map" src="/assets/images/screenshot-test-mozilla-firefox-300x234.png" alt="Melbourne Heat Map" width="300" height="234" srcset="/assets/images/screenshot-test-mozilla-firefox-300x234.png 300w, /assets/images/screenshot-test-mozilla-firefox.png 997w" sizes="(max-width: 300px) 100vw, 300px" /></a></p>
<p>I’m currently in the process of generating heat maps, to indicate the location of Tram Hunter requests. Unfortunately, the code isn’t finished so I can’t release them out in the open yet. I have some Google App Engine bit to sort out first, but I’ll be releasing all the interesting statistics to the Android community.</p>
<p><strong>UPDATE:</strong> The <a href="http://tramhunter.andybotting.com/stats/map/">heat map</a> is now running well on App Engine. The totally new <a href="http://tramhunter.andybotting.com/">Tram Hunter web site</a> is now up and running with lots of <a href="http://tramhunter.andybotting.com/stats/">cool graphs</a> and stuf.</p>
<h2 id="whats-next">What’s next?</h2>
<p>For Tram Hunter, I’m still taking feature requests and bug reports at our <a href="http://code.google.com/p/tramhunter/issues/list">issue tracker</a>, but I think development of this is starting to slow down.</p>
<p>I have been throwing around the possibility of porting it to Maemo/Meego to support the Nokia N900 (although something similar <a href="http://blogs.gnome.org/danni/2009/12/20/melbourne-tram-tracker-for-the-n900/">already exists</a>) and possibly to BlackBerry devices. BlackBerry also uses Java, so it should be quite easy to reuse a lot of code.</p>
<p>I’m also looking into developing another application for timetable information. I have had many requests for an app for buses and trains, so I’m looking to leaveraging some Google Transit code and proving users with an ability to download specially formatted timetables to their handset and use many of the features of Tram Hunter, but in an offline fashion. The idea is that it’ll be generic enough that it can be used for any type of timetable information for anywhere in the world, as long as people are willing to help port the timetable information.</p>Andy BottingI think this post has been a long time in the making, but I thought it might be time to share this little story.Using the Yubikey for two-factor authentication on Linux2010-05-20T22:47:30+00:002010-05-20T22:47:30+00:00https://andybotting.com/using-the-yubikey-for-two-factor-authentication-on-linux<p>The Yubikey is a nice little device. It’s quite simple in design and operation. <img class="alignright size-full wp-image-388" title="Yubikey" src="/assets/images/yubikey.jpg" alt="Yubikey" width="219" height="147" /></p>
<p>The key actually emulating a USB keyboard, which makes it instantly usable on any modern OS. You just press the button on the key to generate a one-time-password (OTP) to validate you. The method works by typing in your password, but before hitting the return key, you press the Yubikey button to finish it off. At the end of the OTP generation, it sends a carriage return itself.</p>
<p>The OTP is then sent to a validation server, either hosted by Yubico themselves, or you can host your own.</p>
<p>I’m going to walk through how you can set the infrastructre for doing two-factor authentication on Debian. In my specific case, the requirement was two-factor with an Active Directory username/password combination and the Yubikey as the second factor.</p>
<p>Unfortunately, the documentation from Yubico is quite average. To top it off, they insist on using multiple Google Code project sites for hosting their software.</p>
<p>This would normally be fine, but in this case, they have a Google Code project for every single little piece of code. Much of the documentation I found relates to older projects which are not supported by Yubico. This makes working out exactly what you need difficult. Within the Google Code project sites, documentation often runs in circles between projects.</p>
<p>In this document, I’ll look at using PAM to auth again the Yubico auth servers first. Once that’s working, I’ll move onto flashing the Yubikey with a new key and using our own Validation System.</p>
<p><strong>NOTE:</strong> This is just some rough notes I put together. You should definitely read the Yubico documentation for this to really make sense.</p>
<h1 id="authenticating-with-the-yubikey-with-pam">Authenticating with the Yubikey with PAM</h1>
<p>Get some dependencies</p>
<pre>apt-get install libpam-dev libcurl4-openssl-dev libpam-radius-auth</pre>
<p>Make ourselves a source directory</p>
<pre>mkdir ~/yubikey; cd ~/yubikey</pre>
<p>Get the current tarball of libyubikey, and install it</p>
<pre>wget http://yubico-c.googlecode.com/files/libyubikey-1.5.tar.gz
tar xf libyubikey-1.5.tar.gz
cd libyubikey-1.5
./configure
make check install</pre>
<p>Get the current tarball of the Yubico C client, and install it</p>
<pre>wget http://yubico-c-client.googlecode.com/files/ykclient-2.3.tar.gz
tar -xf ykclient-2.3.tar.gz
cd ykclient-2.3
./configure
make
make install</pre>
<p>Get the current tarball of the Yubico PAM module, and install it</p>
<pre>wget http://yubico-pam.googlecode.com/files/pam_yubico-2.3.tar.gz
tar -xf pam_yubico-2.3.tar.gz
cd pam_yubico-2.3
./configure
make
make install</pre>
<p>You should end up with your Yubico PAM module ‘/usr/local/lib/security/pam_yubico.so’</p>
<p>We’ll refer to this in our PAM config /etc/pam.d/openvpn</p>
<pre>#
# /etc/pam.d/openvpn - OpenVPN pam configuiration
#
# We fall back to the system default in /etc/pam.d/common-*
#
auth required /usr/local/lib/security/pam_yubico.so id=1 debug authfile=/etc/yubikeyid
auth required pam_radius_auth.so no_warn try_first_pass
@include common-account
@include common-password
@include common-session</pre>
<p>This configuration will tell PAM to hit the Yubico module first. This splits apart your password field into your password and OTP. The OTP is validated against the Validation Servers, and the password is then passed onto the next module. This configuration will use the Yubico auth servers to check your token.</p>
<p>Once you have a working config, we’ll move to setting up our own Validation Servers. We’ll need to specify the URL for that in this config later on.</p>
<p>In that case, we’re also using RADIUS. This could be LDAP if you had an LDAP server available. You should be able to use the standard UNIX credentials (/etc/password, /etc/shadow) also.</p>
<p>The other important piece to note here is the authfile, /etc/yubikeyid</p>
<p>This file lists the mapping between username and the fixed part of your Yubikey. This is the first 12 chars of the Yubikey OTP (e.g. when you press the button)</p>
<pre>abotting:vvcnrdkvevtj</pre>
<h1 id="freeradius-authenticating-against-active-directory-2008">FreeRADIUS authenticating against Active Directory 2008.</h1>
<p>I banged my head against a wall for a while on this one. The trick is that you need at least FreeRADIUS 2.1.6 for AD authentication to work properly.</p>
<p>Add Debian backports to your /etc/apt/sources.list</p>
<pre>deb http://www.backports.org/debian lenny-backports main contrib non-free</pre>
<p>Import the backports key</p>
<pre>wget -O - http://backports.org/debian/archive.key | apt-key add -</pre>
<p>Update and install the new freeradius</p>
<pre>apt-get update
apt-get -t lenny-backports install freeradius freeradius-ldap</pre>
<p>In your radiusd.conf</p>
<pre>ldap {
# Define the LDAP server and the base domain name
server = "ad.yourcompany.com"
basedn = "dc=ad, dc=yourcompany, dc=com"
# Active Directory doesn't allow for Anonymous Binding
identity = "ldap_bind_user@ad.yourcompany.com"
password = password
password_attribute = "userPassword"
filter = "(&(sAMAccountname=%{Stripped-User-Name:-%{User-Name}})(memberOf=CN=Users,DC=ad,DC=yourcompany,DC=com))"
# This fixes Active Directory 2008 access
chase_referrals = yes
rebind = yes
# The following are RADIUS defaults
start_tls = no
dictionary_mapping = ${raddbdir}/ldap.attrmap
ldap_connections_number = 5
timeout = 4
timelimit = 3
net_timeout = 1
}</pre>
<p>In our FreeRADIUS client file /etc/freeradius/clients.conf:</p>
<pre>client localhost {
ipaddr = 127.0.0.1
secret = testing123
nastype = other
}</pre>
<p>Use radtest to test our RADIUS is authenticating properly</p>
<pre>radtest <username> <password> localhost 1 testing123</pre>
<p>Should return Accept.</p>
<p>Set the address and shared secret of the radius server in <strong>/etc/pam_radius_auth.conf</strong>. The password of testing123 was defined in our RADIUS client config.</p>
<pre># server[:port] shared_secret timeout (s)
127.0.0.1 testing123 1</pre>
<p>OpenVPN has an issue with PAM loading the Yubikey module, so we have to LD_PRELOAD the pam module before starting OpenVPN.</p>
<pre>export LD_PRELOAD=/lib/libpam.so.0.81.12; openvpn --config openvpn.conf</pre>
<p>For a permanent fix, at the end of the start_vpn function in /etc/init.d/openvpn, just before the $DAEMON line:</p>
<pre>export LD_PRELOAD=/lib/libpam.so.0.81.12
$DAEMON $OPTARGS --writepid /var/run/openvpn.$NAME.pid \
$DAEMONARG $STATUSARG --cd $CONFIG_DIR \
--config $CONFIG_DIR/$NAME.conf || STATUS=1</pre>
<p>Change the path of /lib/libpam.so.0.81.12 to suit your own system.</p>
<p>I won’t go into the OpenVPN configuration, except that for PAM authentication you need these options in your server config:</p>
<pre>plugin /usr/lib/openvpn/openvpn-auth-pam.so openvpn
username-as-common-name
ns-cert-type server
client-cert-not-required</pre>
<h1 id="personalising-your-yubikey">Personalising your Yubikey</h1>
<p>To host your own Yubikey validation system, you require the secret AES key of your Yubikey. In the past, Yubico could provide this to you. Now, you’re required to flash your Yubikey yourself which will generate a new AES key.</p>
<p>Yubico provide a personalisation tool for Linux, Mac and Windows. If you’re on Windows, you get a nice little GUI. For Linux and Mac, you have a CLI based tool. It’s worth having a look at the ‘Personalization Tool’ page at: <a href="http://www.yubico.com/developers/personalization/">http://www.yubico.com/developers/personalization/</a></p>
<h2 id="installing-the-personalisation-tool">Installing the Personalisation Tool</h2>
<p>Install some dependencies:</p>
<pre>apt-get install libusb-1.0.0-dev</pre>
<p>Grab the latest Pesonalisation Tool tarball from: http://code.google.com/p/yubikey-personalization/</p>
<pre>cd ~/yubikey
wget http://yubico-c.googlecode.com/files/libyubikey-1.5.tar.gz</pre>
<p>Extract, build and install libyubikey</p>
<pre>tar xf libyubikey-1.5.tar.gz
cd libyubikey-1.5
./configure
make
make install</pre>
<p>You’ll need to provide a UID value for flashing your Yubikey. It needs to be 6 characters, and in hexadecimal. You can use this command to generate one for you.</p>
<pre>dd if=/dev/urandom of=/dev/stdout count=100 2>/dev/null | xargs -0 modhex | cut -c 1-10 | awk '{print "vv" $1}'
74657374696e</pre>
<p>You must provide the public name (fixed) parameter in modhex format. The modhex format is a special encoding used to ensure characters sent by the key are always correctly interpreted whatever keyboard layout you use.</p>
<p>You also need to generate yourself a public name for your key. This is known as the ‘fixed’ part, and it’ll be the first 16 chars when you generate your OTP. This will identify your key from anybody else’s.</p>
<pre>dd if=/dev/urandom of=/dev/stdout count=100 2>/dev/null | xargs -0 modhex | cut -c 1-10 | awk '{print "vv" $1}'
vvcnrdkvevtj</pre>
<p>This comamnd generate some random text, does a modhex operation, grabs the first 10 chars, then adds ‘vv’ to the front to make it up to 12.</p>
<p>You’ll be prompted for a passphrase on your AES key. I leave mine blank, but if you do set one, don’t ever lose it. I believe it’ll stop you from re-personalising your Yubikey.</p>
<pre>ykpersonalize -ouid=74657374696e -ofixed=vvcnrdkvevtj
Firmware version 2.1.2 Touch level 1793 Program sequence 1
Passphrase to create AES key:
Configuration data to be written to key configuration 1:
fixed: m:vvcnrdkvevtj
uid: h:74657374696e
key: h:fcaad309a20ne1809c2db2f7f0e8d6ea
acc_code: h:000000000000
ticket_flags: APPEND_CR
config_flags:
Commit? (y/n) [n]: y</pre>
<p>Save this information, as we’ll need it later.</p>
<h1 id="setting-up-yor-own-yubikey-otp-validation-server">Setting up yor own YubiKey OTP Validation Server</h1>
<p>You need to install two things: The Key Storage Module and the Yubico Validation Server. The Key Storage Module (KSM) holds the secret AES key of your Yubikey token, while the Validation Server does the OTP check against the KSM.</p>
<p>In their 2.0 architecture, you can have multiple KSM’s and Validation servers with work together for reduncancy.</p>
<h2 id="ksm-installation">KSM Installation</h2>
<p>Make a working directory, and get the KSM package</p>
<pre>mkdir ~/yubikey && cd ~/yubikey
wget http://yubikey-ksm.googlecode.com/files/yubikey-ksm-1.3.tgz
tar xfz yubikey-ksm-1.3.tgz</pre>
<p>Install the KSM files</p>
<pre>cd yubikey-ksm-1.3
make install</pre>
<h2 id="install-apache2-and-php">Install Apache2 and PHP</h2>
<p>Install Apache2, PHP and MySQL</p>
<pre>apt-get install apache2 php5 php5-mcrypt php5-curl mysql-server php5-mysql libdbd-mysql-perl</pre>
<p>Create the ykksm table</p>
<pre>echo "CREATE DATABASE ykksm;" | mysql -u root -p</pre>
<p>Import the DB schema</p>
<pre>mysql -u root -p ykksm < /usr/share/doc/ykksm/ykksm-db.sql</pre>
<p>Set up some MySQL permissions</p>
<pre>CREATE USER 'ykksmreader';
GRANT SELECT ON ykksm.yubikeys TO 'ykksmreader'@'localhost';
SET PASSWORD FOR 'ykksmreader'@'localhost' = PASSWORD('hYea3Inb');
CREATE USER 'ykksmimporter';
GRANT INSERT ON ykksm.yubikeys TO 'ykksmimporter'@'localhost';
SET PASSWORD FOR 'ykksmimporter'@'localhost' = PASSWORD('ikSab29');
FLUSH PRIVILEGES;</pre>
<h2 id="include-path-configuration">Include path configuration</h2>
<p>Set the include path by creating a file /etc/php5/conf.d/ykksm.ini</p>
<pre>cat > /etc/php5/conf.d/ykksm.ini << EOF
include_path = "/etc/ykksm:/usr/share/ykksm"
EOF</pre>
<p>Make a web server symlink</p>
<pre>make -f /usr/share/doc/ykksm/ykksm.mk symlink</pre>
<p>Set your configuration settings in /etc/ykksm/ykksm-config.php</p>
<pre><?php
$db_dsn = "mysql:dbname=ykksm;host=127.0.0.1";
$db_username = "ykksmreader";
$db_password = "hYe63Inb";
$db_options = array();
$logfacility = LOG_LOCAL0;
?></pre>
<p>Restart Apache2</p>
<pre>/etc/init.d/apache2 restart</pre>
<h2 id="test-the-ksm-server">Test the KSM Server</h2>
<p>Try this URL:</p>
<pre>curl 'http://localhost/wsapi/decrypt?otp=dteffujehknhfjbrjnlnldnhcujvddbikngjrtgh'
ERR Unknown yubikey</pre>
<p>It should return ‘Unknown Key’ until we have imported our Yubikey into the database.</p>
<h1 id="install-the-yubico-validation-server">Install the Yubico Validation Server</h1>
<p>The latest version, and documentation can be found at: <a href="http://code.google.com/p/yubikey-val-server-php/">http://code.google.com/p/yubikey-val-server-php/</a></p>
<h2 id="installation">Installation</h2>
<p>Go to our working source directory, and grab the package</p>
<pre>cd ~/yubikey
wget http://yubikey-val-server-php.googlecode.com/files/yubikey-val-2.4.tgz</pre>
<p>Extract, build and install the server</p>
<pre>tar -zxf yubikey-val-2.4.tgz
cd yubikey-val-2.4
make install</pre>
<p>Create the ykval database and import the schema</p>
<pre>echo 'create database ykval' | mysql -u root -p
mysql -u root -p ykval < /usr/share/doc/ykval/ykval-db.sql</pre>
<p>Install the symlink</p>
<pre>make symlink</pre>
<p>Include path configuration</p>
<pre>cat > /etc/default/ykval-queue << EOF
DAEMON_ARGS="/etc/ykval:/usr/share/ykval
EOF</pre>
<p>Create a htaccess file: /var/www/wsapi/2.0/.htaccess</p>
<pre>RewriteEngine on
RewriteRule ^([^/\.\?]+)(\?.*)?$ $1.php$2 [L]</pre>
<pre>php_value include_path ".:/etc/ykval:/usr/share/ykval"</pre>
<p>Symlink the htaccess file</p>
<pre>cd /var/www/wsapi; ln -s 2.0/.htaccess /var/www/wsapi/.htaccess</pre>
<p>Copy the template config file for the Validation Server</p>
<pre>cp /etc/ykval/ykval-config.php-template /etc/ykval/ykval-config.php</pre>
<p>Edit the file and configure settings in /etc/ykval/ykval-config.php</p>
<pre><?php
# For the validation interface.
$baseParams = array ();
$baseParams['__YKVAL_DB_DSN__'] = "mysql:dbname=ykval;host=127.0.0.1";
$baseParams['__YKVAL_DB_USER__'] = 'ykvalverifier';
$baseParams['__YKVAL_DB_PW__'] = 'password';
$baseParams['__YKVAL_DB_OPTIONS__'] = array();
# For the validation server sync
$baseParams['__YKVAL_SYNC_POOL__'] = array("http://localhost/wsapi/2.0/sync");
# An array of IP addresses allowed to issue sync requests
# NOTE: You must use IP addresses here.
$baseParams['__YKVAL_ALLOWED_SYNC_POOL__'] = array("127.0.0.1");
# Specify how often the sync daemon awakens
$baseParams['__YKVAL_SYNC_INTERVAL__'] = 10;
# Specify how long the sync daemon will wait for response
$baseParams['__YKVAL_SYNC_RESYNC_TIMEOUT__'] = 30;
# Specify how old entries in the database should be considered aborted attempts
$baseParams['__YKVAL_SYNC_OLD_LIMIT__'] = 10;
# These are settings for the validation server.
$baseParams['__YKVAL_SYNC_FAST_LEVEL__'] = 1;
$baseParams['__YKVAL_SYNC_SECURE_LEVEL__'] = 40;
$baseParams['__YKVAL_SYNC_DEFAULT_LEVEL__'] = 60;
$baseParams['__YKVAL_SYNC_DEFAULT_TIMEOUT__'] = 1;
// otp2ksmurls: Return array of YK-KSM URLs for decrypting OTP for
// CLIENT. The URLs must be fully qualified, i.e., contain the OTP
// itself.
function otp2ksmurls ($otp, $client) {
return array("http://localhost/wsapi/decrypt?otp=$otp",);
}
?></pre>
<p>In the above configuration, we’re only expecting to use one Validation Server and one KSM. If you’re planning on having multiple Validation servers and KSM’s, then you’ll be including the other Validation Servers in the SYNC_POOL, and your KSM’s in the URLs at the bottom, returned by the otp2ksmurls function.</p>
<p>Enable the mod_rewrite</p>
<pre>a2enmod rewrite</pre>
<p>Create the ykval database user</p>
<pre>CREATE USER 'ykvalverifier'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON `ykval`. * TO 'ykvalverifier'@'localhost';</pre>
<p>Fix some privileges on our config file</p>
<pre>chgrp www-data /etc/ykval/ykval-config.php</pre>
<p>The Sync Daemon uses the PEAR module System_Daemon so you need to install it:</p>
<pre>apt-get install php-pear
pear install System_Daemon-0.9.2</pre>
<p>Install the init.d script</p>
<pre>ykval-queue install
update-rc.d -f ykval-queue defaults</pre>
<p>Start the daemon</p>
<pre>/etc/init.d/ykval-queue start</pre>
<h2 id="testing">Testing</h2>
<p>Use CURL to test our server is working</p>
<pre>curl 'http://localhost/wsapi/verify?id=1&otp=vvcnrdkvevtefjbrjnlnldnhcujvddbikngjrtgh'</pre>
<p>It should return something like this:</p>
<pre>h=aPCQ4kWJilDgriyEii3j8J8lfuY=
t=2009-04-27T19:08:51Z0100
status=NO_SUCH_CLIENT</pre>
<p>Once we import our Yubikey into the database, we should get a nice ‘status=OK’ message.</p>
<h2 id="importing-your-keys-into-the-ksm-server">Importing your keys into the KSM server</h2>
<p>Refer back to the output from personalising your Yubikey. You’ll need the fixed part (referred to as publicname in the DB), internal name (UID) and our AES key.</p>
<p>This is an entry for our newly personalised Yubikey.</p>
<pre>USE ykksm;
INSERT INTO `yubikeys` (`serialnr`, `publicname`, `created`, `internalname`, `aeskey`, `lockcode`, `creator`, `active`, `hardware`)
VALUES (101209, 'vvcnrdkvevtj', '2010-05-07 15:18:40', '74657374696e', 'fcaad309a20ne1809c2db2f7f0e8d6ea', '000000000000', '', 1, 1);</pre>
<p>This entry is required for our systems to authenticate against the Validation server. I’m not exactly sure about this, as the documentation is somewhat bare. I think you need an administrator-type person’s key details in here. The imporant part is the ID. This values corresponds the the ‘id=’ value in our CURL requests and in our PAM config.</p>
<pre>USE ykval;
INSERT INTO `clients`
(`id`, `active`, `created`, `secret`, `email`, `notes`, `otp`)
VALUES
(1, 1, 1, 'fcaad309a20ne1809c2db2f7f0e8d6ea', 'your@email.addr', 'Any text your want', 'vvcnrdkvevterfbtelvnvkkueenecrlfnlhdjetrhgnk');</pre>
<p>We’ll hit our new Validation Server to make sure it’s working</p>
<pre>curl "http://localhost/wsapi/2.0/verify?id=1&nonce=askjdnvajsndjkasndvjsnad&otp=vvcnrdkvevtjkreuvvlhtubjecbrticjneckgrigkck"
h=KLEb3gOJ4KqQaCVbh8cEvXjH50U=</pre>
<p>It should return something like this:</p>
<pre>t=2010-05-20T11:24:53Z0051
otp=vvvcnrdkvevtjkreuvvlhtubjecbrticjneckgrigkck
nonce=askjdnvajsndjkasndvjsnad
sl=100
status=OK</pre>
<p>In this URL, we’ve added the ‘nonce’ parameter. This just a test to make sure the v2.0 API is working. ‘status=OK’ means it’s all good! If you get ‘NOT_ENOUGH_ANSWERS’, it means it has trouble trying to sync with other Validation Servers.</p>
<p>We’ll get PAM using our new Validation Servers for auth</p>
<p>/etc/pam.d/openvpn</p>
<pre>auth required /usr/local/lib/security/pam_yubico.so id=1 authfile=/etc/yubikeyid url=http://10.68.130.198/wsapi/verify?id=%d&otp=%s debug</pre>
<p>If you watch /var/log/auth.log, you should see the PAM module spitting out some debugging information which may be useful. It also spits out your plain text password too, while you have the debug option on. Make sure you remove this later.</p>
<h1 id="problems">Problems</h1>
<p>If you see an error like this:</p>
<pre>PAM unable to dlopen(/lib/security/pam_yubico.so): /lib/security/pam_yubico.so: undefined symbol: pam_set_data</pre>
<p>you’ll need the LD_PRELOAD trick from above. Something to do with dlopening the PAM module I believe.</p>Andy BottingThe Yubikey is a nice little device. It’s quite simple in design and operation.