Nov 17

fork()ing and fstat()ing in JRuby using FFI on linux

Sometimes, $DAYJOB can get kindof technical. For reasons I won’t go into here because NDA, the following axioms are true for this puzzle:

  • we have to work in JRuby
  • we are in a plugin within a larger framework providing a service
  • we have to restart the entire service
  • we don’t have a programmatic way to do so
  • we don’t want to rely on external artifacts and cron

Now, this isn’t the initial framing set of axioms you understand; this is what we’re facing into after a few weeks of trying everything else first.

So; obvious solution, system('/etc/init.d/ourService restart').
Except that JRuby doesn’t do system(). Or fork(), exec(), daemon(), or indeed any kind of process duplication I could find. Oh-kay, so we can write to a file, have a cronjob watch for the file and restart the service and delete the file if it finds it. Except that for Reasons (again, NDA), that’s not possible because we can’t rely on having access to cron on all platforms.

Okay. Can we cheat?

Well, yes… allegedly. We can use the Foreign Function Interface to bind to libc and access the functions behind JRuby’s back.

require 'ffi'

module Exec
   extend FFI::Library

   attach_function :my_exec, :execl, [:string, :string, :varargs], :int
   attach_function :fork, [], :int

vim1 = '/usr/bin/vim'
vim2 = 'vim'
if Exec.fork == 0
   Exec.my_exec vim1, vim2, :pointer, nil


Of course, I’m intending to kill the thing that fires this off, so a little more care is needed. For a start, it’s not vim I’m playing with. So…

module LibC
   extend FFI::Library

   ffi_lib FFI::Library::LIBC

   # Timespec struct datatype
      class Timespec < FFI::Struct
      layout :tv_sec, :time_t,
      :tv_nsec, :long

   # stat struct datatype
   # (see /usr/include/sys/stat.h and /usr/include/bits/stat.h)
   class Stat < FFI::Struct
      layout :st_dev, :dev_t,
             :st_ino, :ino_t,
             :st_nlink, :nlink_t,
             :st_mode, :mode_t,
             :st_uid, :uid_t,
             :st_gid, :gid_t,
             :__pad0, :int,
             :st_rdev, :dev_t,
             :st_size, :off_t,
             :st_blksize, :long,
             :st_blocks, :long,
             :st_atimespec, LibC::Timespec,
             :st_mtimespec, LibC::Timespec,
             :st_ctimespec, LibC::Timespec,
             :__unused0, :long,
             :__unused1, :long,
             :__unused2, :long,
             :__unused3, :long,
             :__unused4, :long

   # Filetype mask
   S_IFMT = 0o170000

   # File types.
   S_IFIFO = 0o010000
   S_IFCHR = 0o020000
   S_IFDIR = 0o040000
   S_IFBLK = 0o060000
   S_IFREG = 0o100000
   S_IFLNK = 0o120000
   S_IFSOCK = 0o140000

   attach_function :getpid, [], :pid_t
   attach_function :setsid, [], :pid_t
   attach_function :fork, [], :int
   attach_function :execl, [:string, :string, :string, :varargs], :int
   attach_function :chdir, [:string], :int
   attach_function :close, [:int], :int
   attach_function :fstat, :__fxstat, [:int, :int, :pointer], :int

So that’s bound a bunch of libc functions for use in JRuby. But why __fxstat() instead of fstat()? Interesting detail; the stat() function family aren’t in libc, at least not on most modern linux platforms. They’re in a small static library (libc_unshared.a in centOS). There’s usually a linker directive that makes that transparent but here we’re acting behind the scenes so we don’t get that niceity so we directly access the underlying xstat() functions instead.

I need to close some network ports (or the restart goes badly because the child process inherits the ports’ file descriptors and someone didn’t set them to close on exec()). A small helper function is useful here:

# Helper function to check if a file descriptor is a socket or not
def socket?(fd)
   # data structure to hold the stat_t data
   stat = LibC::Stat.new

   # JRuby's IO object types can't seem get a grip on fd's inherited from
   # another process correctly in a forked child process so we have
   # to FFI out to libc.
   rc = LibC.fstat(0, fd, stat.pointer)
   if rc == -1
      errno = FFI::LastError.error
      # Now we do some bit twiddling. In Octal, no less.
      filetype = stat[:st_mode] & LibC::S_IFMT

      if filetype == LibC::S_IFSOCK
rescue => e

And now the actual restart function itself:

def restart
   pid = LibC.getpid
   rc = LibC.chdir('/')
   if rc == -1
      errno = FFI::LastError.error
      return errno

   # close any open network sockets so the restart doesn't hang
   fds = Dir.entries("/proc/#{pid}/fd")
   fds.each do |fd|
      # skip . and .. which we pick up because of the /proc approach to
      # getting the list of file descriptors
      next if fd.to_i.zero?

      # skip any non-network socket file descriptors as they're not going to
      # cause us any issues and leaving them lets us log a little longer.
      next unless socket?(fd.to_i)

      # JRuby's IO objects can't get a handle on these fd's for some reason,
      # possibly because we're in a child process. So we use libc's close()
      rc = LibC.close(fd.to_i)
      next if rc.zero?
      errno = FFI::LastError.error
      return errno

   # We're now ready to fork and restart the service
   rc = LibC.fork
   if rc == -1
      # If fork() failed we're probably in a world of hurt
      errno = FFI::LastError.error
      return errno
   elsif rc.zero?
      # We are now the daemon. We can't hang about (thanks to 
      # JRuby's un-thread-safe nature) so we immediately swap out our 
      # process image with that of the service restart script. 
      # This marks the end of execution of this thread and there is no return.
      LibC.execl '/etc/init.d/ourService', 'ourService', 'restart', :pointer, nil
rescue => e
# Handle errors here (removed for clarity)

An interesting problem to solve, this one. And by “interesting” I mean “similar to learning how to pull teeth while only able to access the mouth via the nose”. But in case it’s of use to someone…

Nov 12

Using VimWiki as a distributed, encrypted lab notebook for programming

So what’s the most important coding tool for a software engineer in the long-term? There’s a pretty decent argument that it’s a lab notebook. I’ve kept one for years and it’s one of the more useful tools I have, but I found that my notebook, apart from not quite measuring up to how a lab notebook should be kept, wasn’t quite as useful as it could be because when you have five or six notebooks covering a bit over a decade’s worth of notes and no search function, then how do you use them daily?

I’ve been looking around for a while for a software version of a paper notebook without much success (this isn’t ludditism – we just don’t have very much that can match what paper and pen can do for this kind of task). I’ve used a simple text editor on a cron job for a few years as a way to track what I do during the day – every 30 minutes, up pops a vim window with a ready-inserted line listing the date and time, I scribble in what I’ve done since the last time the window popped up and I save and quit – it takes about 15 seconds and it gives me a file listing everything I’ve done at the end of the day/week/month/year which is handy for things like preparing reports, doing job reviews, that sort of thing. It’s simplicity itself to set up, just call a simple script using a simpler cronjob entry:

[cc escaped=”true” lang=”bash”]


# Cron script for 30-min activity journal

export DISPLAY=:0
echo -n -e “\n[” `date` “] :\n” >> ~/.journal
/usr/bin/gvim -U ~/.journal.gvimrc -geometry 100×40+512+400 + ~/.journal


[cc escaped=”true” lang=”bash”]

# m h dom mon dow command
0,30 8-20 * * mon-fri ~/.journal.sh


Simple and effective… but not a lab notebook. It does help test a few criteria for what I need that notebook program to do to fit in with my workflow though:

  • It has to be *fast* and familiar to use. ’nuff said, really. 
  • It has to be distributed. I work on lots of machines; I need it to be available on them, or else I’d have to have it on something like a netbook that I’d carry everywhere and that’s just not practical for me (besides, what if I forgot it or it got stolen?)
  • It has to be encrypted. Everything I work on is NDA’d at one level or another; and design/debugging notes would definitely be too sensitive to leave lying around in plaintext format. With a physical notebook, I keep it locked away; but this is supposed to be better than that option…

Colleagues have used various products for this over the years – mindmapping software; emacs in one mode or another; and various other software. But none of those really appealed. Mediawiki did seem to be as good a fit as I could find, but something that depends on an entire LAMP stack to run is hardly lightweight; and while I could host it somewhere public, that’s not really very secure (I’d spend more time making sure the full LAMP stack was up to date and mediawiki too than I want to). Besides, I’d rather this be console-accessible if possible (yes, some of us are still happier that way 🙂 ).

I’ve been using vim since around 1993 or so; at this point it’s wired into my fingers. So when I saw vimwiki, it seemed ideal. For those who’ve not encountered vimwiki before, it creates a directory, and then every file in that directory becomes part of a rudimentary text-based wiki (which it can turn into a set of HTML pages so it can handle images and so forth, but you can also navigate it from within vim). It also has a diary function which works in a sub-directory of the wiki directory.

It doesn’t have any support for encryption or distribution. But that’s quite solvable.

The encryption is easy enough – you could use the blowfish encryption in (post-v7.3) vim but that proved a bit awkward as you had to reenter the password every time you navigated down a link (and I don’t always have post-7.3 vim available). This password entering every minute or so broke up my workflow, so no thanks. My netbook and work laptops all have whole-disk-encryption, so I just left the vimwiki directory as normal on those laptops, and on the machines where I don’t have whole-disk-encryption, I use eCryptFS to create an encrypted directory and put the wiki under that. Very simple indeed, but quite effective. Now even theft of the physical hard drive isn’t a major concern.

The distribution was equally simple; you could use any DVCS, but I’m fond of mercurial, so I decided to use that. You have to tweak the vimwiki script ( .vim/ftplugin/vimwiki.vim ) to call it:

[cc escaped=”true” lang=”vim”]

augroup vimwiki
au! BufRead /home/mdennehy/vimwiki/index.wiki  !hg pull;hg update
au! BufWritePost /home/mdennehy/vimwiki/*  !hg add <afile>;hg commit -m ” “;hg push
augroup END


But that’s a simple tweak at best. And you want to have ssh setup with keys for the easiest workflow, but you have that already, right? 😀

Then just modify the crontab script:

[cc escaped=”true” lang=”bash”]

# Cron script for 30-min activity journal

export DISPLAY=:0
/usr/bin/gvim -U ~/.journal.gvimrc -geometry 100×40+512+400 -c “call vimwiki#diary#make_note(v:count1)” + -c “r !date +’\%n= \%H\%Mh =\%n'”[/cc]

And add an Awesome keybinding and menu entry:

[cc escaped=”true” lang=”lua”]

vimwiki_cmd = “/usr/bin/gvim -U /home/mdennehy/vimwiki/.gvimrc -c ‘call vimwiki#base#goto_index(v:count1)'”

mymainmenu = awful.menu({ items = { { “awesome”, myawesomemenu, beautiful.awesome_icon },

{ “VimWiki”,vimwiki_cmd }

awful.key({ modkey, }, “w”, function () awful.util.spawn(vimwiki_cmd) end),


And now whenever I hit <Mod4>-w from within Awesome, it pops up a gVim window open at the root of the wiki; every 30 minutes it pops up a gVim window in today’s diary page with the time inserted automatically for a log entry; and whenever I hit save or switch buffers, it syncs the files up to a central server’s encrypted area.

Distributed, encrypted, fast and useful. I’ve been using it in the job for the last few months now and it does almost everything I need. I do still keep around the paper notebook though – no matter how good the program, we still don’t have anything that can do everything paper can do (doodle, take cornell format notes, sketch diagrams easily for later capture, that sort of thing), but vimwiki’s search function alone is making it the day-to-day workhorse and it’s making my life a lot easier. Notes on development, patent ideas, job review reports, sysadmin notes, notes on papers I’m writing, and a daily log, all in one easy-to-use package. Damn useful tool.

Apr 10

Python on the Nokia N900

One of the really attractive things about the N900 is the possibility of customising it. You see, I’m a bit of a geek at times. A nerd, if you will. And the idea of being able to tweak the way my phone works appeals to me, whether it be in fixing a bug I find in an app or out-and-out writing a new one for something the manufacturer hadn’t though of (and I think a lot of the iPhone’s appeal to geeks is represented by that too, app store policy issues aside). But that’s not an option on symbian. First off, well, programming for symbian is fairly horrible; and secondly, you’d need the source code to the app which most companies aren’t going to provide without a rather large chunk of change.

But the N900 runs linux and therefore it should be really easy to code for (if you’ve written code on linux before). And since Python is one of the easier ways to write an application, I wondered how long it would take me to get up and running with Python on the N900, from nothing installed through to getting “Hello, World” running. So here’s how it went. Continue reading →