Archives

2008 (183)
2007 (276)
2006 (343)
2005 (56)
2004 (4)
2003 (8)
2002 (7)

Search

Type in this box and hit “enter” to search through all titles, content, authors, along with tags on bookmarks, photos, etc.

Searches for words which bring up more than half of all entries are ignored.

Tags

Top 30 tags I’ve added to blog posts and bookmarks.

beograd (15)
car (15)
code (48)
españa (24)
food (20)
geek (67)
holiday (17)
java (18)
kingsland (19)
kingsland road (22)
london (255)
madrid (16)
madrid march 2007 (16)
madríd (16)
olympic (15)
olympic torch (15)
olympic torch relay (15)
philosophy (28)
php (16)
programming (17)
railway (15)
road (24)
serbia (18)
shoreditch (70)
spain (24)
station (16)
street (47)
torch relay (15)
whitechapel (15)
whitechapel road (15)

About me

I live and work around Shoreditch, London. My obsession with making Internet stuff leads me to spend my days heading up the tech side at POKE. What you’re looking at is entirely my doing, though, and as you’ve probably guessed, in no way reflects POKE’s views on anything, at all, ever.

In addition to providing me with a soapbox, this site tracks what I’m up to online using feeds from Flickr, del.icio.us and others.

View my profile on LinkedIn

.

Regular reads

Sunday, October 5, 2008

Yaws

“Yet Another Web server”, this time written in the parallel/concurrency fans’ wünderkind, Erlang.

Tuesday, September 16, 2008

Shindig - an Apache incubator project for OpenSocial and gadgets

Apache-licensed open-source reference OpenSocial implementation
socialsite: Home

Developer tools, kits, materials for implementing OpenSocial gadgets (widgets) and working on OS implementations

Sunday, August 3, 2008

Sphinx - Free open-source SQL full-text search engine

Adds a searchable external index of specified data, meaning fast searches without db joins/FULLTEXTs etc.

Friday, July 25, 2008

Comet Daily » Blog Archive » Introducing HTTP Channels

“Comet solution for RESTful real time data synchronization architecture”. Change notification, syncing, etc.

Monday, July 14, 2008

php-fpm: a smoother PHP/FastCGI process manager

I’ve been running PHP apps with the standard PHP FastCGI server behind nginx for a couple of years now, and in that time have worked up a set of tools to manage PHP processes with multiple configuration profiles. This has been based around my slightly hacked version of Alexey Kovyrin’s PHP–FCGI spawn script, along with a chkconfig–compatible init script I’ve written as a front–end to control it, taking in some simple per–profile configuration in /etc/sysconfig/php-fcgi.

Various contributors to the English nginx mailing list have posted in that time singing the praises of Andrei Nigmatulin’s php-fpm, a patch to the PHP source which adds “FastCGI Process Management” to the standard php-fcgi binary. It’s apparently in use in some pretty heavily–loaded sites, and I’ve had it in mind to check it out, but as my setup has been stable (if not exactly full–featured) over the last few projects, I haven’t had a huge impetus to get and do it. Today, however, I finally had enough downtime to check it out, and now I’m wishing I’d done it earlier.

My existing method basically worked like this:

· init script reads configuration file, with one “profile” per line detailing location of php.ini file, interface and port to run on, number of child processes to run, number of requests to serve before a re–spawn, etc.;

· init script uses these details to construct a command–line to call Alexey’s spawn script, once for each profile;

· spawn script constructs environment and command–line argument list to spawn a set of PHP processes for each profile entry.

This has worked fine, but as so often with these things, the init script became a bit unwieldy with additions over time, and is still unable to do anything elegant like graceful restarts on the PHP daemons.

php-fpm addresses these issues and more. The “profile” configuration is put into a sensible, clear configuration file (php-fpm.conf) which allows you to specify a number of named PHP process “pools”, each with its own detailed FastCGI server and PHP configuration.

The documentation’s somewhat light, and mostly in Russian, but it has all you need to get going, and the configuration file is easy to read. Once you’ve configured the pools you want (in my case sets of named dev/stage/live setups on different ports, so as to keep include_paths — and therefore library code — properly staged), you just need to run php-cgi --fpm. From there on, you can send various signals to the master process including SIGQUIT for a graceful stop, SIGUSR1 to cycle log files, and SIGUSR2 for a graceful reload/restart. The master process ID is stored in $PHP_PREFIX/logs/php-fpm.pid.

I’ll probably write a simple chkconfig/init wrapper to send these signals to the master using e.g. /etc/init.d/php-fpm graceful, but that’s about all I’ll need to do in order to replicate and extend my existing setup.

Not only does this simplify and tidy up my PHP–FCGI setup enormously, it also adds a number of convenient extra points, including IP–restriction and a nice fix for the fix for the “empty error” page problem. Intuitively it “feels” a lot more solid, and I’m looking forward to trying it out on the next suitable project. Nice bit of Russian coder humour there on the “extra points” page, too. Thanks, Andrei!

Sunday, July 13, 2008

Setting up “yum” on RedHat Enterprise Linux

I just found myself in the situation of needing to install a load of software on a RHEL 4 box which had not had up2date set up. “Simple,” I thought, “it’s RPM-based, so just install yum and all will be well. yum’s nicer than up2date anyway, so”.

A cursory Google threw up this guide to doing just that, but I found that the list of RPMs provided was incomplete, possibly due to the age of the article. With the duplicates removed, package versions matched, and downloads sourced from the up-to-date CentOS and pbone, the set now installed without dependency problems, but left a non-functioning yum installation:

Setting up Install Process
Setting up repositories
not using ftp, http[s], or file for repos, skipping - Null is not a valid release or hasnt been released yet
Cannot find a valid baseurl for repo: update
Error: Cannot find a valid baseurl for repo: update

I hadn’t delved into yum’s config or repository setup much before, as on most non-RHEL rpm-based distributions it tends to work out of the box; I’d added other repositories, notably Dag Wieers’, but not looked at the format much. Imagine my delight on realising that now was my chance.

A first glance at the repository definition in /etc/yum.repos.d/CentOS-Base.repo suggested trying to use the baseurl entry which is commented out by default, rather than the mirrorlist. No joy, but it gave a pretty obvious clue:

http://mirror.centos.org/centos/Null/os/i386/repodata/repomd.xml: [Errno 14] HTTP Error 404: Not Found
Trying other mirror.
Cannot open/read repomd.xml file for repository: update
failure: repodata/repomd.xml: from update: [Errno 256] No more mirrors to try.
Error: failure: repodata/repomd.xml: from update: [Errno 256] No more mirrors to try.

Got it? That nasty Null is caused because the repository definition file uses a variable $releasever which, as man 5 yum.conf tells us, is taken from the currently installed version of the package named as distroverpkg in /etc/yum.conf - which by default is centos-release (presumably taken from the RHEL redhat-release convention). Thus the only step necessary to get it working was to install the centos-release package. The baseurl entry in the repository definition could be commented out again.

So here, for your edification, and as an »aide-memoire« for me, is my list of the packages required to get yum working correctly on RHEL4:

centos-release
centos-yumconf
python-elementtree
python-sqlite
python-urlgrabber
sqlite
yum
yum-metadata-parser

Joy.

Thursday, July 3, 2008

Emery Berger - The Flux Programming Language

High-level construction kit for gluing together discreet chunks of pre-built (in C) server functionality to make an optimised custom server application without having to deal with any concurrency issues. They've written real-world stuff with it.

Saturday, June 21, 2008

Cookies are for Closers: Oren Hurvitz’s Blog » LinkedIn Architecture

Discussion of LinkedIn's back-end architecture. Particularly interesting bit about in-memory data graph; also, apparently no ORM. Eh.

Monday, June 9, 2008

ESI - Edge Side Includes

Akamai initiative - simplistically, configurable fragment distribution including to CDN etc. Could be nice.