Igor’s scrawls

There’s a lot being written about Google’s collection of “private data” from WiFi networks using scanning equipment in its Street View cars. The Daily Beast says ”it’s not paranoia if Google is really snooping on you”, and that Google “collected private data from non-password protected Wi-Fi networks”; the Register informs us that “Google may have collected emails and other private information”; BoingBoing says that the search company ‘snooped’ “private data people sent over unencrypted wireless networks”; and on it goes.

Whatever you might think about Google, and whether or not you like the idea of the company holding data on you - let alone of its software as an automated arbiter of whether, for example, your face or car number-plate is correctly excised from the Street View maps - this réportage is disingenuous, and particularly disappointing because it’s coming from such generally solid sources. (I’m glad to note that Ars Technica’s coverage keeps it sensible.)

There’s been plenty of discussion recently of Facebook’s privacy policies, with luminaries like Danah Boyd writing lucidly about the issues there - which is only right - but it seems to be making for an atmosphere in which lazy journalists are playing on people’s reasonable concerns about their online privacy in order to make a big headline.

Of course, this is hardly the first time that that’s happened, and, equally, this isn’t the first instance in which Google’s approach to privacy has also been subject to scrutiny - and in some of those instances, found seriously wanting - but the way in which this particular episode is being presented serves merely to add sludge to already muddy waters, and these particular straits, treacherously complicated though they can seem, are important. The clue to the misrepresentation is in the terminology: “private data”, and “unencrypted”, “non-password protected” WiFi networks.

Obviously Google has a responsibility to ensure its software works properly and doesn’t compromise people. Obviously if it’s engaging in large-scale data collection it has a responsibility to ensure that such collection is done safely and respectfully. The reality of software engineering is that software is written by people, and people make mistakes, even in systems that are designed to look for mistakes made by people. (Obviously, it’s a shame for Google that they’ve opened themselves up through such a mistake to further criticisms about their privacy record, given recent events.)

This, however, is not the real issue. The real issue here is that technology journalists are writing stories implying that Google is secretly snooping on our private lives, on the basis that it’s been collecting information which people have been broadcasting, unencrypted, to the world at large.

This is new ground for most people, and answers to questions regarding whether Google - or indeed anyone with a WiFi card and some software - should be able to do this are not self-evident. It’s complex ground, too; the questions are not just related to technology but also ethics and, consequently, law. Articles such as those quoted above over-simplify, making unstated assumptions which aren’t apparent to many readers, and thus misrepresent this important material to exactly those people who most need to have it correctly represented.

The problem is that we’re diving head-first into a massively more complex information society, predicated on spiralling levels of technological complexity. This opens myriad issues in terms of privacy, data protection and, crucially, the comprehension of these issues by the people most affected by them. Like, if you give a shit about the security or privacy of your information, it’s down to you to take basic precautions like enabling authentication and encryption on your home network. (Let’s not even bother with the fact that the vast majority of these “private” emails make most of their transit in plain sight over the public Internet, encrypted home WiFi or not.) Clearly this makes the issues involved into big stories; technology writers know this, and take it upon themselves to inform their readerships about it. Which is as it should be.

The thing is, this is important, and the people writing about it have a responsibility to inform their readers in a level, even-handed way. If they focus instead on whatever makes the bigger story, because that sells more newspapers/magazines/ad impressions, then they do those readers as great a disservice as the companies about whom they monger their headline-grabbing scares.

Douglas Rushkoff talked compellingly at this year’s SXSW about his “Ten Commandments for a Digital Age”. The main thrust of his talk was that in this new information technology landscape, if we’re not to be completely manipulated by the biases of the technology involved, or that of the technologists who create it, we must either learn to manipulate those systems directly ourselves, or at the least we recognise that technology has biases, and is not neutral.

This clearly applies to previously existing media, as the technology journals are painfully demonstrating: the current wailing about Google’s data-gathering mechanisms seems a pretty clear example of how individual people need to learn to recognise those biases for themselves, because those who profess to inform them about the issues intrinsic to the technological advances are equally beholden to their own, pre-existing biases, amplified by scale and distribution in their new global context. Plus ça change, plus c’est la même chose.

It wasn’t just Osborne. There was an infinitely more malevolent, and manifestly less incompetent, presence on board. That’s right, it’s your friend and mine: THE PRINCE OF DARKNESS!

But will the old Bullingdonian’s erstwhile boating companion now take full advantage of this latest nepotistic opportunity - on the back of his hopeless friend’s extraordinary elevation at the hands of a hapless fate - for yet more unelected, avowedly non-partisan, portfolio-less rounds of sinister manipulation at the heart of morally bankrupt government? If so, you’ll be able only to:

» WATCH as his ruthless PFI agenda subsumes further swathes of public service money and control into the pockets of his Big Co. pals!

» SCREAM as, empowered by a notional “mandate” borne of an abortive election, he siphons off ever-increasing percentages of GDP into murky slush funds remote-controlled by corporate fraudsters and large-scale private criminals!

» DESPAIR as healthcare, transport, the Post Office - hell, whatever he can get his hands on - collapse into the grasping, silently merciless hands of international oligarchs, and Maggie’s grim forecast of a British society entirely unsponsored by government finally comes true!

MANDELSON. Coming soon to an opportunistic, mismatched, ethically compromised coalition near you.

This week it was announced that the Royal Mail privatisation was to be delayed until after the next election. All very well, but the last 30 years of brutal corporate hegemony seem to have left our economic, social and political intellectual landscape so ravaged that in spite of the grotesque plutocratic machinations of the recent “credit crunch”, “bail-outs” and “recession” (read: fiscal coup), the issues are frequently presented as party-political, policy-neutral electioneering, and apparently no-one even wants to consider the fundamental ideological issue here, which is (in my view unfortunately) a deeply unfashionable one: that public service exists to serve the public.

So, let’s try to get this straight. I’m going to get simplistic about this, because frankly that’s what we seem to need to do in order to get the point across. According to the privatisers’ mantra spun by the Prince of Darkness and his various little wizards, the Royal Mail was to be privatised because “it wasn’t working as a business”. Since when was it a “business”? It’s a public service. It exists to serve the public, not owners or shareholders. It’s OK to run it at a loss if need be. That’s why we pay tax, so that public service can serve the public. How complicated does this have to be?

“Ah, but it’s inefficient.” So? Fix it. “The only way to fix its inefficiency is to introduce competition, because profit is what motivates people to succeed.” Well, even if we take that logical leap of faith as gospel, fine: introduce a profit motive to incentivise individuals within an organisation, give them targets and objectives, get them to feel that there’s a personal point for them in striving to make the organisation work more effectively - but don’t introduce profit as a motivation for running a public service. It’s trivially obvious that it will lead to a reduction in services that don’t generate profits, which is not what a public service is about.

Transport, hospitals, post. If people insist on having private versions because they want to spend their hard-earned cash on extra bells and whistles, that’s fine - but there’s no reason to take it as a rationale for flogging the lot and ultimately removing the base level of service that a publicly-elected and -funded government has a responsibility to provide. If you’re going to outsource the whole machinery of state and reduce taxes to a bare minimum, then setting aside the politics and ideology, you’d at least have a consistent argument for emasculating the public sector to the benefit of private capital. If, however, you’re going to pose as a left-of-centre party and maintain any levels of taxation, then rationalising the sale of public services to that private capital is nothing more than a ruse, a larceny of the sort we stared goggle-eyed at in post-Soviet Russia’s collapse into lawless oligarchy.

New “Labour”. How dare you use the name.

I’d already got about halfway home this evening when I decided that enough was enough, and it was now raining hard enough to merit hailing a taxi to get the rest of the way. A couple of them just didn’t see me; a third slowed down and then, for some inexplicable reason, sped off apparently on seeing me. The fourth saw me, pulled over, and let me get in without even asking where I was headed. A good Samaritan, I thought.

We passed the 7 or 8 minutes’ ride in pleasant enough conversation - what did I do, had I been doing it long, what were the people like - until it came to the point about 500 yards from the drop-off opposite my home when he decided to drop the biggy.

“Quick question?”
“Yes?”
“Do you believe in God?”
“… No.”
“Did you see my sign?”
[ A quick look reveals a sign on the front side of the glass barrier saying “JESUS IS LORD. HE DIED FOR US.” ]
“Ah. No, I didn’t see that.”
“Well, do you know about Jesus?”
“I know a bit about him. Sharp guy.”
“Sharp guy, huh. Well, I think he was God, and he died for us and rose from the dead.”
“Do you.”
“Yes. I just thought I’d let you know that.”
“OK. So, £5.60? Here’s £7. Keep the change.”
“Thanks. I just thought I’d share that with you before you got out.”
“OK. Well, have a good night.”

So. Um. Hello? Is there anybody in there? If you’re going to proselytise, oughtn’t you to start a bit sooner? I mean, you left it a bit late there dude, all I had to was just get out of the cab; you didn’t even give yourself time to corner me into a circular theological dispute I can never win even if I choose to engage in it because evidence denies faith and yeah yeah yeah. I don’t think Jesus would have been too impressed. Although thinking about it, he probably would have forgiven you. Sigh. JESUS WIN

Republican senator Brad Sherman claims that several members of congress were threatened with martial law if they didn’t support the bank bailout bill:

Naomi Wolf offers her analysis of this event in light of others including the first deployment of US military on home turf since 1807 and the changes in the chain of military command implemented by Bush and Rumsfeld:

When someone like Wolf goes on record saying that “a coup”, “an armed insurrection” has taken place in the USA, and that its populace needs to “fight back” by organising, rising up, imprisoning the president and reclaiming control of the state, I don’t know about you, but my personal alarm bells start ringing.

I’ve been running PHP apps with the standard PHP FastCGI server behind nginx for a couple of years now, and in that time have worked up a set of tools to manage PHP processes with multiple configuration profiles. This has been based around my slightly hacked version of Alexey Kovyrin’s PHP–FCGI spawn script, along with a chkconfig–compatible init script I’ve written as a front–end to control it, taking in some simple per–profile configuration in /etc/sysconfig/php-fcgi.

Various contributors to the English nginx mailing list have posted in that time singing the praises of Andrei Nigmatulin’s php-fpm, a patch to the PHP source which adds “FastCGI Process Management” to the standard php-fcgi binary. It’s apparently in use in some pretty heavily–loaded sites, and I’ve had it in mind to check it out, but as my setup has been stable (if not exactly full–featured) over the last few projects, I haven’t had a huge impetus to get and do it. Today, however, I finally had enough downtime to check it out, and now I’m wishing I’d done it earlier.

My existing method basically worked like this:

· init script reads configuration file, with one “profile” per line detailing location of php.ini file, interface and port to run on, number of child processes to run, number of requests to serve before a re–spawn, etc.;

· init script uses these details to construct a command–line to call Alexey’s spawn script, once for each profile;

· spawn script constructs environment and command–line argument list to spawn a set of PHP processes for each profile entry.

This has worked fine, but as so often with these things, the init script became a bit unwieldy with additions over time, and is still unable to do anything elegant like graceful restarts on the PHP daemons.

php-fpm addresses these issues and more. The “profile” configuration is put into a sensible, clear configuration file (php-fpm.conf) which allows you to specify a number of named PHP process “pools”, each with its own detailed FastCGI server and PHP configuration.

The documentation’s somewhat light, and mostly in Russian, but it has all you need to get going, and the configuration file is easy to read. Once you’ve configured the pools you want (in my case sets of named dev/stage/live setups on different ports, so as to keep include_paths — and therefore library code — properly staged), you just need to run php-cgi --fpm. From there on, you can send various signals to the master process including SIGQUIT for a graceful stop, SIGUSR1 to cycle log files, and SIGUSR2 for a graceful reload/restart. The master process ID is stored in $PHP_PREFIX/logs/php-fpm.pid.

I’ll probably write a simple chkconfig/init wrapper to send these signals to the master using e.g. /etc/init.d/php-fpm graceful, but that’s about all I’ll need to do in order to replicate and extend my existing setup.

Not only does this simplify and tidy up my PHP–FCGI setup enormously, it also adds a number of convenient extra points, including IP–restriction and a nice fix for the “empty error” page problem. Intuitively it “feels” a lot more solid, and I’m looking forward to trying it out on the next suitable project. Nice bit of Russian coder humour there on the “extra points” page, too. Thanks, Andrei!

I just found myself in the situation of needing to install a load of software on a RHEL 4 box which had not had up2date set up. “Simple,” I thought, “it’s RPM-based, so just install yum and all will be well. yum’s nicer than up2date anyway, so”.

A cursory Google threw up this guide to doing just that, but I found that the list of RPMs provided was incomplete, possibly due to the age of the article. With the duplicates removed, package versions matched, and downloads sourced from the up-to-date CentOS and pbone, the set now installed without dependency problems, but left a non-functioning yum installation:

Setting up Install Process
Setting up repositories
not using ftp, http[s], or file for repos, skipping - Null is not a valid release or hasnt been released yet
Cannot find a valid baseurl for repo: update
Error: Cannot find a valid baseurl for repo: update

I hadn’t delved into yum’s config or repository setup much before, as on most non-RHEL rpm-based distributions it tends to work out of the box; I’d added other repositories, notably Dag Wieers’, but not looked at the format much. Imagine my delight on realising that now was my chance.

A first glance at the repository definition in /etc/yum.repos.d/CentOS-Base.repo suggested trying to use the baseurl entry which is commented out by default, rather than the mirrorlist. No joy, but it gave a pretty obvious clue:

http://mirror.centos.org/centos/Null/os/i386/repodata/repomd.xml: [Errno 14] HTTP Error 404: Not Found
Trying other mirror.
Cannot open/read repomd.xml file for repository: update
failure: repodata/repomd.xml: from update: [Errno 256] No more mirrors to try.
Error: failure: repodata/repomd.xml: from update: [Errno 256] No more mirrors to try.

Got it? That nasty Null is caused because the repository definition file uses a variable $releasever which, as man 5 yum.conf tells us, is taken from the currently installed version of the package named as distroverpkg in /etc/yum.conf - which by default is centos-release (presumably taken from the RHEL redhat-release convention). Thus the only step necessary to get it working was to install the centos-release package. The baseurl entry in the repository definition could be commented out again.

So here, for your edification, and as an »aide-memoire« for me, is my list of the packages required to get yum working correctly on RHEL4:

centos-release
centos-yumconf
python-elementtree
python-sqlite
python-urlgrabber
sqlite
yum
yum-metadata-parser

Joy.

Hot on the heels of Spot the Bull comes POKE’s next big campaign for Orange, Balloonacy. It’s a balloon race across Internet. No, really. It’s in its sign–up phase at the moment; the race itself starts on June 23rd.

You can launch a balloon or sign your site up to be part of the map, and if your balloon gets the furthest you can win a holiday in Ibiza.

It’s the biggest project we’ve built using our home–grown, bare–bones, remote–MVC framework, “Death Star”, and the first in which we’ve integrated it with AS3.

A lot of POKErs have been involved in this, including Iain who came up with the idea in the first place. Design is by Marc, Nicky and Dickon. In the client-side tech team, we have extreme Flash & pattern action by Dezza, with code & Papervision help from Gabes, and lovely Flash 9 sheen from POKE’s very own Caroline B. Stepping back from the Flash, we have front-end build and Death Star CMS delights from Greg; JS integration wizardry and Death Star coding from Mattias; Death Star coding, AMF services and database tomfoolery by Nilesh; system & platform architecture and team leading by mine own evil hand. Project managed by Mike who’s done his damnedest to keep us all sane. Ish.

For several years one of e. e. cummings’ selections of his poems had sat calmly on my bookshelf, unread and almost forgotten, but, unbeknownst to me, waiting patiently for life to prepare me sufficiently to grasp its content.

All the while I’d known nothing of cummings’ work save a few quirky, cheekily risqué pieces to which I’d been pointed over time, and had rashly assumed that they were what it was all about — so when, in a mood for some impish light-heartedness, I stuffed that slim volume into my pocket and dived into the first few stanzas on an empty eastward Saturday evening Tube, the strength of the undercurrent pulled me completely off course.

Two pieces in particular left me stunned into silent contemplation — who knows if the moon’s a balloon, conjuring in my mind Chagall’s eery, idealised yet somehow troubled world of airborne lovers (particularly Promenade and Lovers in the Red Sky); and Humanity i love you: a sly sucker-punch, an indulgent sigh of gentle resignation tailing finally to a hoarse rattle of anger and despair.

Here is an elegant and innocent beauty, a playful naïveté shot through with a mournful, quietly desolate seam of profound and mature comprehension, the understated expression compounding the contrast’s potency. An unexpectedly powerful discovery, for me, of an unheralded eloquence and grace.

There’s so much activity at the moment around web development using niche and previously esoteric languages that the recondite is fast becoming the norm: recherché, my dear, is the new accessible. One can’t help but wonder in some cases how far the developer of a given zoom–to–web framework has set out with some clear design goals in mind, carefully considered specific features of various languages in the context of those goals and eventually settled on one; conversely how far languages which natively support continuations, functional or concatenative programming are simply too cool not to have three–step web app frameworks, or indeed whether the implementation of such a framework has become a rite of passage for a langue débutante. For such considerations to stray into the realms of whether particular languages are selected purely on the basis of how arcane their syntax may be would be nothing less than cynicism, which is obviously untoward in technical discussion.

Nonetheless niche is sometimes nice, focus is frequently fantastic, and it goes (almost) without saying that a tightly–focussed development effort by some extremely smart people can, and often will, bear fine fruits. Recent ponderings in the Clark–brain have concerned The Right Way To Get The Job Done, and whether a “right” way might actually exist — more to follow on that topic, but for me, it’s all about that same ol’ focus, tools for tasks, and the Unix Way — so to that end, please find herewith a beauty parade of some of the linguistic and mechanical delicacies currently whetting my appetite, and lined up for perusal at some point in the not–too–distant.

Factor Stack–based programming definitely is cool.

Merb MVC framework Looks to do what it says on the tin, neatly, without any fuss, and hopefully without eating gigabytes of RAM for entrées.

Ragel state machine compiler If it’s good enough for Zed, it’s good enough for me.

Seaside If you want old–school cool with a nu–skool polish, it doesn’t come much better than Smalltalk — even if the Smalltalk community site does try pretty darn hard to convince you otherwise.

Finally, while we’re talking continuations in web frameworks (though post-AJAX Seaside seems to be sidelining them somewhat): even if the mere thought of a “full–stack web application framework with tools and APIs to implement most common web features” is enough to make my knees itch, using continuations but still maintaining control over HTTP nuts and bolts has a cool factor of at the very least Xe+28, and the recent burst of POJO–speak has re–lit my hitherto Bean–sickened and J2EE–weakened Java fire — all of which factors combine to suggest that RIFE is likely to get a look in.