Graeme's Meandering analysis Thu, 16 Apr 2020 12:13:52 +0000 en-US hourly 1 Why we are never prepared for a crisis Thu, 16 Apr 2020 12:13:52 +0000 From 2005 Sri Lanka has been well prepared for a tsunami, unfortunately it was entirely unprepared in 2004. From 2021 onwards I have do doubt that the world will we well prepared to deal with a pandemic. We are always ready to fight the last war..

In the case of the COVID-19 pandemic we cannot claim it was unforeseen. Many people had warned of it, pandemics have occurred before and it was inevitable others would in the future. It was no black swan event.

There were tsunami warning systems, and people knew the 2004 tsunami was heading towards South Asia, but no one had given any thought to how to pass warnings to countries outside those involved in the warning system, and while they dithered, people died.

We will now be well prepared for a viral pandemic for the next few decades but there might not be another this serious for a century. Will we really stay prepared for that long?

In the meantime there are other things that may happen, and dealing with one disaster does not make us any better prepared for another. How well are we prepared for another Carrington Event, event for example? Very few people have even head of it, as a solar storm did little damage in 1859 so it was not a disaster then, but it would be one now, knocking out electricity, telecommunications, positioning systems like GPS, and a lot more. This is despite a near miss in 2012 which should have acted as a wake-up call.

Many of the changes in response to COVID are actually making us more vulnerable solar storms and the like. More working from home, greater use of electronic payments, more online shopping, and the general reliance on electronics which will simply stop working.

When we do successfully anticipate and prevent a disaster, many people regard it as a non-event – take the millennium bug, for example. Many people say “nothing happened, so it was a pointless scare”. Nothing happened because it was dealt with.

There are many other disasters we are unprepared for. Meteoroid strikes sound like something out of science fiction or prehistory, so they get filed as “not real”. They are real, and have occurred not much longer ago than the last killer pandemic. The Tunguska Event happened in Siberia in 1908, so, although it destroyed many tens of millions of trees, it did not kill many people. That was luck, it could just as easily have hit a densely populated area.

In Arthur C Clarke’s book 2001, a meteoroid defense system was built after Venice was destroyed by one. Sadly, that is probably what it will take to persuade humanity to prepare for it. In the meantime we are doing very little.

Space weather is on the UK’s “National Risk Register Of Civil Emergencies” but without public discussion and scrutiny it is impossible to know what, if anything, is being done to help us cope. There are no serious plans anywhere to deal with meteoroids, not even real warning systems. These are threats far worse than COVID – they really could be the end of the world, or at least civilization and most human life – but lets “cross that bridge when we get to it”, and its far too late.

]]> 0
Climate treaties cheat the environment. Fri, 13 Jul 2018 11:06:35 +0000 Climate treaties suffer from a problem that is pervasive in our society. It is the same problem that is destroying British state schools, makes public sector out-sourcing fail, and cripples businesses. Once you set a numerical target, the metric becomes more important that what it measures.A good example of the problem is the EU’s pushing of diesel cars. This has, thankfully, been reversed, but it remains a good example of bad consequences.  There are still a far greater number  diesel cars on the road than there otherwise would have been and this is not a problem that is easily solved.

I have a diesel car. We do low annual mileage and much of our driving is through open country,  so I am pretty sure the environmental costs of scrapping it and replacing it with a new vehicle would massively outweigh any gains from running a cleaner car. I suspect that eventually the government will encourage people to scrap regardless of this: because it helps meet the numbers.

The British government defended the policy on the grounds that “it was not known” at the time that diesel was bad? The EU did not know about diesel particulates? Really?

The EU still requires that diesel contains a proportion of bio-diesel. This, again, meets the treaty targets because it is a renewable. The fact that we are chopping down forests to achieve this. In case it is not obvious this policy is that the forests that  remove CO2 from the atmosphere are being cut down, and we are causing localised climate and other environmental damage, and  we are worsening the already terrible mass extinction.

The British governments subsidises the Drax power plant in Yorkshire because it uses “renewable” wood instead of dirty coal. This, once again, means chopping down priceless forests and increasing CO2 emissions and using the only fuel that is actually dirty than coal. Once again, the underlying problem is that the metric is out of touch with reality, and the metric is what matters because it has become a treaty obligation.

There are two political problems:

  1. The need to have fixed targets, which requires a metric, and a metric will never reflect the reality of as complex a problem as this.
  2. The influence on government policy of vested interests such a businesses that have products to sell to solve the problem.

The latter problem is worth expanding on, Have you ever wondered why there is so much focus or reducing the use of fossil fuels, and so little on preserving the forests that are vital to removing CO2 from the atmosphere? The answer is simple. There is money to be made in selling new power plants, new cars, and all the rest. A lot of money. There is no money to be made in not doing something. Guess which the world’s invariably “business friendly” governments prefer?

Not just a CO2 shortage – the economy is broken Thu, 12 Jul 2018 09:27:19 +0000 Shortages happen. A shortage of a gas that is vital to the manufacture of everything from beer to pain killers may look like just another unfortunate occurrence, but it is really a product of the way a “neo-liberal” economy works: globalisation and centralisation.Big businesses centralise production in a small number of plants, this means that unfortunate timing, a small number of closures happening at the same time has a large effect on the supply. If we had a large number of smaller plants, one of my suggestions in How to Fix Capitalism,  then it would be statistically highly unlikely that the same proportion of production would coincidentally shut down at the same time.

Globalisation is to blame for two reasons. Firstly it encourages centralisation. Secondly it encourages imports which makes it harder for anyone to anticipate or plan for the problem and cuts domestic production even further. It makes central planning (which does not have a great track record) even harder.

The reason we have this system is supposedly the pursuit of efficiency, and it is possible that prices of ammonia (of which CO2 is a by-product) and CO2 are lower as a result. I have some doubts (there are other motives for scaling up, and there is plenty of evidence that agency conflicts and the advantage to managers is the real motive), but we can leave that aside for now.

A clearer problem is that the costs of the shortage are widely spread out across the economy, rather than falling on those who operate the plants. In other words, the risk and reality of shortages are externalities.

Combine externalities with the market fundamentalist ideology of the “invisible hand” and the, again ideological, taking of homo economicus not just a useful simplification in economic models but as a guide to reasonable, even moral, behaviour, and we have all the ingredients for the problem. It is not the manufacturers duty to even warn of the problem, let alone try to ameliorate it. If people are expected to have a selfish motivation, what duty is expected to customers, let alone wider society?

So, in summary, if we had lots of small firms, instead of a few big ones, we would avoid this sort of problem. If we had a bit less free trade, it would help as well. Finally, if we had different ideologies and attitudes, we could plan for it. However, these are not three separate issues: they are all products of the market fundamentalist/neo-liberal/pro-business (call it what you will) ideology.

Django signals are evil Thu, 12 Oct 2017 14:20:52 +0000 I was trying to figure out what a Django app was doing today. It turned out that the original developer had decided to monkey patch a third party app. I hardly need say that monkey patching is evil (i.e. a last resort), but one of the things I needed to check along the way was that there was no code being triggered by a (Django) signal, and the problems it causes are very similar to monkey patching.The problem with signals is that it is not obvious to someone else reading your code what is going on. I had a similar problem a few weeks ago when I found a signal was already doing the same as code I was writing in a view. In both cases it added a lot of extra time (proportionate to the fairly simple problems I was fixing) to work out what was going on.

The is exactly the same as the worst of the problems caused by monkey patching. It makes code less readable. A signal could be anywhere in a project, and it could do anything. You look at a view and think you know what it does, but there could be code anywhere that does something you do not know about. Readability counts.

Of course monkey patching is worse. For one thing the risk of incompatibilities break on upgrade is much higher: signals use a well defined API, whereas monkey patching is entirely unpredictable and may change anything.

What are the alternatives to signals? You can often put the code somewhere else. In fact you can always put the code somewhere else if you are willing to fork when signals come from other people’s code. For model signals you can usually use the save method instead, for others you may be able to to subclass or wrap views.

Like monkey patching signals should be used reluctantly, because the alternative is even worse.


Trump’s immigration policy compared to the UK’s Sun, 05 Feb 2017 12:27:17 +0000 There has been near universal condemnation of Trump’s immigration policies, but it seems to be that they simply do in one stroke what most European countries have done incrementally. The UK is a fairly typical European country in this respect so lets see how it compares.

This is a rough summary, and I welcome corrections, but the overall picture is pretty accurate.

Trump UK
All no Syrian refugees in Allow some in after previously admitting virtually none
Prevent people likely to be refugees from boarding flights Prevent people likely to be refugees from boarding flights
Allow in no-one from seven unstable and violent countries Make entry extremely difficult for anyone from most of the world
Might change rules for highly skilled workers – no one knows in what way yet It is already very difficult for highly skilled workers to get visas
Scapegoats immigrants and panders to Islamaphobes Scapegoats immigrants and panders to racists
Easy visas for rich people, even if they are violent criminals Easy visas for rich people, even if they are violent criminals

What is the difference in motives of end results? The main difference is one of rhetoric. Trump has said, in that past, that he wants to keep out Muslims, British politicians use more subtle language, but they make sure that racists are reassured that immigration will be reduced.

AirBNB hosts can block bad reviews – and my AirBNB horror story Fri, 03 Feb 2017 11:42:51 +0000 If an AirBNB “host” needs to give a refund because the accommodation was terrible, they can block the bad review by cancelling instead of refunding. It does mean a 100% refund, but that is clearly preferable to bad reviews and AirBNB have confirmed that this is deliberate.

Some people have pointed out that the same thing is possible on sites like eBay – but I think holiday accommodation is different. You cannot return the product for a full refund, and the precious days of your holiday are gone forever.

The way I found out was, of course, a booking that went really badly.  We needed three rooms (me,, my wife, our two daughters and a friend) in Nuwara Eliya, in Sri Lanka’s mountains, so we thought booking a house rather than our usual place made sense, so I used AirBNB for the first time. I found a lovely looking bungalow that had everything we wanted.

We arrived late at night after a long drive – eight hours including some stops, and driving on winding mountain roads is tiring. As we arrived at night it was far to late to find somewhere else

The first problem cropped up even before we got there. Even though the listing says breakfast is provided, the manager asked us to buy things for breakfast. The listing said that a cook was available, and that was one of our main criteria in choosing accommodation as we had a lot to pack into three days we would be there. As we were taking our bags in we asked the caretaker about the cook, and he told us there was no cook, but he could cook breakfast (more on that later!).

The listing still says the accommodation is provided on a bead and breakfast basis, and that a cook is available.

Then we found that the place was filthy: the bathrooms had not been cleaned,  there was litter under beds and cigarette butts in a laundry basket..We ate an Indian takeaway and thought we would get a good nights sleep and sort it all out in the morning.

We then got the next little surprise which was a rat scuttling across the floor. Needless to say it scared the kids. I slammed a room door shut to prevent it getting into a bedroom, and we put the kids in their room with instructions not to open the door until we had chased the rat out of the house. We could not find it so we decided to go to sleep anyway.

We phoned the manager , and he promised that both he and the owner (the latter is the “host” in the AirBNB system) would come there at 9am the following morning and sort out the problems.

When everyone else had gone to bed, I decided to have a shower. Each room had an en-suite bathroom with a separate water heater and I soon discovered that the one in our room did not work. I did not want to disturb the kids or our friend, so I went to sleep without a shower, which I hate. However, I did not hate that as much as the reason the hot water was not working:

Socker hanging off wall above shower

Yes, this is a socket hanging out of the wall, in the bathroom, right above the shower head. I had thought the floor level electric bar heater in the bedroom, crammed between the bed and the floor length curtains was a hazard, but this was even worse. That said, it was the heater that nearly started a fire because my wife pushed cushion (one of those neck support ones) off the bed so that it fell right next to it.

Come next morning, the caretakers ability to cook did not stretch as far as frying eggs, and as we wanted fried eggs we cooked them ourselves. We made sure we were ready at 9am, but an hour later neither the manager or Somerset (who gets named after a county?), the owner of the property, had turned up, and neither was answering phone calls.

We would have been rather stuck at this point, but while I sent messages through AirBNB, my wife managed to work out that the manager and the host were both employees of the near by Hill Club. This is a beautiful private club founded by British planters in the 19th century. One clue was that we discovered that the laundry was done there, so we had immaculately clean sheets and towels in contrast to the rest of the place.

So, when no one turned up at the house, we turned up at the hill club and asked for the two of them. After much prevaricating, and claims by the girl at reception that she could not find them, the manager turned up. In the meantime a little audience of staff were watching discreetly as we hung around. It possibly helped that we bumped into a member my wife knew.

We were then promised that Someset was on his way to the house so we returned to wait for him. I rather suspect he was at the club at left after us, as both of them turned up together by car.

In the meantime I read the reply on AirBnB. Somerset simply refused to believe our complaints, saying the rat and the dirt were “not possible”, and claiming that the listing had been “rectified long ago” with regard to the cook and bed and breakfast (see the screenshot above) and ending “IT IS OBVIOUS THAT YOU ARE WANTING TO CUT SHORT YOUR STAY IN NUWARA ELIYA AND HENCE ARE CREATING THESE ISSUES“.

He got out of the car, and his first words were to tell us that we had no business looking for him at the club. I suspect it would have been embarrassing for him if we did, but I will skip the reason why as it would be unnecessarily vindictive to way any more publicly.

He carried on ranting and was downright insulting. He told us that he would give us a full refund as he did not want money from “people like you”. In the meantime I had complained to AirBNB and sent them photos proving the issues with regard to dirt and electrical safety/

While all this was happening I had contacted AirBNB, send them photographs of the problems (not the missing cook or breakfast – it is hard to photograph an absence of cook). I did find them very helpful at this point..

Having got the refund agreed, and the evidence gathered in case he failed to deliver it, we booked ourselves into the nearby Gold Club to which I belong for the rest of the stay in Nuwara Eliya. Somerset calmed down a bit before we left and his tone was rather more conciliatory, though far from apologetic.

The next bit of fun started when I tried to leave a review. I found that AirBNB’s system had left an automatic review saying the host had cancelled the booking. That meant I could not leave review. After exchanging 14 emails with them, in the course of which I was told I could leave a review, I was eventually told this:

The fact of the matter is that because this reservation was cancelled by your host neither of you should be able to leave a review and the system has reverted to that. It is not an error, it is the normal functioning of our system.

A subsequent email explained their reasoning:

With regards to your point about hosts being incentivised to cancel instead of having reviews on their profile, there isn’t actually an incentive to do that as when a host cancels they receive an automated review saying that the cancellation was made and they also fully refund the guest and may incur financial penalties for cancelling as well as other account related penalties.

I am not convinced. The fact is that there is a mechanism that allows hosts to avoid the very worst reviews, and the financial penalties are limited if they are refunding all or most of the cost of the booking. Regardless of that, the end result is that the reviews are biased because hosts do cancel and thereby replace what would have been the very worst reviews with bland cancellation notices (and I cannot even see that in this case).

In spite of this, I received another reminder to leave a review from AirBNB – containing a review link that is redirected to a message telling me I cannot “edit this review”.

A disturbing post-imperial flaw in the British psyche Wed, 22 Jun 2016 09:59:28 +0000 I could almost subtitle this “why the Brexit debate turned nasty”, although the problem is far older and deeper than anything it directly deals with. It is something I never realised, possibly because I am not British by birth, until I realised that people who vehemently disagreed with each other shared a common, mistaken, assumption.

The assumption is this. That the UK is a small and unimportant country. You hear this from people opposing immigration because Britain is a “small over-crowded island”, and you hear it from people who advocate remaining in the EU because they regard the UK as too small to be viable.

By any objective measure this is false:. It has the 22nd largest population in a world with over 200 countries and territories. More importantly, the UK is the world’s fifth largest economy. The UK is not in the same league as the US economically, or a handful of huge countries by population, but it is comparable to just about anywhere else.

I have never heard a Sri Lankan complain their country is over-crowded (a smaller and poorer country with a higher population density). Singapore seems to have done quite nicely as a small country. Australians seem to feel no great need seek to be part of a political union. Of course many countries form military alliances or trade treaties, but very few seek political union.

I have never heard anyone in the UK refer to other countries with a similar population as small, only their own country. Neither have I heard anyone outside the UK refer to it as a small and unimportant country. If it was, would the impact of Brexit on the EU be such a problem?

I can think of two hypotheses to explanation this. The first, and I think the more likely, is that the British psyche has never adjusted to being just another country, rather than part of the world’s largest political union. The other is that the UK is so dominated by the US view of the world, that simply being smaller and less powerful than the US is unacceptable.

This explains both the attraction of the EU, and the dislike of it. It explains the UK’s poodle role with regard to the US. Some people are not content with not playing a leading role in the EU, as in the empire. Others seek to be an adjunct to the most successful new empire (the US).

Then consider the EU. It is in many ways an imperial project, forming a political union covering many nations and cultures and melding them into one: the most ambitious such project since the Roman Empire. Looked at this way, the EU, however dysfunctional, is a comfort blanket for those who feel unsafe outside an empire.

What has the EU ever done for us? Sun, 05 Jun 2016 10:15:59 +0000 What has the EU ever done for us? Mostly, a lot of harm.

Consistently favoured corporate interests over public interests

The EU is far more insulated from public pressure than national governments but even more prone to listening to corporate lobbyists. It is not transparent enough about lobbying for us to even know what the real expenditure is, but we know it a lot.
The greatest single case of this is in the supposedly free trade, but really corporate welfare, TTIP treaty which the EU is pushing hard, despite widespread public opposition. Who wants the treaty? Corporate lobbyists. It also binds countries permanently to particular policies, undermining democracy.

Made the financial crisis a lot worse, and that is just the start

The Euro ties many, very diverse economies, into a single interest rate. Without full political union (e.g. the EU cannot directly raise tax, and there are limits on its spending power) means the EU cannot quickly take money away from countries where Euro rates are too low and give it to countries where rates are too high. This weakness is what lead to the crisis in countries like Greece.

Most single currency areas are countries, and that redistribution happens and it largely happens automatically (regions that are doing well pay more tax, regions that are doing badly get more welfare). The EU has a currency union without the political union that is needed to make it work.

The UK was lucky to be outside the Euro, which reduced the damage – but it still suffered some indirect damage. It also faces the prospect of this happening again and again, and on top of that EU policy will increasingly be set by the needs of the Euro area – so the UK can stay out of the Euro and have unsuitable policies, or join and get the full blow of each crisis. A great choice.

Helped crooked politicians and paedophiles keep their past secret

The EU’s right to be forgotten (part of EU law, not human rights under the entirely separate European Convention) has allowed people with unsavoury pasts to remove references to their offences on the web. In the case of the politician this undermines democracy (because information about how a candidate behaved in office is hidden from voters) and in all cases in undermines free speech.

What is even better is that people outside the EU can find this information.

Imposed stupid and harmful regulation

Those in favour of the EU point to tabloid examples of bad regulation that turn out to false. However, the fact is that the EU does create a lot of harmful, and down right stupid, regulation. Two that I have dealt with is the cookie law and VATMOSS.

The cookie law was supposed to protect privacy. With a very few exceptions, the best it ever achieves is that sites pop up notices that tell you that if you continue using the site you agree to their cookie policy.

The only way sites can track whether or not you have agreed is to set a cookie, so anyone who turns cookies off, or only allows selected sites to set cookies, gets bombarded with notices on every page or every EU based site. This is deterrent to protecting your privacy with things like Cookie Whitelist. I have also notices a sharp increase in the sites I use that will not work without cookies since the cookie law was passed.

It has also encouraged tracking without cookies, by using browser fingerprinting, which is much harder to protect against.

VATMOSS is even worse. It is more expensive to comply with, particularly for small businesses that are below the VAT in their home countries, who now have to register for VAT if they make a single sale (of affected products) to another EU country. Some small business have stopped selling digital products to other EU countries, others have closed or faced huge costs. Here is the story of a company that relocated from the UK to Singapore as a result.

In short, the EU’s regulations seem to achieve the opposite of the stated purpose of the EU. It undermines trade, and complete fails to understand small business or technology.

Made books, and a lot of other things, more expensive and less easily available

The EU requires a minimum copyright duration of life + 70 years. This is great news if you are a publisher, or a successful author’s or painter’s grandchild who wants a pension boost. For the rest of us, it makes books and other works, more expensive, and it also means that anything that remains in extended copyright that it is not commercially attractive to publish (e.g. will sell in large numbers) will simply be unavailable, as free ebook distributors will be unable to give away copies.

The EU’s excuse for this was that it would harmonise copyright laws, so we could freely trade copyright works across the EU without worrying about legal differences. The odd thing is that it did not make life + 70 a fixed copyright duration, but a minimum, so countries like the UK and France can have extended copyright on some works, so the law is not actually harmonised.

I will not discuss the ludicrousness of a duration that means works created while Queen Victoria was on the throne remain in copyright in the 21st century (for example, many of Shaw’s works were written then, and remain in copyright until 2020).

Caused floods

The EUs counter arguments are that recent changes (“greening” measures) to the regulation mean that this is either no longer true, or less true, or and harm is now down to how national governments implement things. However all these relate to changes in the last few years, ignoring the damage done over many decades.

Taylor Swift on Apple Music – translated Mon, 22 Jun 2015 05:41:21 +0000 As Taylor Swift seems to have difficult saying what she means, so I thought I would provide a plain English translation of what she has to say about Apple Music.

Here is the original, but its all quoted below.

I write this to explain why I’ll be holding back my album, 1989, from the new streaming service, Apple Music. I feel this deserves an explanation because Apple has been and will continue to be one of my best partners in selling music and creating ways for me to connect with my fans.

I make lots of money from iTunes, and this could be profitable too, so I had better do enough grovelling first to let me change my mind if I have to.

I respect the company and the truly ingenious minds that have created a legacy based on innovation and pushing the right boundaries.

They are really good at marketing, just like me.

I’m sure you are aware that Apple Music will be offering a free 3 month trial to anyone who signs up for the service. I’m not sure you know that Apple Music will not be paying writers, producers, or artists for those three months.

It is shocking that anything new in the music industry does not make even more money out of fans.

I find it to be shocking, disappointing, and completely unlike this historically progressive and generous company.

Apple is really good at extracting money from consumers, so we want a slice.

This is not about me. Thankfully I am on my fifth album and can support myself, my band, crew, and entire management team by playing live shows.

This is entirely about me, but people are horribly unsympathetic when a super-rich 25 year old complains she is not making enough money.

This is about the new artist or band that has just released their first single and will not be paid for its success. This is about the young songwriter who just got his or her first cut and thought that the royalties from that would get them out of debt.

That is assuming that lots of people listen to them on Apple Music, and no one buys downloads or CDs or anything, and assuming unknowns people actually make lots of money from their first single. Its not like this is an industry where a few stars make all the money of something!

This is about the producer who works tirelessly to innovate and create,

A lot of my rich and successful friends and colleagues will not make enough money out of this either.

just like the innovators and creators at Apple are pioneering in their field…but will not get paid for a quarter of a year’s worth of plays on his or her songs.

I think a bit more sucking up here will make me sound better.

These are not the complaints of a spoiled, petulant child. These are the echoed sentiments of every artist, writer and producer in my social circles who are afraid to speak up publicly because we admire and respect Apple so much

These are the complaints of a spoiled, petulant bunch of rich people who want more money, but are scared of upsetting Apple too much.

We simply do not respect this particular call.

We will respect a call that makes us more money.

I realize that Apple is working towards a goal of paid streaming. I think that is beautiful progress.

It would be really great if people paid to listen to streaming services, and then paid again to buy downloads. No physical product, gross margin of almost 100%. That is what I call beautiful.

We know how astronomically successful Apple has been and we know that this incredible company has the money to pay artists, writers and producers for the 3 month trial period… even if it is free for the fans trying it out.

Apple needs this to work badly enough that my friends and I can squeeze some money out of them, even if they are not making any money out of it.

Three months is a long time to go unpaid, and it is unfair to ask anyone to work for nothing.

I want the money now.

I say this with love, reverence, and admiration for everything else Apple has done. I hope that soon I can join them in the progression towards a streaming model that seems fair to those who create this music. I think this could be the platform that gets it right.

Pay me from the start, then I’ll be happy.

But I say to Apple with all due respect, it’s not too late to change this policy and change the minds of those in the music industry who will be deeply and gravely affected by this.

We can squeeze pretty hard if we want to.

We don’t ask you for free iPhones. Please don’t ask us to provide you with our music for no compensation.

…because the cost of producing an iPhone is near zero just like the cost of producing a copy of music. I may use free software to run my website, but I need more money than a geek! Its not like anyone ever produced art or music without being assured of royalties first.




Beating Node.js with TCL Sun, 14 Jun 2015 20:58:43 +0000 This is partly a reaction to people who talk as if Node.js is unique, and partly to test my code against something that has seen production use. There are all sorts of problems with doing this sort of comparison, and while I would have liked to compare more servers, used a better environment, performance tuned everything, done more measurements etc. but I think what I have done is enough to prove my point.So just what is my point? That it is possible (in fact rather easy) to write a high performance event driven server in TCL. I also dislike server-side Javascript and really think the use case for things like Node should be cases where there is a lot of code that can usefully be shared between the client and the server (i.e. you have large chunks of the same code running on both), and not for other, to quote the Node website, “fast, scalable network applications”.

I did some very rough and simple testing. I ran both servers (Node and this)  and Apache Bench on my laptop. To ameliorate the effects of running everything locally, used taskset to run the servers being tested on one core, while Apache Bench ran on another. Of course I also shut down anything not needed during testing, and I am aware that by putting both on the same core I am assuming that both would use negligible CPU when not responding to requests (seems reasonable?).

Both servers ran a minimal “Hello World”. I was not familiar with Node but in many ways it was a fairer comparison than I expected: Node is pretty low level, relies on nested callbacks etc. Comparing Node to something like Tornado would be less fair. I used the version of Node that was in the Ubuntu 14.10 repos, so it is not the latest and greatest, but it is likely to be still in production use by many people. Anyway, it is new enough to compare with my code from 2008!

The code for the minimal TCL app:

source dandelion.tcl

proc hello_world {sock headers settings body} {
puts $sock [dict get $::dandelion::first_line 200]
puts $sock “Content-Type: text/plain\n”
puts $sock {Hello World!}
close $sock

::dandelion::init handler hello_world port 8081
vwait forever

The Node code is the HTTP hello world from How to Node.

I ran Apache Bench for 5,000 requests with varying levels of concurrency, (except for 2000 concurrent requests I raised it to 6000 requests). The graphs show results:

The x-axis of the second graph shows the natural log of the number of concurrent connections. The concurrency numbers are the same as in the first graph (1, 10, 100, 500, 1000 and 2000).

Dandelion, my light, embeddable, TCL server has higher performance at low load and performance declines more slowly. The identical performance with no concurrency probably reflects the performance of Apache Bench and the OS more than the servers.

If you think this may not be a consistent result, take a look at a graph of the worst performance from Dandelion against Node’s best:

I am fairly confident that that single point where Dandelion does worse is the result of a data entry error: the worst Dandelion results at that concurrency (1,000) is well below the range of the others (at any concurrency), and within the range of the Node results, and the best of the Node results at that concurrency looks suspiciously high too (not so clear cut because Node results varied more at the same concurrency, and there were better results at lower concurrencies).

So does this prove that everyone should be using TCL? No, Node wins when you need to share code with the front end, and there are event driven servers in many other languages. What it does prove is that there is room for TCL event driven servers, and it has satisfied me that Dandelion can perform well. I do no know how its performance compares to TCLHttpd or Wub, which are undoubtedly more sophisticated, let alone to all the other event driven web app servers around (Twisted, Tornado, Lighttpd (which is scriptable in Lua so could serve as an app server on these lines) etc.

It also may encourage me to dust off the Dandelion code, give it a decent name (I am terrible at coming up with names, so suggestions welcome), and put it in a repo (Fossil for TCL code?). Is there a use case for it (answers welcome)?

I also feel that TCL has missed an opportunity somewhere. Its creator was a proponent of event driven programming. It has had an event driven web and app server for at least 15 years but now, when everyone is using event driven internet servers (not just for the web) it is rarely used and few people even consider it.

]]> 20