Here is the site. http://www.caramba-theatre.co.uk/ Just in case it disappears or changes here is a screenshot of it as it is.

As you can see it looks like a particularly bad Geocities site from the 1990s. I am sure many people will mock it. Here is why I think it is a good site.

1. The most important piece of information, the details of the next production, are right there on the home page. No hunting around.
2. Everything else I wanted to know is on a link to the about page.
3. Because the site is so simple it works perfectly on mobile without anyone having to do extra work.
4. It is accessible (for the blind_ as far as I can tell. The image correctly has an empty alt tag (as the information in it is repeated as text). Most websites get this wrong.

It has some faults. The background makes the link text hard to read. There are some broken links and “under construction” pages and out of date content. That is also true of lots of great looking sites, and is to be expected on a site maintained by a volunteer with limited time. The readability issue is easily fixed by deleting the background image (I tried blocking it and the site looked much better).

It is also far more useful to someone who wants to go to a performance than their Facebook page, the “more modern” way of doing it in many eyes. Of course, their Facebook page is likely to have other roles: to share what you are doing with friends and family, or to give people who may be interested in joining a taste of what is going on.

Of course a bigger and more commercial organisation such as the better known theatre company in town (The Royal Shakespeare Company) could not get away with having a site like this. They need to look good, keep sponsors happy and so on. In short, if the PR aspect of a nice design is important, then you need to prioritise it. If you want to put your users first, then stuff it.

Incidentally, the tiny theatre where they usually perform (the Bear Pit) has a nicer bar than the RSC too.

Some people have pointed out that the same thing is possible on sites like eBay – but I think holiday accommodation is different. You cannot return the product for a full refund, and the precious days of your holiday are gone forever.

The way I found out was, of course, a booking that went really badly.  We needed three rooms (me,, my wife, our two daughters and a friend) in Nuwara Eliya, in Sri Lanka’s mountains, so we thought booking a house rather than our usual place made sense, so I used AirBNB for the first time. I found a lovely looking bungalow that had everything we wanted.

We arrived late at night after a long drive – eight hours including some stops, and driving on winding mountain roads is tiring. As we arrived at night it was far to late to find somewhere else

The first problem cropped up even before we got there. Even though the listing says breakfast is provided, the manager asked us to buy things for breakfast. The listing said that a cook was available, and that was one of our main criteria in choosing accommodation as we had a lot to pack into three days we would be there. As we were taking our bags in we asked the caretaker about the cook, and he told us there was no cook, but he could cook breakfast (more on that later!).

The listing still says the accommodation is provided on a bead and breakfast basis, and that a cook is available.

Then we found that the place was filthy: the bathrooms had not been cleaned,  there was litter under beds and cigarette butts in a laundry basket..We ate an Indian takeaway and thought we would get a good nights sleep and sort it all out in the morning.

We then got the next little surprise which was a rat scuttling across the floor. Needless to say it scared the kids. I slammed a room door shut to prevent it getting into a bedroom, and we put the kids in their room with instructions not to open the door until we had chased the rat out of the house. We could not find it so we decided to go to sleep anyway.

We phoned the manager , and he promised that both he and the owner (the latter is the “host” in the AirBNB system) would come there at 9am the following morning and sort out the problems.

When everyone else had gone to bed, I decided to have a shower. Each room had an en-suite bathroom with a separate water heater and I soon discovered that the one in our room did not work. I did not want to disturb the kids or our friend, so I went to sleep without a shower, which I hate. However, I did not hate that as much as the reason the hot water was not working:

Yes, this is a socket hanging out of the wall, in the bathroom, right above the shower head. I had thought the floor level electric bar heater in the bedroom, crammed between the bed and the floor length curtains was a hazard, but this was even worse. That said, it was the heater that nearly started a fire because my wife pushed cushion (one of those neck support ones) off the bed so that it fell right next to it.

Come next morning, the caretakers ability to cook did not stretch as far as frying eggs, and as we wanted fried eggs we cooked them ourselves. We made sure we were ready at 9am, but an hour later neither the manager or Somerset (who gets named after a county?), the owner of the property, had turned up, and neither was answering phone calls.

We would have been rather stuck at this point, but while I sent messages through AirBNB, my wife managed to work out that the manager and the host were both employees of the near by Hill Club. This is a beautiful private club founded by British planters in the 19th century. One clue was that we discovered that the laundry was done there, so we had immaculately clean sheets and towels in contrast to the rest of the place.

So, when no one turned up at the house, we turned up at the hill club and asked for the two of them. After much prevaricating, and claims by the girl at reception that she could not find them, the manager turned up. In the meantime a little audience of staff were watching discreetly as we hung around. It possibly helped that we bumped into a member my wife knew.

We were then promised that Someset was on his way to the house so we returned to wait for him. I rather suspect he was at the club at left after us, as both of them turned up together by car.

In the meantime I read the reply on AirBnB. Somerset simply refused to believe our complaints, saying the rat and the dirt were “not possible”, and claiming that the listing had been “rectified long ago” with regard to the cook and bed and breakfast (see the screenshot above) and ending “IT IS OBVIOUS THAT YOU ARE WANTING TO CUT SHORT YOUR STAY IN NUWARA ELIYA AND HENCE ARE CREATING THESE ISSUES“.

He got out of the car, and his first words were to tell us that we had no business looking for him at the club. I suspect it would have been embarrassing for him if we did, but I will skip the reason why as it would be unnecessarily vindictive to way any more publicly.

He carried on ranting and was downright insulting. He told us that he would give us a full refund as he did not want money from “people like you”. In the meantime I had complained to AirBNB and sent them photos proving the issues with regard to dirt and electrical safety/

While all this was happening I had contacted AirBNB, send them photographs of the problems (not the missing cook or breakfast – it is hard to photograph an absence of cook). I did find them very helpful at this point..

Having got the refund agreed, and the evidence gathered in case he failed to deliver it, we booked ourselves into the nearby Gold Club to which I belong for the rest of the stay in Nuwara Eliya. Somerset calmed down a bit before we left and his tone was rather more conciliatory, though far from apologetic.

The next bit of fun started when I tried to leave a review. I found that AirBNB’s system had left an automatic review saying the host had cancelled the booking. That meant I could not leave review. After exchanging 14 emails with them, in the course of which I was told I could leave a review, I was eventually told this:

The fact of the matter is that because this reservation was cancelled by your host neither of you should be able to leave a review and the system has reverted to that. It is not an error, it is the normal functioning of our system.

A subsequent email explained their reasoning:

With regards to your point about hosts being incentivised to cancel instead of having reviews on their profile, there isn’t actually an incentive to do that as when a host cancels they receive an automated review saying that the cancellation was made and they also fully refund the guest and may incur financial penalties for cancelling as well as other account related penalties.

I am not convinced. The fact is that there is a mechanism that allows hosts to avoid the very worst reviews, and the financial penalties are limited if they are refunding all or most of the cost of the booking. Regardless of that, the end result is that the reviews are biased because hosts do cancel and thereby replace what would have been the very worst reviews with bland cancellation notices (and I cannot even see that in this case).

In spite of this, I received another reminder to leave a review from AirBNB – containing a review link that is redirected to a message telling me I cannot “edit this review”.

I did some very rough and simple testing. I ran both servers (Node and this)  and Apache Bench on my laptop. To ameliorate the effects of running everything locally, used taskset to run the servers being tested on one core, while Apache Bench ran on another. Of course I also shut down anything not needed during testing, and I am aware that by putting both on the same core I am assuming that both would use negligible CPU when not responding to requests (seems reasonable?).

Both servers ran a minimal “Hello World”. I was not familiar with Node but in many ways it was a fairer comparison than I expected: Node is pretty low level, relies on nested callbacks etc. Comparing Node to something like Tornado would be less fair. I used the version of Node that was in the Ubuntu 14.10 repos, so it is not the latest and greatest, but it is likely to be still in production use by many people. Anyway, it is new enough to compare with my code from 2008!

The code for the minimal TCL app:

source dandelion.tcl

proc hello_world {sock headers settings body} {
puts $sock [dict get $::dandelion::first_line 200]
puts $sock “Content-Type: text/plain\n”
puts $sock {Hello World!}
close $sock
return
}

::dandelion::init handler hello_world port 8081
vwait forever

The Node code is the HTTP hello world from How to Node.

I ran Apache Bench for 5,000 requests with varying levels of concurrency, (except for 2000 concurrent requests I raised it to 6000 requests). The graphs show results:

The x-axis of the second graph shows the natural log of the number of concurrent connections. The concurrency numbers are the same as in the first graph (1, 10, 100, 500, 1000 and 2000).

Dandelion, my light, embeddable, TCL server has higher performance at low load and performance declines more slowly. The identical performance with no concurrency probably reflects the performance of Apache Bench and the OS more than the servers.

If you think this may not be a consistent result, take a look at a graph of the worst performance from Dandelion against Node’s best:

I am fairly confident that that single point where Dandelion does worse is the result of a data entry error: the worst Dandelion results at that concurrency (1,000) is well below the range of the others (at any concurrency), and within the range of the Node results, and the best of the Node results at that concurrency looks suspiciously high too (not so clear cut because Node results varied more at the same concurrency, and there were better results at lower concurrencies).

So does this prove that everyone should be using TCL? No, Node wins when you need to share code with the front end, and there are event driven servers in many other languages. What it does prove is that there is room for TCL event driven servers, and it has satisfied me that Dandelion can perform well. I do no know how its performance compares to TCLHttpd or Wub, which are undoubtedly more sophisticated, let alone to all the other event driven web app servers around (Twisted, Tornado, Lighttpd (which is scriptable in Lua so could serve as an app server on these lines) etc.

It also may encourage me to dust off the Dandelion code, give it a decent name (I am terrible at coming up with names, so suggestions welcome), and put it in a repo (Fossil for TCL code?). Is there a use case for it (answers welcome)?

I also feel that TCL has missed an opportunity somewhere. Its creator was a proponent of event driven programming. It has had an event driven web and app server for at least 15 years but now, when everyone is using event driven internet servers (not just for the web) it is rarely used and few people even consider it.

Security

WordPress has a terrible track record, and WordPress sites are the most frequently targeted by attackers. There have been massive automated attacks mounted on millions of WordPress sites simultaneously. WordPress sites are easy for attacking web crawlers to identify without human intervention so the cost of attack is very low on a per site basis.

WordPress advocates will point to improvements in WordPress itself, and claim it is a lot more secure than it used to be. However, it still suffers from bad design, and a poor attitude – for example, WordPress developers actually encourage letting WordPress alter its own files because they is how their update mechanism works.

Even if WordPress is now adequately secure, even its defenders will usually concede that there are many insecure themes an plugins. As the attraction of WordPress is its wide range of themes and plugins this leaves you with limited choices:

1. Use WordPress only with bundled themes and plugins. This leaves little reason to use WordPress at all.
2. Security audit the themes and plugins you use: suddenly its not so cheap or fast to set up a WordPress site.
3. Develop all themes and plugin especially for your site: this makes WordPress an expensive solution if your site requires any real customisation.
4. Only use WordPress if you are sure it will do what you want with minimal customisation and only themes and plugins you are sure are secure.
5. Risk it!

Even being this careful, you still have to deal with the fact that a WordPress site is far more likely to be attacked. Even an unsuccessful attack can cause problems (slower site performance, bandwidth consumption from repeated attacks etc.). Certainly, you can (doing a bit more work) solve these problems, but even then any security hole in WordPress is more likely to lead lead to a breach because it is a popular target for automated attacks.

It is not a framework

People use WordPress for all kinds of websites, and even something that are better described as web apps. They have a bad case of “when all you have is a hammer, everything looks like nail”. Anyone who has used a proper framework will tell you that developing a custom site using WordPress is a far longer and less productive process than using framework like Django, Ruby on Rails, or Symfony. If you are a developers, compare this to this, or even better, this.

Similarly frameworks provide productive ways of generating forms that match database tables, or creating a custom admin interface. The work required to deliver the same custom functionality is dramatically lower.

A (good) framework also takes care of a lot of security issues for you — Django, my favourite framework, generates properly escaped database queries by default, and requires CSRF protection on forms by default, etc.

It uses PHP

PHP is a horrible language, and PHP code is harder to maintain. A developer should remember that there are always a minimum of two developers working on any project — you, and you a few months later, and the other one is an idiot. Hard to read and understand PHP code will create more work later. This is just one of PHP’s many, many faults, which are brilliantly covered in depth in A Fractal of Bad Design.

Again, the end result is to make development slower and maintenance more expensive.

PHP advocates like to point to the many large sites like Facebook that use PHP. PHP has worked so well for Facebook that they have resorted to forking it to address some of its more egregious flaws, but they are still constrained by the need for compatibility so it is still far from being a good language. Facebook also uses many other languages: they have released open source components in Ocaml, Java, C++. D, Haskell and more so they presumably use all those somewhere.

Lots of developers, does not mean lots of good developers

A lot of people like the fact that there are more WordPress developers than those working on any other web platform, many working at cheap rates. The reason for this is that it has a lower barrier to entry: it is easy to learn enough to install WordPress and some plugins, perhaps learn a bit of PHP for simple customisation… now you are a developer!

Of course there are plenty of good PHP developers, but I very much doubt there are more good PHP developers than Python or Ruby web developers. On top of that, good PHP developers are unlikely to be pitching for WordPress work and are probably focusing on more skilled stuff using frameworks. If you want cheap, WordPress is for you, but remember who works for peanuts.

What WordPress is good for

I still run my blog on WordPress, and I have used it for some small sites. It is a very capable blog platform, and is acceptable for small brochure type sites. If you are sure that is all your need, and all you are are likely to need, and have a limited budget, then maybe WordPress is a good option, but:

1. Ensure you install is security hardened: there is a lot you can do to make it harder for automated attacks to identify your site as WordPress based and make sure it is regularly updated. I will be blogging about that soon.
2. Use a host that gives you ssh access so that you can use wp-cli to run updates: otherwise you will either have to use the horribly insecure built in update mechanism, or update manually.
3. You have confidence in the security of any plugins and themes you use.
4. Pay for a decent developer. No one with real skills is going to work for £5/hour, even in a low cost country.

If it is so bad, why is it popular?

It is not a case of it being popular despite it being bad, but it being bad because it is popular. As long as WordPress was used for what it is best at (a blogging platform) it was fine (apart from some security issues). Even extending its usage to personal and small websites was fine. The problems are:

1. It is being used for things it was nor originally intended for,
2. It is so hugely popular that it is by far the most popular target for automated attacks.

The first of these causes a productivity problem: it is better to use the right tool for the job. The second means that security problems are far more likely to be found (by the bad guys) and exploited. I am talking here about automated attacks that attack thousands, or even hundreds of millions, of sites, in the hope of finding some that have a particular weakness. From the point of view of someone running one of those attacks, WordPress is probably the most attractive single target because it is popular,

I am not dead set against WordPress, but I am looking for alternatives, especially for small non-blog sites (perhaps for blogs as well) so please look out blog posts on how that goes.

Smarterer is being used for recruitment, and its business model relies on that. The tests are free to take, but employers pay for access to job applicants scores. Smarterer scores can play a large part in who gets hired.

Smarterer uses an 800 point scale, not unlike GMAT. Scores close to 800 are supposed to be very impressive (I recall impressing people more with my 780 GMAT score than I did with any real life achievement) and get an appropriate “master” description. One assumes that “expert” means someone is pretty good at something, and even “proficient” must surely indicate that someone would be a competent employee in a job that relied on that skill. What else could these as an assessment of a potential employee?

My Smarterer scores right now (not all are shown on my public profile):

SEO (Master – 796)
C (Master – 796)
Corporate Finance (Master – 791)
Financial Analysis (Master – 791)
Management Accounting (Master – 785)
HTML (Master – 782)
Statistics & Probability (Expert – 775)
CSS (Expert – 767)
MySQL (Expert – 764)
Python (Expert – 746)
Linux (Expert – 733)
SQL (Expert – 717)
Apache (Expert – 713)
Financial Accounting (Proficient – 653)
Python Workshop (Proficient – 650)
Financial Markets & Securities (Proficient – 639)
PHP (Proficient – 594)
Web Design (Proficient – 561)
Investment Banking Fundamentals (Proficient – 487)
Django (Proficient – 483)
PostgreSQL (Proficient – 461)
Microeconomics (Familiar – 434)
HTML5 (Familiar – 338)
Trading Options (Beginner – 274)

My Management Accounting score is far higher than my Financial Accounting score. I have studied the two subjects to about the same level. My overall training in finance gives me a feel for the concepts behind management accounting, but I have a lot more experience of financial accounts, albeit interpreting and analysing them rather than preparing them. I would have expected to do much better in Financial Accounting, but Smarterer thinks quite the opposite. I am inclined to put this down to a mixture of luck and, possibly, a flaw in Smarterer’s adaptive system for picking questions.

There are many other inconsistencies in my scores. Different scores in two different tests on Python, one little better than my score in the PHP one. I have used PHP only to write some simple WordPress and WolfCMS plugins, and similar light use, whereas I have used Python a lot over the last few years. My Django score is actually lower than my PHP score. I write Django all the time but rarely write PHP.

My scores in Statistics, SQL, and a number of other tests may be accurate if you interpret them in the correct context. I am no statistician or DBA, but I am far ahead of the average accountant or MBA graduate in my understanding of statistics. You cannot really interpret the results of these tests without spending time evaluating the level of the test.

For a number of tests, it is not clear what the objective is. What job does the skill relate to? Trading Options and Investment Banking Fundamentals are not tough enough to be useful to the employers to whom they may be relevant, and are too specialist to be of interest outside the speciality.

Now for the outrageously wrong tests. Sadly, this means I have to repudiate my two best scores.

There is no way I am a master SEO. I wish I was. I know the basics, but I also know my limits. My feeling is that the score reflects the general low level of competence in an industry full of snake oil, guess work, and fearful clients.

Even better is my equally high score in C. My total experience of C programming consists of an unsuccessful attempt to write a game for the Atari ST as a schoolboy and reverse engineering a few hundred lines of C (essentially working out the exact financial calculations being done by undocumented code) nearly 10 years ago. I also read about half the K & R book (the classic book on the language) about six months ago. This makes me a master?

The truth is that the C test is better described as an IQ test that assumes a knowledge of C. It tests got basic knowledge of the language, while the harder questions are essentially puzzles written in C. There is nothing that tests the skills that would be needed to write software in C.

Part of the problem is that it is hard to devise multiple choice questions for most subjects. I have seen meaningful multiple choice questions all the way up to postgraduate level, but they are very hard to set. It requires both deep subject knowledge and the very specific skill of devising good questions.

Part of the problem is that expertise is, by its nature, hard to crowd source. It may work is Smarterer gains massive scale, but I doubt it, and it is certainly not working now. Even where there are multiple tests on a subject, there is no useful way of quantifying quality.

Smarterer is great fun and I enjoy it. Use it as a subject specific trivia game. Relying on it in its current state would be mistake.

For those unfamiliar with financial markets what they are claiming is that using information on their website (the “disclosure table”) to produce a list like this:

1. Acme plc is subject to a bid by None plc. Offer made 12th December 2012. Bidder name disclosed 5th January 2013.
2. Another plc is subject to a bid by an undisclosed bidder. Started 10th January 2013/

for about a dozen lines is a breach of copyright.

This seems entirely wrong to me for several reasons:

1. The information should be as widely circulated as possible, and impeding its distribution makes markets less efficient. This is the opposite of what the Takeover Panel should be doing.
2. It impedes people’s ability to comment on the table, and therefore to comment on the market. This, again, is the opposite of what the Panel should want to achieve.
3. The information is not created by the Takeover Panel, it is reported to them by the companies concerned.
4. A regulator of markets should not be a manager of copyright works. Unlike a commercial publisher it has a monopoly and it should not seem to exploit this in any way.
5. It is a collection of facts, not a creative work. I only want the information not the layout or arrangement. It is also doubtful whether it falls under database right (which they are not claiming anyway) given the table itself required no real effort to compile.
6. It is news, and it was never an (admitted?) intention of copyright law to impede the distribution of news.

As an aside for those more interested in copyright than financial markets, this is yet another counter example to the argument that copyright provides an incentive to create works. The Takeover Panel would have no choice but to produce the table whether or not copyright existed, but given copyright they choose to exercise it.

As requests for IDE recommendations on the django-users mailing list/Google Group, and as I assume that anyone developing with Django knows they should search for answers on the net before asking, here is a summary of the most often recommended options.

• Pycharm: proprietary, expensive, full featured with lots of Django specific features including GUI for project creation, extensive auto-completion even in templates,. Very often recommended on the mailing list. $100 for individual license, free for open source and classroom use. Review here.
• Aptana Studio 3: open source, Eclipse/PyDev based and available as an Eclipse plug-in as well as stand alone, full featured with Django code completion etc. Often recommended on mailing list. Comparative review with Komodo, Pycharm, WIng IDE, Netbeans.I
• Komodo: Komdo Edit is an open source editor, Komodo IDE is a proprietary based on it. Some features of the IDE can be added to Komodo Edit with plugins. Both support Django templates fairly well (including highlighting syntax errors). IDE costs $195 for a single license, $87 year for support and upgrades.
• WingIDE: proprietary, full featured, Django support including template debugging. Single license $95 for non-commercial use, $245 for comemrcial use. Support and upgrades $89/year.
• Gedit: a text editor, but it can be turned into a fairly full featured Django IDE with plugins.
• Geany: Lightweight, but surprisingly good auto-completion. No support for Django templates (HTML highlighting only). Not a replacement for any of the above, but light weight. I use it on my netbook.
• Ulipad: Another lightweight option, and what I intend to try next. A text editor with some IDE features, and a Django support plugin.
• Emacs with various modes: no doubt very powerful, but not directly comparable to the others.
• Vim plus plugins: not really comparable with any of the above other than Emacs, and I really do not want to get into that war.

What I have omitted:

• Anything that is not cross platform
• Plain text editors
• Anything lacking documentation (Eric, at least as far as its Django support goes), positive reviews or recommendations on the mailing list.

I currently use Geany, but intend to try Ulipad, Gedit and, possibly, Eric. Geany is pretty good for something that light, but has some shortcomings: unintelligent (if generally effective) auto completion, no support for Django template language (highlights as HTML), and some GUI imperfections.

Twitter focuses on what is interesting

The point of Twitter is to follow people who are interesting, regardless of whether you know or like them, and regardless of whether they have any interest in you (unless they actively block you from following them).

This contrasts with Facebook or Linked In which require some form of acceptance form the other person. Facebook does now have a Twitter like “follow” feature, but its is secondary and not widely used. Facebook also has fan pages for one way relationships, but most people do not have one.

I would be quicker to stop following someone boring than someone obnoxious: very much the opposite of what I would do on Facebook. Interesting people are not necessarily likeable, and likeable people are not necessarily interesting.

The end result is that a typical Twitter user follows large numbers of interesting people (interesting means that their tweets are interesting). This means that a quick glance is likely to find a high proportion of engaging content. In particular it has a high proportion of content that invites further discussion.

Twitter lets you scan a lot of content quickly

Because of Twitter’s 140 character limit on tweets, it is possible to scan a lot of tweets quickly, increasing the odds of quickly finding one that is interesting enough to respond to or that contains a really interesting link.

This increases the odds of a casual glance at Twitter turning into a discussion or leading to a site that is time consuming to read.

Twitter becomes hostile more easily

Twitter encourages users to follow interesting strangers. This is Twitters great strength. It also means that discussions can turn hostile without any social consequences. I cannot imagine anyone on Facebook saying something like “I hate people like you” (something I was told on Twitter a few weeks ago), because they are all either personal friends or family, or belong to some place or community which means they have some real life connection with me.

Because Twitter is more focused on topics than on people it is also more frequently used to discuss controversial topics, and there is no restraint about bringing up topics that may upset people: for example a blog post I recently read on abortion had a paragraph at the start warning that it contained graphic details that could be upsetting to those who have had abortions or miscarriages, that kind of restraint is unusual on Twitter — partly because the terse 140 character limit leaves little room for prefaces or tactful phrasing.

The result is both high engagement (few people quietly back off from arguments once started) and stress.

Twitter is often immediate

In interesting tweet very often gets fast responses, which can lead to an immediate extended discussion. While this is also true of some other forms of communication, social networks are unique in the combination of fast response, high reach (tweets are seen by every who follows the tweeter and anyone mentioned in a tweet) makes fast responses much more likely.

This makes Twitter far more likely to engage a user for a long time than a forum is.

Twitter covers all topics

Because people typically tweet on multiple topics, and follow people interested in different topics, it covers far more that topics that may engage a users. It is perfectly possible to look at Twitter to look for tweets about investment and find lots of interesting tweets on politics or religion (often that one wants to respond to). This happens to me quite frequently.

Twitter is often partly work related

Many people tweet about work related topics (giving one a great excuse to tweet during working hours), but few have the discipline to stick to just that. Once you are on all the distractions are right in front of you.

Try finding an excuse to spend working hours on Pinterest!

Twitter is public and demanding

Most people make their Twitter streams public, so people with similar interests can find and follow them. The problem is that producing a stream of terse comments that you are willing to put in writing in public is demanding. Mistakes are public, lack of fact checking is noticed. It takes effort, but as the effort is demanded in 140 character bursts, it is easy to fail to realise how much time and effort it takes in the course of a day.

The problems are the benefits

The problems are the benefits, not just a consequence, or even just an inevitable consequence. The point of Twitter is to find interesting people with interesting ideas and interesting discussions.

One of the great advantages of Twitter is that, because if you follow someone you see their tweets on all topics it exposes users to a diversity of opinions: especially when combined with its ability to find interesting strangers from anywhere in the world to follow (that sentence would have sounded creepy before Twitter…). This is especially valuable in an age when social media and a huge choice of broadcast and print media make it easy to retreat into a world where conflicting opinions disappear from view.

There is nothing wrong with Twitter. It does what it is meant to, but at a price.

Imagine how outraged people would be if Microsoft altered Windows to only run software purchased through MS. Now imagine that they went further than tried to take charge for every thing that was read, watched, or listened to on every Windows PC?

This is exactly what Apple have been doing with iPhone’s and iPods. Software can only be installed through Apple’s own “App store”.

Now Apple have prevented ebook reader apps (i.e. software) like those from Amazon and Kobo from directing users to buying ebooks from those companies websites: all purchases that apps promote (or made through apps) must be made through apple.

Apple have done what I asked you to imagine Microsoft doing. With both Apple and Microsoft planning app stores for their PC operating systems (i.e. MacOS and Windows), we are likely to see this spread. It is unlikely that the app stores will initially be the only way to install software, but once the mechanism is in place, it is likely to get locked in.

Its worth pointing out that Linux has had the benefit of app stores (sans payment mechanisms) for a decade, but without the vendor control. Apples innovation consisted of doing what Linux does, in a technically inferior and less flexible way (no dependency handling), but integrating the payment mechanism.

It is not just an operating system that can give you this basis for making suppliers of stuff for your platform pay. What Apple is doing to Amazon, is exactly what Amazon is positioning itself to do to authors and publishers. Amazon already takes a substantial proportion of the retail price of an ebook (30% of what is left after deducting a distribution charge), a huge amount for essentially listing the book on a website and handling a payment. This is more than the author typically gets for doing the hard work. As the Kindle’s market position goes stronger they will be in a position to increase this if they wish.

Facebook now requires game developers to accept Facebook credits for in-game payments. Again, this means Facebook keeps 30%, a lot more than the game developers ever had to pay normal payment processors (sounds familiar).

The common strategy is this: you build a platform, people build applications or content for your platform, and you then charge them for the privilege. They are already committed, and you control access to the users giving you a monopoly over selling to those users.

Telecoms companies attempts to charge websites that their customers use (ending what is called “net neutrality”) is a similar strategy. You own the customers and charge for access. I suppose we should count ourselves lucky that they are not charging businesses that want to receive phone calls from their customers).

Ultimately this is about controlling customers. You lock the customers into your platform. The software industry has long done this, but limited its exploitation of it to selling upgrades. Now, this control can be exploited in many ways. You have a group of customer to whom you are a gatekeeper: its like owning all the shops in town, but without the pesky attention from competition authorities that doing that would attract.

The issues

• Design
• Development
• Usability
• Content and functionality
• Domains and structure
• Search engine optimisation

It is easy make the common mistake of thinking that a website is primarily defined by its design and appearance. This is just one element among many, and design decisions rarely have long term consequences: you can always change the design later.

Page addresses

We will start by getting a key technical fundamental out of the way: it should all get easier after the next two paragraphs. The meaning of a URL: the address of a web page, something like http://example.com/section/page.html.

http:// (the “protocol”) says fetch this from a website, the example.com specifies the site to fetch it from, and the rest of specifies a particular page. The http:// is needed because there are other ways of connecting example.com other than to fetch a web page: file transfers, remote logins, sending email, etc.

Historically the web pages were files stored on a computer running the site, and the “/section/page.html” would specify the file (.html specifies a web page) called page.html in the directory (folder) section. Nowadays the systems running your website may interpret it differently. However, it is still good practice make sections of your site in what look like different folders. Logical organisation makes your site easier to understand.

The other important points about URLs are:

1. They should not change: visitors bookmarks and links from other sites can be lost.
2. It is good practice to keep them short and meaningful
3. Leave technical information (such as the “html” above) out.
4. If you do have to change them, make sure that there are “301 redirects” set up to send visitors and search engines to the new URL. You do not need to know how a 301 works, just make sure someone tests that the old pages do redirect.

For more detail read cool URLS don’t change by Sir Tim Berners-Lee, the inventor of the web. It also rebuts some of the excuses developers come up with for changing URLs.

Domain names

The domain name is what identifies your site. It is something like example.com.

The domain must be registered in your name. If you allow a web developer or designer to register it in their name, they will have ultimate control over the site (as well as email address on that domain, and possibly other services).

You need to make some choices about your domain name:

1. Is it better to register a global domain (such as .com or .org) or a national one (such as a .co.uk or a .lk)?
2. Should you use a full name or an abbreviation
3. Should you use a company name, or a generic term for what it sells?

Some general rules:

1. If your site has a strong, permanent geographic focus that lies within a country, use a domain within that country’s code (“ccTLD”). Sites about a particular place but aimed at global visitors are a borderline case.
2. Keep it short enough for people to type remember and type in easily.
3. It is usually more important to be memorable and and distinct than to incorporate generic terms. Sometimes you can do both, like damnsmalllinux.org. A distinct name is easy to find with a search engine like Google, even if people have forgotten the exact domain name.
4. You should register and use a domain within a top level domain relevant to your purpose: a commerical organisation should use .com, .co.uk etc., a non-profit should use .org, .org.uk.
5. It is worth registering the alternatives. If you use myname.co.uk, you should register myname.com (if its available), and obvious misspellings (like mynam.co.uk).
6. You should avoid switching away from an established domain. If you must do so, then retain ownership of the old domain and set up a “301 redirect” to the new site.
7. Some countries allow a domain directly under the ccTLD. You can have myname.im or myname.lk. These are short and memorable, but the rules vary by country.
8. Less well known generic domains (like .biz and .info) are generally to be avoided, as are misused (i.e. like .tv) country domains. There are many exceptions to this rule, especially if you really need a short domain name (like bit.ly).
9. The rules for country domains ultimately depend on that country’s laws. Beware of foreign domains: it could lead litigation in a foreign country. For the same reason, it is preferable to use a registrar in your own country or one with a legal system that puts the registered owner in a strong position (note: this is rapidly changing, and there are problems in countries with otherwise strong legal systems such as the US and the UK)

Software to run your site

Unless your site is absolutely tiny (literally one or two pages) you need software that will allow you to easily add and update content on your website. This is usually a content management system (CMS) that lets you login, write the text of pages into forms, and upload images.

The good news is that that there are a lot of very good open source CMSs: no license fees to pay, no licensing process to manage, and you can pay developers to alter or extend to system to fit your needs. If you stick to these you are reasonably safe from licensing and legal issues. In theory, you may run into problems over the copyright of custom work done for you. In practice such problems are rare, but you should check that you have either ownership or a clear license.

A CMS will give you a what you need to run a simple site: pages of text, a blog or news section, forums, perhaps even an online store or basic social networking. Sometimes it is better to write a CMS that fits your needs. This is most commonly because you want a specially designed database that “understands” how you content is structured.

My own financial encyclopedia is a good example. It needs alphabetic ordering, an alphabetic list of short definitions linked to articles (sometimes more than one short definition per article), categories with category groups, etc. — all best served by a properly structured database.

The good news is that a custom CMS is not as difficult or expensive to do as you might fear. It will be a lot more expensive than just installing a standard CMS and doing a little customising, but the existence of “web frameworks” means a developer no longer has to re-invent the wheel to write a custom CMS.

People to build your web site

Web designers are the people who make your site look right. It is a common mistake to think of a web site as primarily a matter of design. This may be true if a site is very simple with very little actual functionality — a page of two or text and pictures.

The more functionality a site has, the less the design matters. Unless there is an existing CMS that delivers exactly the functionality you need, you need a developer to write what you do need.

If a site is well engineered the design can be changed without re-writing the functionality. This means it is not important to get the design right first time. Just get it working.

A competent designer will deliver a website that will work well in all the common web browsers (at the moment Internet Explorer, Firefox and Chrome). They will also deliver something that works reasonably at different screen/window sizes. It should also work with screen readers used by the blind, and be comprehensible to search engines (these two are linked).

If you want to test the design consider using Firefox with its web developer extension. You will not need all its extensive functionality, but it has a lot thats useful including disabling Javascript, disabling styles (in its CSS menu), disabling cookies, disabling images, small screen rendering, and resizing the window to preset sizes. Remember Google and blind users see neither images nor visual styling!

A good designer will also take care of “on-site” SEO (or work in a team that includes an SEO specialist), and at least make an effort to deal with usability issues (or, again, work with a specialist).

Dealing with developers is a bit more difficult. Testing technical competence and technical quality without technical knowledge is tricky. A good basic principle is to hire people who are intelligent, and who have a genuine interest in the technology they use: people who do it primarily because they enjoy it, rather than it being just a job.

If they are doing a lot of customisation, or writing a custom CMS, it is even more difficult to test whether they understand the conceptually difficulties of database design. You might have to resort to bluff or finding an expert to check it. It goes beyond the scope of this article to discuss this at length, but an appallingly high proportion of web developers do not understand the basics relational databases.

If you have in-house developers and database administrators, you may be able to use them to check of the quality of web developers work. This assumes that they are competent themselves which is just as hard to verify!

Testing your site

A site requires testing in several areas.

1. That the functionality works.
2. Performance: how fast pages load etc. There are many tools to do this, but you need to do some testing that matches your customers’ locations and connection speed.
3. Usability: whether visitors can do what they want to on the site. Can they find the information they want? Can they make purchases easily? Can they register for forums or mailing lists easily?

Research by Jakob Nielson, the leading expert on website usability, suggests testing on small groups assigned tasks to carry out (“find the red widget user manual on our site”) leads to huge improvements in usability. He found that even minimal proper testing has significant benefits.

Content and functionality

It is well worth spending time thinking about what functionality you can include. For example, suppose you are a manufacturer of consumer products, it may see obvious that you want a product catalogue on your site, and links to retailers who will sell your products (both online and off-line), and possibly recruitment and investor relations sub-sites. However, there is a lot more you can do. A few examples:

• Product manuals for existing customers: very useful for portable products that may be used away from home!
• Information for retailers, maintenance people etc.
• Forums for customers to discuss your products and related issues. This can also reduce customer support costs.
• Other information useful to customers and prospective customers.
• Information for prospective dealers.

There are far more possibilities, and there are overlooked possibilities on most web sites whether corporate, small business or non-profit.

Search Engine Optimisation: good site design

You usually want as many people as possible to find your site though search engines (like Google and Bing). The basics of search engine optimisation (SEO) are not difficult, but the practice is hard. You need to ensure that you site does two things:

1. Tell search engines (Google and Bing are usually the ones that matter) what each page is about, so it can be returned for the right searches.
2. Convince search engines that it is a high quality site, so that it comes high up a page of search results

The first, and easier, part of search engine optimisation lies in “on-site” factors. How the content on you site looks to search engines. Important issues are:

1. The page text contains the keywords people search for. Over do it by repeating words a lot and the search engines will penalise you for trying to fool them.
2. The page title and headings contain the keywords (over do it and you will get penalised)
3. Important pages have lots of links from other pages, and the text in the links contains the keywords (overdo it and it might backfire — do you see a pattern here?).
4. Images have alternate text. If no alternate text is appropriate then there should be blank alternate text to indicate this.
5. The site works with Javascript turned off.
6. Headings are properly formatted, so the search engines know they are headings: it is possible to make text look like a heading (visually) properly marking it as a heading.

So, how can you check all this? A rough check of the most important things is not that difficult:

1. Use a text only web browser like links or w3m. This gives you a better idea of what a site looks like to search engines like Google, and to blind users with a screen reader. You will see alternative text in terns of pictures, proper headings should be distinct (links centers them). Most text only browsers do not support Javascript either.
2. Use Firefox with the web developer extension. As mentioned above you can turn off Javascript and images. It can also “linearise” a page to see its content in the same order as search engines do. From CSS > Disable Styles > All Styles you can see a “bare” page in a similar manner to a text only browser. Disables styles and images and Javascript and check the page is still usable
3. Page titles will appear in the window title bar or on the tab, depending on your browser and settings.
4. Check any text that pops up when you hover over links and pictures.
5. The Firefox web developer toolbar’s information section will display a list of links from a page, meta on the page, and much more.

The meta information is additional information about a page that is not normally visible to visitors. but is likely to be used by search engines and automated systems. Full coverage of this is well beyond the scope of a relatively non-technical article.

Most of the above is really just good web design practice. SEO usually requires a bit more.

Search engine optimisation: ranking high

Doing the above will tell the search engines what a site is about. This may be enough. In other circumstances there will be a lot of competition so you have to persuade the search engines that your site should rank above other containing the same information. It all depends on how many sites have pages containing the words that will be typed into search engines by the people you want to visit your site.

A corporate information site you want found by people searching using your company name is usually a pushover. If you are selling certain branded pharmaceuticals it is very, very difficult. Most sites fall somewhere in-between.

A common mistake is to assume that ranking well for a fairly small number of easy to rank for terms means that you have achieved the ranking you need. This usually means that you are neglecting opportunities. Think about what other searches you could benefit from.

Suppose you own a hotel in a small town. You come top in Google for the name of the hotel, and for “[town name] hotel”. This is a good start, but you can also find potential customers through searches such as “[town name] holiday” , “[county name] hotel”, “[town name] places to stay”, and many, many others. There are ways to research which are most important, but for the moment I just want to make the point that ranking for a few phrases for which there is no real competition is probably not enough.

The main way in which you persuade search engines to rank pages from your site highly is to get links to your site (especially links to the pages you want to rank highest) from other highly ranked pages. There are Firefox extensions that will show the Google “page rank” for the current page, but beware that this number is usually out of date, fluctuates, and is only a very rough indication.

It is easy to over-enthusiastic about link building, but there are pitfalls from the unwary.

• Links from relevant sites are the most valuable, links from sites on unrelated subjects are often worthless.
• Swapping links is less valuable than getting one way links. Swapping links with spammy sites (or linking to them for other reasons) is damaging.
• Links that search engines think have been paid for will be discounted.
• Sudden surges, or patterns of links that look unnatural to search engines, will be discounted.
• Links from links pages and directories, or, to a lesser extent, any page with lots of links on it are less valuable.
• Lots of links from spammy sites, or sites suspected of selling links.

The very best way to get links is to make your site so good that people link to in unasked.

SEO: warnings

SEO is not always the answer. For very competitive search terms it may not be feasible or affordable to get a site to the top.

There are also huge numbers of supposed SEO experts who are either dangerously ignorant, or who deliberately use risky approaches that provide short term gains (until they get paid).

Assume that anyone offering SEO services that go beyond the basic good practice described is a snake-oil salesman unless proved otherwise.

Sometimes SEO will not achieve its objectives, or may not be sufficient by itself. You can put your site, with your choice of wording, on top of the search results by buying advertisings from search engines. In addition you can buy advertising on other sites (both the major searh engines will act as intermediaries to arrange this) very cost effectively.

The purpose of your site

One question you need to ask yourself before you start is what you are actually trying to achieve. One of the easiest mistakes to make is to limit yourself to just part of what you could achieve. Assuming you are running a business:

• Can it generate sales?
• Can you provide post sales support? Can that cut costs?
• Can you improve customer loyalty by adding community features (such as forums) to the site?
• Can you gather information on customers or potential customers who visit the website?
• Can it support dealers or other businesses that work with you?
• Is it a potential recruitment tool?
• Is it about your products or your company? Some companies have separate websites for multiple brands, and a corporate information website as well.
• Which countries or other geographical areas are your target? Do you need multiple websites?

The list of thing you can achieve may be different for a non-profit website, but it is just as varied. For example a political website may be aimed at persuading waverers, informing existing supporters, providing resources to active campaigners, raising funds, organising activities, or all of these.

Flexibility

You may also need to think beyond current needs. What may you do in the future? Is the system flexible enough to cope? A major advantage of a site written using a web framework rather than just installing an off the shelf CMS is that you can almost always add new functionality without replacing the system.

It is also important to ensure that your data (customer information, product information, whatever else you site may need), that is needed by the site is kept in formats that can be accesses and processed by whatever systems you may use in the future. This is a good argument for using open source software, and an equally strong argument for using properly designed databases.

What I have not covered

There is much that is important that is out of scope, but you should at least know what the key issues are:

• Security: both securing the site and the server it runs on against hackers, and securing communications between the site and those who run it (and possibly visitors as well). This is particularly important for finance and e-commerce sites, but its always better to err on the side of caution in assessing security needs.
• Technical choices: omitted for obvious reasons.
• Scalability: the size to which a site can grows, especially in its ability to handle large numbers of visitors. If you need it you can afford to pay a specialist consultant.
• Writing for the web: integrating writing with SEO, and ensuring the style of content suits the web.
• User management and community management (for forums etc.).