The is exactly the same as the worst of the problems caused by monkey patching. It makes code less readable. A signal could be anywhere in a project, and it could do anything. You look at a view and think you know what it does, but there could be code anywhere that does something you do not know about. Readability counts.

Of course monkey patching is worse. For one thing the risk of incompatibilities break on upgrade is much higher: signals use a well defined API, whereas monkey patching is entirely unpredictable and may change anything.

What are the alternatives to signals? You can often put the code somewhere else. In fact you can always put the code somewhere else if you are willing to fork when signals come from other people’s code. For model signals you can usually use the save method instead, for others you may be able to to subclass or wrap views.

Like monkey patching signals should be used reluctantly, because the alternative is even worse.

I did some very rough and simple testing. I ran both servers (Node and this)  and Apache Bench on my laptop. To ameliorate the effects of running everything locally, used taskset to run the servers being tested on one core, while Apache Bench ran on another. Of course I also shut down anything not needed during testing, and I am aware that by putting both on the same core I am assuming that both would use negligible CPU when not responding to requests (seems reasonable?).

Both servers ran a minimal “Hello World”. I was not familiar with Node but in many ways it was a fairer comparison than I expected: Node is pretty low level, relies on nested callbacks etc. Comparing Node to something like Tornado would be less fair. I used the version of Node that was in the Ubuntu 14.10 repos, so it is not the latest and greatest, but it is likely to be still in production use by many people. Anyway, it is new enough to compare with my code from 2008!

The code for the minimal TCL app:

source dandelion.tcl

proc hello_world {sock headers settings body} {
puts $sock [dict get $::dandelion::first_line 200]
puts $sock “Content-Type: text/plain\n”
puts $sock {Hello World!}
close $sock
return
}

::dandelion::init handler hello_world port 8081
vwait forever

The Node code is the HTTP hello world from How to Node.

I ran Apache Bench for 5,000 requests with varying levels of concurrency, (except for 2000 concurrent requests I raised it to 6000 requests). The graphs show results:

The x-axis of the second graph shows the natural log of the number of concurrent connections. The concurrency numbers are the same as in the first graph (1, 10, 100, 500, 1000 and 2000).

Dandelion, my light, embeddable, TCL server has higher performance at low load and performance declines more slowly. The identical performance with no concurrency probably reflects the performance of Apache Bench and the OS more than the servers.

If you think this may not be a consistent result, take a look at a graph of the worst performance from Dandelion against Node’s best:

I am fairly confident that that single point where Dandelion does worse is the result of a data entry error: the worst Dandelion results at that concurrency (1,000) is well below the range of the others (at any concurrency), and within the range of the Node results, and the best of the Node results at that concurrency looks suspiciously high too (not so clear cut because Node results varied more at the same concurrency, and there were better results at lower concurrencies).

So does this prove that everyone should be using TCL? No, Node wins when you need to share code with the front end, and there are event driven servers in many other languages. What it does prove is that there is room for TCL event driven servers, and it has satisfied me that Dandelion can perform well. I do no know how its performance compares to TCLHttpd or Wub, which are undoubtedly more sophisticated, let alone to all the other event driven web app servers around (Twisted, Tornado, Lighttpd (which is scriptable in Lua so could serve as an app server on these lines) etc.

It also may encourage me to dust off the Dandelion code, give it a decent name (I am terrible at coming up with names, so suggestions welcome), and put it in a repo (Fossil for TCL code?). Is there a use case for it (answers welcome)?

I also feel that TCL has missed an opportunity somewhere. Its creator was a proponent of event driven programming. It has had an event driven web and app server for at least 15 years but now, when everyone is using event driven internet servers (not just for the web) it is rarely used and few people even consider it.

The reasons I dislike these still apply: they feel less responsive than the other IDEs I have tried, and the UI is not particularly comfortable.

Pycharm does have a lot to like as well. It does a lot to help: for example it will warn you about modules that are imported that are not in your current virtualenv. It has good autocompletion, call tips, a debugger on par with those in Wing and Komodo. It also makes helpful suggestions: for example. it spotted a call to set() that could be replaced with a set literal. It also classifies warnings as weak or strong, and tells you how many there are of each in a file.

On the other hand I still have the UI, it feels sluggish  and it is a memory hog (600MB with a small project open). Given that other IDEs do what I want, I see no compelling reason to buy it.

Liclipse looks promising but us far weaker. Its auto-completion saw not good, and it seemed to have no particular strengths. Nothing in particular to recommend it as far as I am concerned.

I tried Eric5 as the recently released Eric6 has problems on Ubuntu 14.10. These can apparently be fixed by recompiling one of the QT libraries, but I am not going to got to that much trouble to try something.

Eric seems to have everything, even its own web browser. One may expect that a web browser bundled as a help viewer would be a minimal wrapper around a rendering engine, but the Eric developers think differently. It is a full featured web browser. The one thing it did not seem to want to do was actually display any help. This is a good example of what is wrong with Eric, lots of rough edges and no documentation. I have not been able to discover whether there is any way of setting per project virtual environments.

Eric has what look like a good debugger, but it is erratic. It showed useful data for an error, but I cannot persuade it to break at breakpoints. Wing IDE does exactly what I expect.

Both have nice code browsers, what show all modules, classes, functions etc. in a single tree.

Wing has one big advantage over everything else: its auto-complete is the best I have seen, and it seems able to get everything right.  Wing is expensive and there are no concessions for individual developers, something that both Komodo and Jebrains (Pycharm) offer, other than that prices are similar, but for freelancers and others paying for their own software, Wing is a lot more expensive.

Wing has some disadvantages. I find the UI cluttered compared to the clean look of Geany, Ninja or Komodo. Configuring a project for Django causes it to add TEMPLATE_DEBUG=True to the main settings file, which means manual fixes for some configurations. It is only an irritant, and it tells you its done it, but I dislike unasked auto-edits of code.

I also probably need to go back and compare debuggers as that seems to be a strong point of Wing IDE, and will be important to compare it fairly to Ninja and Python. its Django template debugging does not really seem to offer very much more than Django’s own eorror messages and seems rather less useful than the alternative error page in django_extensions, but for general Python debugging it seems pretty good.

If I narrowed down my choice to Wing and Eric, I would probably pick Wing in spite of the high price, because it works. Given the other options available, Wing’s price is a deterrent given that the only thing I really want from Wing that other IDEs cannot match is the auto-completion. Will getting a higher proportion of difficult auto-completions right boost my productivity?

Part 4: Liclipse and PyCharm

Ninja has very little support for web development apart from syntax colouring for Django tags. Even the browser preview just opens the file in the browsers, whereas both Komodo and Geany (with an extension) can open a preview URL.

Moving on to Python, I cannot decide whether Ninja or Komodo has better completion. Ninja correctly completes attributes (like .objects) or Django models imported into views, which Komodo does not. Komodo deals correctly with ElementTree imported as etree, which Ninja does not. Ninja fails to complete imported objects, but the Kai plugin fixes this After two days of side by side use, neither is obviously better, so its near enough a draw,

Komodo has calltips, Ninja does not. Komodo detects files changed on disk, Ninja does not. Ninja lints for Python 2 to 3 issues and suggests fixes that can be applied with a click, making it very easy to wrote Python 3 compatible Python 2, Komodo does not. Komodo has more plugins, Ninja has more Python specific plugins.

Komodo is heavier, using more RAM and CPU, but except for the times Komdo’s CPU usage really does up, both are fine on any reasonably modern machine – except when Komodo suddenly decides it wants to use 100% CPU.

Komodo supposedly has refactoring support, but it does not seem to do much more than Ninja’s find usages.

I also tried opening an old TCL project in Komodo. It does have code completion that works reasonably, and rather limited call tips, but it is not as good as I would have expected from a company so involved in TCL. However most TCL devs will probably by the Tcl Studio version, which has more TCL specific features, so that may be a lot better.

If I wanted a single IDE that works with multiple languages, or I did a lot of Javascript, I would probably buy Komodo. It is certainly far better than Ninja for a Django developer who does a lot of front end work

For a mostly Python developer, it is very difficult to pick a winner. On the whole I marginally prefer Ninja, and it is free and open source. I can see myself using Ninja for Python and Komdo Edit for HTML, JS and XML (I do not do enough of these to justify buying Komodo IDE).

Part 3: Eric and Wing

One note on open source vs proprietary and free vs paid. I prefer open source all things being equal, but if the proprietary product is better I will use it. This is particular true of development tools where all the data is text files so there is no real danger of vendor lock-in.

I am prepared to pay if it will improve my productivity.

What I knew before I started:

As I said, I have used Geany a lot, and I like it a lot: it is fast, lightweight, has reasonable syntax checking, has (not terribly effective) auto-completion, a very nice UI, and some very useful plugins. It also does a reasonable job of HTML, CSS and Javascript. I have it set up so it runs flake8 as a “build” command. The biggest shortcoming is the auto-completion, and I like it enough that I considered writing a better Python auto-complete plugin myself, and I may still do that.

I have also used Komdo Edit a fair bit, with some addons it makes a reasonable light IDE. It also has a UI I like, and it has good syntax checking and auto-completion for Python HTML, Django templates, XML, Javascript, CSS and supports a good many other languages (from what I hear, many equally well). I have not tried Komodo IDE before, but it is a full version of Komodo Edit, so it is familiar, just with a whole lot of extra features. I want to like Komodo IDE as I like Komodo Edit

I have tried Pycharm and do not feel comfortable with it. Just do not like the UI. It is a purely personal preference, but its not for me. The same goes for Eclipse, though maybe for Liclipse.

I know Vim with a bunch of plugins can be very powerful and productive. Like Komodo it supports a lot of languages (far more, in fact).

I have also use Ninja-IDE before. I like the UI, it has erratically good completion, but a lot of code checking – not just linting, but Python 3 compatibility as well. It has one huge flaw: it fails to detect and warn of files changes on disk.

The shortlist

I looked for IDEs that have the features that are missing from Geany, are reasonably popular, are able to handle multiple projects in different virtualenvs, are maintained, and run on Linux. This left me with:

• Ninja
• Komodo IDE
• Eric
• GVim
• Wing IDE

Eclipse and Pycharm omitted because I dislike them. I would also like to try Liclipse (although I dislike Eclipse, Liclipse is supposed to be significantly different from Eclipse + PyDev). Editra omitted because I cannot get Python auto-completion to work.

I may also try Leo.

First impressions

Komodo IDE has all the strengths of Komodo Edit, but does not add a lot other than a better source tree and refactoring support. The refactoring in both Komodo and Wing IDE only seems to rename classes in the current file, which makes it fairly useless. I may be wrong and I will try to fix this problem: I know from past experience that things like this are sensitive to correct configuration. Ninja’s “find usages” seems to work better for me at the moment, although it does appear to only do a simple strong search.

Komodo and Ninja still have the best UIs, and I am comfortable with them as with Geany.

Ninja has lots of plugins, as does Komodo, but most Komodo plugins (sorry, “addons” as it follows Mozilla definitions as it uses the same platform) are yet to be updated to work with Komodo 9, and Ninja’s are only very briefly described or documented so they need to be installed to see what they really do.

I am also disappointed that a new stable version has not been released for 20 months, and this means a lot of important fixes (warning about changed files, for example!) are only in the “daily” version.

At the moment, I think that if I was using other languages it supported I would go for Komodo, but given that almost all my work is Python, I can afford Ninja is better. I will probably continue to use Komodo Edit for XML and Javascript, but that is really just light use.

My first impression of Eric is not good, but it may just be different.

The two most important things to do next are:

1. putting in some effort into checking the Vim configuration is optimal, and spending enough time with it to learn at least the important short-cuts, and,
2. checking Komodo and Wing project configuration is correct,
3. trying Eric properly, and,
4. installing Leo.

Part 2: Ninja and Komodo

Source code is the text commands that tell a software program what to do. Whereas software from the likes of Microsoft contains secret source code that is developed and maintained by the company’s own programmers, in open source code software the lines of code are open for every programmer to view, use, modify and improve.

All perfectly correct, but if a developer, or even a manager, of a small software business does not know what source code it, they have a serious problem.

It then goes on, quite correctly, to explain that open source software is covered by copyright, and license terms must be followed, but its explanation of the terms and the legal consequences is rather poor:

They stipulate that any software created from, or even containing, lines of their open source code must be made publicly available to all other developers.

This, in practice, means one of the GPL licences. This misleading as the criterion is not that the software “contains” lines of code. The commonest version of it is that:

any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof

This can included works that link to GPL libraries (although not if the library uses the LGPL variant), or that is for some reason a derived work (as defined by copyright law). That said, proprietary equivalents also have licences, that can be even more complex.

There are, at a conservative estimate, around 70 licences governing the use of open source code

True, but only a handful are widely used: versions two and three of the GPL license and its LGPL variant, the Apache license, the MIT and BSD licenses (which are very similar), and, perhaps, the Mozilla Public License.

It is also misleading about the legal consequences of a breach:

you may be forced to make freely available the software that you’ve slaved over. Although the licences may not actually preclude you from trying to sell that software also, if users know they can get it for free then they have much less incentive to buy it from you.

Not true, because you have other options. You have to stop distributing the software or open the source code or get a license for use in a proprietary application. You may have to pay damages for breach of copyright, but this rarely happens. If you can sell your software as web app (installed on your own servers) the GPL does not restrict your sales at all (because you are not distributing it outside your own organisation) unless it uses the fairly rare Affero GPL variant. You may be able to negotiate a commercial license from the copyright holder to use the open source code in your proprietary app (some open source developers base their business model on this).

The article then goes on to cover “virus problems”, but the author does not seem to know what a virus is. He refer to ” the recent Shellshock and Heartbleed viruses”. Neither Heartbleed nor Shellshock were viruses: they were vulnerabilities (holes that could be used to infect a system with a virus, among other ways of attacking a system).

If a bug in your software introduces a virus into your client’s systems then you’ll not only face acute professional embarrassment, but also a potential negligence lawsuit.

The first problem with this, is that it could happen with a proprietary library or platform as well. Unless you write everything from scratch, using no external libraries (which is usually impossible) whatever third part software may have security issues. Going by track records, using open source will reduce this risk.

The other problem is that the risk of a negligence lawsuit is tiny. Heart-bleed affected most of the world’s secure web servers (i.e. any page with an address starting “https” and showing that lock icon in web browser). It affected banking sites, and other things people depend on. There were similar bugs found in other major encryption libraries, both proprietary and open source (Apple’s, Microsoft’s, and Mozilla’s).  How many lawsuits resulted? The only one I am aware of was against the NSA who knew about the problem but failed to warn people, not against any software developer.

If there was a significant risk of being held responsible for security vulnerabilities, the software business would be very different, and several (if not most!) of the big software companies would have been sued into oblivion years ago. I am not suggesting that this would be a good thing, as strict security review for everything would make software development too expensive.

Similarly with Shell Shock, except that it is a less widespread thread. The ways I can think of it happening are an insecure server configuration (which can happen with proprietary software) together with user data passed to a newly started process (probably a CGI script started with Bash), or through failure to sanitize user input, or writing a CGI script in Bash (why would anyone do that?).

The useful information in this article can be summarised as “if you use open source libraries or code, check the licences, and, in particular, make sure you comply with GPL terms on distribution and Apache terms on patents”. That, and a link to the GPL FAQ would tell most people what they need to know.

we must agree Fossil [..] much easier and friendlier to use.

It is, and it is not. I do recommend it, and it does provide a a lot of functionality and it is easy to learn and use.

Why I like Fossil

Fossil wins hands down in terms of the time taken to learn the command line, and the ease of use of the command line. I like the work-flow Fossil encourages and have experienced very few problems after several years of use. I use Git because it is what all my clients choose. I do mean all: the only projects I use Fossil on are my own, and where the client leaves the choice of version control system entirely up to and I expect to work on it alone. Long term lock-in is not an issue as Fossil can export a repo to Git.

Fossil’s built-in web based GUI is also easier to use than Gitk. It does not do everything (for example, you cannot commit from the GUI) but given that this is a developer tool and the command line is easy to use this is not really a problem. It also means you can use the same GUI locally and on a server. Git is very powerful, but Fossil does what I want and more, and Git’s additional features often end up making it slower to use and creating more room for error — the need to stage changes is a good example of this.

In my view Git has two real advantages:

1. Automatic detection of renames. Fossil can be awkward if, for example, you move a lot of files around, for get to rename in Fossil, and then run fossil addremove.
2. The ability to completely roll back mistakes. Whether this is a good thing or not is a matter of opinion and Fossil is the way it is by design.

These advantages are outweighed by ease of use, and the convenience of having a ticket-tracker and wiki without any installation or integration work.

I am sure that Git has massive advantages for some people, particularly for large projects with huge numbers of collaborators. It was, after all, designed for the Linux kernel: approaching 18 million lines it is, as far as I know, the largest single computer program (in the sense of a single executable and excluding linked libraries), and requiring collaboration between a huge number of individuals in a huge number of organisations. For individual developers and small teams, I prefer Fossil

Git’s big advantage

Having said that Fossil is better, I now come to what changes everything: the tools and services available for Git. Installing Fossil with a wiki and an issue tracker is just a matter of installing Fossil. No configuration needed (except to set appropriate user permissions on a server). However, if you are using Git, you just register with Github (and pay if you need keep your repositories private), and you have much better issue tracking than Fossil (I suspect the wiki is better as well, but I have not used it). Fossil requires only simple server configuration, but Github requires none at all. Bitbucket provides a similar service, gives your private repositories free, and you can use Mercurial if you do not like Git. If you are working on open source software these services make it very easy for others to fork your code, work on it, and make pull requests.

If you do not like to use a repository controlled be someone else, you can use something like Gitlab, Gitblit, Trac or Redmine to provide issue tracking and wiki integrated with Git.  A bit more work to set up, and you lose “social coding” advantages of Github (or similar) but you still have an excellent wiki and issue tracker, and a lot of flexibility — Trac, in particular, has a lot of plugins.

Once you have any of the above running, the ease of use advantage lies with Git. have you ever tried to teach a non-geek to use Fossil tickets? I have, and its not easy. It cannot do email notifications easily and a lot of people rely of them, and that is probably enough to make Fossil’s issue tracking unusable in many scenarios.

On the desktop you have a choice of GUIs. I use Gitg, which is nicer to use than Fossil’s web interface, although it does seem to lack the ability to diff arbitrary commits. Apart from all the standalone GUIs there are plugins for file managers and GUIs.

Fossil’s wiki and tickets are stored in the repo and are distributed do they may still be preferable for people who spend a lot of time working offline. Other than that, Fossil’s advantages, other than the easier to use command line are pretty much negated.

Fossil’s developers claim that having everything in a single executable makes it easier to install. This may be true for operating systems like Windows that have primitive software installer (although, with Windows 8, even Windows has a “store”), but on (Ubuntu) Linux I just open the installer, type “git” into the search bar, tick “git” and “gitk”, and I have everything I need. If I want the latest version I am likely to need to compile Fossil, whereas I can install the latest version of Git from an Ubuntu PPA. On ease of installation and upgrading, there is no difference if you do not care about having the very latest version, and Git is easier if you do, at least if you are on a Unix family OS (including MacOS – I notice MacPorts currently has the latest version of Git).

I still prefer Fossil (just) because of its command line, and the simple work-flow it encourages (not requires). I also like the robust repository format (it is in a SQLite database, which is a very safe and well tested option), but deciding which is easier to use for a particular project is no obvious.

Other contenders

Of course Git and Fossil are not the only DVCSs in the world, still less are they the only VCSs. However, I prefer DVCSs (multiple copies of everything, means multiple backups of everything), and they are what I have real world experience of using. I strongly prefer open source (or at least easy export to a portable format) for any data of value (I have studied to much financial theory to ignore the option value in being able to switch). I want something actively developed, with a community and tools around it. The only real options, apart from Fossil and Git, seem to be Mercurial and Bazaar. Mercurial seems to be more widely used, but both have decent tool integration. Both seem to be a reasonable compromise between Fossil and Git, with a more user friendly command line than Git, but more of an ecosystem than Fossil. Bazaar development seems to have stagnated and it is the less popular of the two, so Mercurial seems a safer choice. More active development means that even the (contested) advantages Bazaar claims over mercurial are likely to be transient.

Both Mercurial and Bazaar are written in Python (which is good, as I am not any good at C), but Mercurial was chosen to develop Python itself in, partly because it is what Python developers (meaning the smart people who develop the language itself) prefer Mercurial. I currently prefer Fossil for my own projects, but I definitely want to try Mercurial.

Immediate problems for consumers

The root problem is the denial of consumer choice. I am writing on a laptop, which, being PC hardware, is extremely flexible. On it, I can install any version of Microsoft Windows I have install media for, any one of a number of Linux distributions (from fully fledged ones like Ubuntu, to Google’s cut-down Chromium/Chrome OS), Android, FreeBSD and its relatives, and a number of lesser known operating systems. I can also do the same with an Apple Mac (Macs are just PCs made by Apple). I cannot do this with tablets and smart phones.

The most immediate problem is the security problems this causes. Many Android devices cannot be upgraded, because users are dependent on the device vendor for updates, so security issues are not fixed. Any Android device quickly becomes highly vulnerable to viruses and hackers.

Apple is much better in this respect, but is deeply anti-consumer in its strategy of customer control and lock-in. Software must be bought through Apple’s App Store, and software that competes with Apple is banned. Apple takes a large  cut on the price books music and films bough through any app (or the app is banned). hardware works only with Apple devices (Android devices often work with PC hardware, from mice to mobile wireless dongles) to raise the price of switching to a competitor, media bought from Apple will only run on Apple devices (or, currently, on PCs with Apple software).

The long term threat to innovation

The leading tablet and phones operating systems are Android and Apple’s mobile version of MacOS. Android is based on Linux, and MacOS on a combination of FreeBSD and Mach. Linux, FreeBSD and Mach are open source operating systems that exist because PCs provided a cheap hardware platform that they could run on, giving them a much larger users base than would have been possible on specialist hardware and greatly bringing down the cost of development.

There also have been, and are, many experimental, research, and early stage operating systems that benefit from running on cheap hardware. These may come to obvious prominence by becoming widely adopted, or they may provide a a base to develop variants, or they may just pioneer ideas that later become more widely adopted.

Tablets could have made a great, cheap research platform. They could have provided scope for children to experiment with cheap devices and become the next generation of innovative software developers — as the Raspberry Pi allows but with a far more widely available mobile platform. Start-up companies could have modified and reprogrammed tablets to create completely new devices without needing the huge amounts of capital to create a custom device from scratch.  Learning, experimentation, and innovation are being pushed onto specialist hardware, and will therefore become less affordable.

The greatest danger is that PCs will become more like tablets. Microsoft has taken the first step towards this be requiring that all PCs that can run Window 8 supprt “secure boot” — i.e. they will only run operating systems that the manufacturer permits. At the moment, secure boot can be turned off — but for how long? If PCs become as closed as tablets, then what cheap platform will innovators have to work on? What platform will consumers have that disruptive innovation can run on?

So what is the real purpose? It varies, but the key in most cases is to control consumers. Consider the the Amazon Kindle and the DRM Amazon applies to books. Again, the content is available without DRM through the Kindle Cloud Reader: although, as far as I have been able to find out it is in any case fairly simple to remove the DRM from Kindle ebooks, so no one bothers the more awkward process of intercepting the DRM free content in Cloud Reader.

The real purpose of Kindle DRM is to make it expensive to switch to to another device. A consumer who has a collection of books and wants to switch to another ebook reader must purchase new copies of every single book they want to keep. This ties consumers down, maintaining market share against existing competitors and making it very difficult for a new entrant into the market to gain share at all. The tech savvy minority will strip the DRM, but for the majority of customers switching will simply not be an option.

Of course many services have exceptions to DRM: much of the music Apple sells is now DRM free, and Amazon Kindle software can be used to read Kindle books on an Android tablet (and the Cloud Reader to read on any platform), but the remainder is still a very high barrier to switching. Furthermore, none of these are guaranteed to remain available: they may turn out to be a  way of leading customers into lock-in, and may be withdrawn once their purpose has been sufficiently served. In Apples case its customers are locked in in multiple ways, not just DRM, so selling some content without DRM does not loosen the lock very much.

The bottom line is that DRM is a way of limiting competition; a way of routing around free market competition.