Eurovision Bingo

Some people say that all Eurovision songs are the same. That’s probably not quite true, but there is perhaps a hint of truth in the suggestion that some themes tend to recur from year to year. Hence, I thought, Eurovision Bingo.

I wrote some code to analyse a directory full of lyrics, normally those from the previous year of the competition, and work out the frequency of occurrence of each word. It will then generate Bingo cards, with sets of words of different levels of commonness. You can then use them to play Bingo while watching this year’s competition (which is on Saturday).

There’s a Github repo, or if you want to go straight to pre-generated cards for this year, they are here.

Here’s a sample card from the 2014 lyrics:

fell cause rising gonna rain
world believe dancing hold once
every mean LOVE something chance
hey show or passed say
because light hard home heart

Have fun :-)

You Couldn’t Make It Up

I was in the middle of debugging some code when a background Slashdot tab from 10 minutes ago suddenly started playing a sponsored video. Truly and genuinely, the opening of this video contained the following:

Did you know that it takes you 15 minutes to get back into the work zone after being interrupted by an alert or message?

Yes. Yes, Slashdot, I did…

R.I.P. api-dev.bugzilla.mozilla.org

Complaining about bugzilla.mozilla.org (BMO) is a Mozilla project activity as old as the hills. Back in 2009, it was realised by the Foundation that to make everyone happy was (and still is) an impossible task, and I was given a mandate to “help people solve their own problems”. So around September 2009, I released the first version of my Bugzilla API proxy software, BzAPI. This software presented a clean, well-documented RESTful interface on the front end, and did all sorts of things on the back end (XML, CSV, RPC, HTML scraping) that developers no longer had to worry about. We made a dev server VM for it so people could try it out – api-dev.bugzilla.mozilla.org.

It was popular. Extremely popular. People started building things, and then more things, all of which depended on this server for Bugzilla data. For various reasons, IT never got around to building a production instance, and so over the last five years, I’ve been maintaining this core piece of Mozilla project infrastructure, which was depended on by TBPL and many, many other tools which interfaced with Bugzilla. At its peak, it serviced 400,000 requests per day.

Over the intervening years, BMO itself acquired a REST API which slowly became more capable, and then a BzAPI-compatible API shim was implemented on top of it by the excellent dkl, so people could change their code to access BMO directly just by updating the endpoint URL. After a few false starts, requests to api-dev.bugzilla.mozilla.org are now served directly by BMO, via that shim code. Earlier today, the api-dev VM was finally powered down.

Here’s to you, api-dev. Good job.

Using Instantbird to Connect to IRC Servers Requiring a Username and Password

[Update 2014-01-16: A point of clarification. There are two possible ways to send a password for IRC. One is supported in the Instantbird UI – it’s the one that automatically identifies your nick with NickServ, the bot which makes sure people don’t steal other people’s nicks. The other, which is rarer but which I needed, involves sending a password to connect at all, using the PASS command in the IRC protocol. That is what is documented here.]

I was trying to do this; turns out it currently requires about:config manipulation and is not documented anywhere I can find.

Using about:config (type /about config in a message window, or access via Preferences), set the following prefs:

messenger.account.accountN.options.serverPassword
messenger.account.accountN.options.username

to the obvious values. Other useful tip: if the IRC server uses a self-signed cert, connect to it on the right port using Firefox and HTTPS, and you can save the cert out of the warning/exception dialog you get. You can then import it into Instantbird using the deeply-buried Certificate section of the Advanced Preferences and it will trust the cert and connect. (I think this is what I did, although memory is hazy.)

slic 1.0.0 Released

I’ve just released version 1.0 of some new software called slic, which I’ve been using to do license analysis on Firefox OS. From the README:

This is slic, the Speedy LIcense Checker. It scans a codebase and identifies the license of each file. It can also optionally extract the license text and a list of copyright holders – this is very useful for meeting the obligations of licenses which require reproduction of license text. It outputs data in a JSON structure (by default) which can then be further processed by other tools.

It runs at about 120 files per second on a single core of a 3GHz Intel I7 (it’s CPU-bound, at least on a machine with an SSD). So you can do 100,000 files in less than 15 minutes. Parallel invocation is left as an exercise for the reader, but could easily be bolted on the side by dividing up the inputs.

The code is Python, and it uses a multi-stage regexp-based classifier, so that with families of licenses it starts with a more generic classification and then refines it via checking various sub-possibilities. Future potential enhancements include a hash-based cache to avoid doing the same work more than once, and integration with a popular online spreadsheet tool to help manage exceptions and manual license determinations.

Ctrl-Shift-T Not Working in Ubuntu

I just spent ages debugging the fact that my Ctrl-Shift-T key stopped working in Ubuntu. This is my attempt to write a blog post that search engines will find if anyone else has the same problem. Ctrl-Shift-T is “new tab” in Terminal, and “reopen last closed tab” in Firefox, so it’s rather useful, and annoying when not working.

After lots of research and debugging, it turned out to be the following. I recently installed a bunch of VoIP apps, including Jitsi. Jitsi decided to run on startup, in a way which was invisible in the UI, and override a bunch of my keybindings, in a way which didn’t make it clear that it was the app capturing them. :-( So if anyone ever finds Ctrl-Shift keybindings stop working (seems like Jitsi takes A, H, P, L, M and T), and sees output like this in xev:

FocusOut event, serial 37, synthetic NO, window 0x5400001,
    mode NotifyGrab, detail NotifyAncestor

FocusIn event, serial 37, synthetic NO, window 0x5400001,
    mode NotifyUngrab, detail NotifyAncestor

KeymapNotify event, serial 37, synthetic NO, window 0x0,
    keys:  34  0   0   0   32  0   4   0   0   0   0   0   0   0   0   0   
           0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 

instead of the expected KeyPress / KeyRelease events, then look for some app which has pinched your keys. I spent ages looking through the Compiz config settings, finding nothing, but then started killing processes until I found my keys were working again.

Survey on FLOSS Contribution Policies

In the “dull but important” category: my friend Allison Randal is doing a survey on people’s attitudes to contribution policies (committer’s agreements, copyright assignment, DCO etc.) in free/libre/open source software projects. I’m rather interested in what she comes up with. So if you have a few minutes (it should take less than 5 – I just did it) to fill in her survey about what you think about such things, she and I would be most grateful:

http://survey.lohutok.net is the link. You want the “FLOSS Developer Contribution Policy Survey” – I’ve done the other one on Mozilla’s behalf.

Incidentally, this survey is notable as I believe it’s the first online multiple-choice survey I’ve ever taken where I didn’t think “my answer doesn’t fit into your narrow categories” about at least one of the questions. So it’s definitely well-designed.

Kingdom Code UK

Kingdom Code is a new initiative to gather together Christians who program, to direct their efforts towards hastening the eventual total triumph of God’s kingdom on earth. There’s a preparatory meet-up on Monday 15th September (tickets) and then a full get-together on Monday 13th October. Check out the website and sign up if you are interested.

(There’s also Code for the Kingdom in various cities in the US and India, if you live nearer those places than here.)

Accessing Vidyo Meetings Using Free Software: Help Needed

For a long time now, Mozilla has been a heavy user of the Vidyo video-conferencing system. Like Skype, it’s a “pretty much just works” solution where, sadly, the free software and open standards solutions don’t yet cut it in terms of usability. We hope WebRTC might change this. Anyway, in the mean time, we use it, which means that Mozilla staff have had to use a proprietary client, and those without a Vidyo login of their own have had to use a Flash applet. Ick. (I use a dedicated Android tablet for Vidyo, so I don’t have to install either.)

However, this sad situation may now have changed. In this bug, it seems that SIP and H.263/H.264 gateways have been enabled on our Vidyo setup, which should enable people to call in using standards-compliant free software clients. However, I can’t get video to work properly, using Linphone. Is there anyone out there in the Mozilla world who can read the bug and figure out how to do it?

To Serve Users

My honourable friend Bradley Kuhn thinks Mozilla should serve its users by refusing to give them what they want.

[Clarificatory update: I wrote this post before I’d seen the official FSF position; the below was a musing on the actions of the area of our community to which Bradley ideologically belongs, not an attempt to imply he speaks for the FSF or wrote their opinion. Apologies if that was not clear. And I’m a big fan of (and member of) the FSF; the below criticisms were voiced by private mail at the time.]

One weakness I have seen in the FSF, in things like the PlayOgg and PDFReaders campaigns, is that they think that lecturing someone about what they should want rather than (or before) giving them what they do want is a winning strategy. Both of the websites for those campaigns started with large blocks of text so that the user couldn’t possibly avoid finding out exactly what the FSF position was in detail before actually getting their PDF reader or playback software. (Notably missing from the campaigns, incidentally, were any sense that the usability of the recommended software was at all a relevant factor.)

Bradley’s suggestion is that, instead of letting users watch the movies they want to watch, we should lecture them about how they shouldn’t want it – or should refuse to watch them until Hollywood changes its tune on DRM. I think this would have about as much success as PlayOgg and PDFReaders (link:pdfreaders.org: 821 results).

It’s certainly true that Mozilla has a different stance here. We have influence because we have market share, and so preserving and increasing that market share is an important goal – and one that’s difficult to attain. And we think our stance has worked rather well; over the years, the Mozilla project has been a force for good on the web that other organizations, for whatever reason, have not managed to be. But we aren’t invincible – we don’t win every time. We didn’t win on H.264, although the deal with Cisco to drive the cost of support to $0 everywhere at least allowed us to live to fight another day. And we haven’t, yet, managed to find a way to win on DRM. The question is: is software DRM on the desktop the issue we should die on a hill over? We don’t think so.

Bradley accuses us of selling out on our principles regarding preserving the open web. But making a DRM-free web is not within our power at the moment. Our choice is not between “DRM on the web” and “no DRM on the web”, it’s between “allow users to watch DRMed videos” and “prevent users from watching DRMed videos”. And we think the latter is a long-term losing strategy, not just for the fight on DRM (if Firefox didn’t exist, would our chances of a DRM-free web be greater?), but for all the other things Mozilla is working for. (BTW, Mitchell’s post does not call open source “merely an approach”, it calls it “Mozilla’s fundamental approach”. That’s a pretty big misrepresentation.)

Accusing someone of having no principles because they don’t always get their way when competing in markets where they are massively outweighed is unfair. Bradley would have us slide into irrelevance rather than allow users to continue to watch DRMed movies in Firefox. He’s welcome to recommend that course of action, but we aren’t going to take it.

Using Deja Dup (Ubuntu Built-In Backup) with rsync.net

This is one of those posts which is trying to save other people the hassle I’ve had.

Ubuntu One is shutting down. So I want to switch some backups over to use my rsync.net account. With U1, I used Ubuntu’s built in backup system, which is based on a program called Deja Dup. I would like to use that program with rsync.net too.

However, Deja Dup auto-prepends a “/” to the beginning of the path you give it. (There’s a bug on this behaviour.) And I don’t have root on the rsync.net server. (A good thing.)

So, in order to use Deja Dup with a server where you don’t have root, you need to find and give it a full path. Execute:

ssh 42@ch-s001.rsync.net pwd

(where 42 is your user number, and ch-s001 your server name) to find out the path to your home directory, and then, in the Deja Dup settings, prepend that to the name of the directory you want to store your backups in. You’ll also need to set up passwordless login.

Why Is Email So Hard?

I want programs on my machine to be able to send mail. Surely that’s not too much to ask. If I wanted it to Tweet, I’m sure there are a dozen libraries out there I could use. But sending mail… that requires the horror that is sendmail.rc or trying to configure Postfix or Exim, which is the approximate equivalent of using an ocean-going liner to cross a small creek.

Or does it? I just discovered “ssmtp“, which is supposed to be a program which does the very simple thing of accepting mail and passing it on to a configured mailserver – GMail, your own, or whatever. All you have to do is tell it your server name, username and password. Simple, right?

Turns out, not so simple. I kept getting auth failures, whichever server I tried. ssmtp’s verbose mode seemed to show no password being shown, but that’s because it doesn’t show passwords in logs. Once I finally configured a mailserver on port 25 (so no encryption) and broke out Wireshark, it turned out that ssmtp was sending a blank password. But why?

The config file doesn’t have sample entries for the “username” and “password” parameters, so I typed them in. Turns out, the config key for the password is “AuthPass”, not “AuthPassword”. And ssmtp doesn’t think to even say “Hey, you are trying to log in without a password. No-one does that. Are you sure you’re not on crack?” It happily sends off a blank password, because that’s clearly what you wanted if you left the password out entirely.

<sigh> There’s 45 minutes of my life I won’t get back.