7 Hours…

…is now the absolute maximum time you have to patch your website after a remote vulnerability is announced in software you are using. The path from patch -> Internet-wide attack can be at least that short.

This is interesting in a number of ways; firstly, the time of day at which you publish your vulnerability will advantage or disadvantage people in different geographies. Want to put US companies at higher risk? Publish your vulnerability at 10pm Pacific Time. Not happy with the Koreans for some reason? Publish at 10pm Korean time.

It means hosting companies now need (if they didn’t have them before) 24-hour security teams watching all possible announcement points, and the ability to deploy patches across their entire base of hosts (tens of thousands of machines) in a small number of hours. There isn’t time for an audit – “Am I using Software X version Y anywhere?” You need to know that already, and where.

Rebel Alliance Ideas

Chris Beard has been encouraging us to think like the rebels; what can we do that other people won’t do? How can we make an impact?

Here are some of my thoughts:

  • The internet, in global average, is getting less reliable, slower and more laggy. Finish Janus and persuade our mobile partners to deploy it and default to it. Your Firefox OS phone now accesses the net faster than an Android phone.
  • Make Firefox OS connect by default to openwireless.org access points, and encourage Firefox OS users to run them. There’s a virtuous circle here. More net in more places; a global movement of being generous with net access.
  • Finish Daala faster, by finding people other than the core team to do everything except design the codec and write the algorithms (e.g., testing, speed optimizations, fuzzing, writing Windows Media plugins). We need to get the word out that this project is critical.
  • Show the core free software community, who have great influence over tech choices and who should be our natural allies, that we care about them. Be the first organization ever to make a free-from-bottom-to-top mobile phone (running Firefox OS) and give some help to the Replicant team to port to it as well, just to prove we mean it and it’s real.
  • Make it possible to search for specifically open source software in the Marketplace, and show we believe it “promotes the development of the Internet as a public resource” by promoting apps which are open source.
  • Ship Collusion (which has been in the works for years), even if there’s not a perfect mapping between what it shows you and what’s actually bad. Make sites feel they have to justify all their 3rd party links.

What are your ideas?

New Class of Vulnerability in Perl Web Applications

We did a Bugzilla security release today, to fix some holes responsibly disclosed to us by Check Point Vulnerability Research, to whom we are very grateful. The most serious of them would allow someone to create and control an account for an arbitrary email address they don’t own. If your Bugzilla gives group permissions based on someone’s email domain, as some do, this could be a privilege escalation.

(Update 2014-10-07 05:42 BST: to be clear, this pattern is most commonly used to add “all people in a particular company” to a group, using an email address regexp like .*@mozilla.com$. It is used this way on bugzilla.mozilla.org to allow Mozilla Corporation employees access to e.g. Human Resources bugs. Membership of the Mozilla security group, which has access to unfixed vulnerabilities, is done on an individual basis and could not be obtained using this bug. The same is true of BMO admin privileges.)

These bugs are actually quite interesting, because they seem to represent a new Perl-specific security problem. (At least, as far as I’m aware it’s new, but perhaps we are about to find that everyone knows about it but us. Update 2014-10-08 09:20 BST: everything old is new again; but the level of response, including changes to CGI.pm, suggest that this had mostly faded from collective memory.) This is how it works. I’m using the most serious bug as my example. The somewhat less serious bugs caused by this pattern were XSS holes. (Check Point are going to be presenting on this vulnerability at the 31st Chaos Communications Congress in December in Hamburg, so check their analysis out too.)

Here’s the vulnerable code:

my $otheruser = Bugzilla::User->create({
    login_name => $login_name, 
    realname   => $cgi->param('realname'), 
    cryptpassword => $password});

This code creates a new Bugzilla user in the database when someone signs up. $cgi is an object representing the HTTP request made to the page.

The issue is a combination of two things. Firstly, the $cgi->param() call is context-sensitive – it can return a scalar or an array, depending on the context in which you call it – i.e. the type of the variable you assign the return value to. The ability for functions to do this is a Perl “do what I mean” feature.

Let’s say you called a page as follows, with 3 instances of the same parameter:


If you call param() in an array context (the @ sigil represents a variable which is an array), you get an array of values:

@values = $cgi->param('foo');
['bar', 'baz', 'quux']

If you call it in a scalar context (the $ sigil represents a variable which is a scalar), you get a single value, probably the first one:

$value = $cgi->param('foo'); 

So what context is it being called in, in the code under suspicion? Well, that’s exactly the problem. It turns out that functions called during hash value assignment are evaluated in a list context. However, when the result comes back, that value or those values are assigned to be part of uthe hash as if they were a set of individual, comma-separated scalars. I suspect this behaviour exists because of the close relationship of lists and hashes; it allows you to do stuff like:

my @array = ("foo", 3, "bar", 6);
my %hash = @array;
{ "foo" => 3, "bar" => 6 }

Therefore, when assigning the result of a function call as a hash value, if the return value is a single scalar, all goes as you would expect, but if it’s an array, the second and subsequent values end up being added as key/value pairs in the hash as well. This allows an attacker to override values already in the hash (specified earlier), which may have already been validated, with values controlled by them. In our case, real_name can be any string, so doesn’t need validation, but login_name most definitely does, and it already has been by the time this code is called.

So, in the case of the problematic code above, something like:


would end up overriding the already-validated login_name variable, giving the attacker control of the value used in the call to Bugzilla::User->create(). Oops.

We found 15 instances of this pattern in our code, four of which were exploitable to some degree. If you maintain a Perl web application, you may want to audit it for this pattern. Clearly, CGI.pm param() calls are the first thing to look for, but it’s possible that this pattern could occur with other modules which use the same context-sensitive return feature. The generic fix is to require the function call to be evaluated in scalar context:

my $otheruser = Bugzilla::User->create({
    login_name => $login_name, 
    realname   => scalar $cgi->param('realname'), 
    cryptpassword => $password});

I’d say it might be wise to not ever allow hash values to be assigned directly from functions without a call to scalar.

Prevent Territoriality

Watch out for participants who try to stake out exclusive ownership of certain areas of the project, and who seem to want to do all the work in those areas, to the extent of aggressively taking over work that others start. Such behavior may even seem healthy at first. After all, on the surface it looks like the person is taking on more responsibility, and showing increased activity within a given area. But in the long run, it is destructive. When people sense a “no trespassing” sign, they stay away. This results in reduced review in that area, and greater fragility, because the lone developer becomes a single point of failure. Worse, it fractures the cooperative, egalitarian spirit of the project. The theory should always be that any developer is welcome to help out on any task at any time.

— Karl Fogel, Producing Open Source Software

Licensing Policy Change: Tests are Now Public Domain

I’ve updated the Mozilla Foundation License Policy to state that:

PD Test Code is Test Code which is Mozilla Code, which does not carry an explicit license header, and which was either committed to the Mozilla repository on or after 10th September 2014, or was committed before that date but all contributors up to that date were Mozilla employees, contractors or interns. PD Test Code is made available under the Creative Commons Public Domain Dedication. Test Code which has not been demonstrated to be PD Test Code should be considered to be under the MPL 2.

So in other words, new tests are now CC0 (public domain) by default, and some or many old tests can be relicensed as well. (We don’t intend to do explicit relicensing of them ourselves, but people have the ability to do so in their copies if they do the necessary research.) This should help us share our tests with external standards bodies.

This was bug 788511.

Survey on FLOSS Contribution Policies

In the “dull but important” category: my friend Allison Randal is doing a survey on people’s attitudes to contribution policies (committer’s agreements, copyright assignment, DCO etc.) in free/libre/open source software projects. I’m rather interested in what she comes up with. So if you have a few minutes (it should take less than 5 – I just did it) to fill in her survey about what you think about such things, she and I would be most grateful:

http://survey.lohutok.net is the link. You want the “FLOSS Developer Contribution Policy Survey” – I’ve done the other one on Mozilla’s behalf.

Incidentally, this survey is notable as I believe it’s the first online multiple-choice survey I’ve ever taken where I didn’t think “my answer doesn’t fit into your narrow categories” about at least one of the questions. So it’s definitely well-designed.

Praise and Criticism

Praise and criticism are not opposites; in many ways, they are very similar. Both are primarily forms of attention, and are most effective when specific rather than generic. Both should be deployed with concrete goals in mind. Both can be diluted by inflation: praise too much or too often and you will devalue your praise; the same is true for criticism, though in practice, criticism is usually reactive and therefore a bit more resistant to devaluation.

— Karl Fogel, Producing Open Source Software

Wounds from a friend can be trusted, but an enemy multiplies kisses.

Proverbs 27:6

Kingdom Code UK

Kingdom Code is a new initiative to gather together Christians who program, to direct their efforts towards hastening the eventual total triumph of God’s kingdom on earth. There’s a preparatory meet-up on Monday 15th September (tickets) and then a full get-together on Monday 13th October. Check out the website and sign up if you are interested.

(There’s also Code for the Kingdom in various cities in the US and India, if you live nearer those places than here.)

Google Safe Browsing Now Blocks “Deceptive Software”

From the Google Online Security blog:

Starting next week, we’ll be expanding Safe Browsing protection against additional kinds of deceptive software: programs disguised as a helpful download that actually make unexpected changes to your computer—for instance, switching your homepage or other browser settings to ones you don’t want.

I posted a comment asking:

How is it determined, and who determines, what software falls into this category and is therefore blocked?

However, this question has not been approved for publication, let alone answered :-( At Mozilla, we recognise exactly the behaviour this initiative is trying to stop, but without written criteria, transparency and accountability, this could easily devolve into “Chrome now blocks software Google doesn’t like.” Which would be concerning.

Firefox uses the Google Safe Browsing service but enhancements to it are not necessarily automatically reflected in the APIs we use, so I’m not certain whether or not Firefox would also be blocking software Google doesn’t like, and if it did, whether we would get some input into the list.

Someone else asked:

So this will block flash player downloads from https://get.adobe.com/de/flashplayer/ because it unexpectedly changed my default browser to Google Chrome?!

Kudos to Google for at least publishing that comment, but it also hasn’t been answered. Perhaps this change might signal a move by Google away from deals which sideload Chrome? That would be most welcome.

Email Account Phishers Do Manual Work

For a while now, criminals have been breaking into email accounts and using them to spam the account’s address book with phishing emails or the like. More evil criminals will change the account password, and/or delete the address book and the email to make it harder for the account owner to warn people about what’s happened.

My mother recently received an email, purportedly from my cousin’s husband, titled “Confidential Doc”. It was a mock-up of a Dropbox “I’ve shared an item with you” email, with the “View Document” URL actually being http://proshow.kz/excel/OLE/PPS/redirect.php. This (currently) redirects to http://www.affordablewebdesigner.co.uk/components/com_wrapper/views/wrapper/tmpl/dropbox/, although it redirected to another site at the time. That page says “Select your email provider”, explaining “Now, you can sign in to dropbox with your email”. When you click the name of your email provider, it asks you for your email address and password. And boom – they have another account to abuse.

But the really interesting thing was that my mother, not being born yesterday, emailed back saying “I’ve just received an email from you. But it has no text – just an item to share. Is it real, or have you been hacked?” So far, so cautious. But she actually got a reply! It said:

Hi <her shortened first name>,
I sent it, It is safe.
<his first name>

(The random capital was in the original.)

Now, this could have been a very smart templated autoresponder, but I think it’s more likely that the guy stayed logged into the account long enough to “reassure” people and to improve his hit rate. That might tell us interesting things about the value of a captured email account, if it’s worth spending manual effort trying to convince people to hand over their creds.

HSBC Weakens Their Internet Banking Security

From a recent email about “changes to your terms and conditions”. (“Secure Key” is their dedicated keyfob 2-factor solution; it’s currently required both to log in and to pay a new payee. It’s rather well done.)

These changes will also enable us to introduce some enhancements to our service over the coming months. You’ll still have access to the full Internet Banking service by logging on with your Secure Key, but in addition, you’ll also be able log in to a limited service when you don’t use your Secure Key – you’ll simply need to verify your identity by providing other security information we request. We’ll contact you again to let you know when this new feature becomes available to you.

Full details of all the changes can be found below which you should read carefully. If you choose not to accept the changes, you have the right to ask us to stop providing you with the [Personal Internet Banking] service, before they come into effect. If we don’t hear from you, we’ll assume that you accept the changes.

Translation: we are lowering the security we use to protect your account information from unauthorised viewing and, as long as you still want to be able to access your account online at all, there’s absolutely nothing you can do about it.