Bugzilla API 0.9 Released – INCOMPATIBLE CHANGES

Version 0.9 of the Bugzilla REST API has been released.

This version has INCOMPATIBLE CHANGES in the data format – some field and parameter names have changed (see below). Therefore, the “/latest” installation on https://api-dev.bugzilla.mozilla.org has not been updated with it yet. It will be updated on


Wednesday 2nd March

which is 1 month from today. At that point, all clients using that installation must be able to deal with both old and new formats.[0] If you want to only support one format, you can update your code and migrate to the /0.9 installation at any point before then.

Also, 0.9 fixes BzAPI to deal with some late-breaking Bugzilla 4.0 changes. If you are deploying Bugzilla 4.0, this is the version you will need.

People deploying installations of 0.9 against their own Bugzillas should take note of the “Upgrading” section below.

New In 0.9

  • Field name changes to match XML-RPC API
  • Bugzilla 4.0rc2 now fully supported

Compatibility Notes

  • Four field names on the “Bug” object have changed, to match the names chosen for the Bugzilla XML-RPC API':
    • reporter -> creator
    • is_reporter_accessible -> is_creator_accessible
    • token -> update_token
    • is_everconfirmed -> is_confirmed

    In each case, you will need to change your code to look for the first hash value and, if it’s not found, look for the second one.

  • The emailN_reporter field is now called emailN_creator, to match the
    above change.

  • Versions 0.6.1 and 0.7 are now no longer available on the api-dev
    server.

Upgrading Your BzAPI

BzAPI now comes with a Bugzilla extension, suitable for 3.6 and above, which replaces (most of) the patches that we used to ship. Unless you are on an earlier version of Bugzilla, please remove the old patches (except the XML-RPC one) and install the extension instead. You may want to symlink it so it automatically gets updated in step when you upgrade BzAPI.

Those choosing not to do this will still need to take a new version of config.json.tmpl from the patches/ directory.

File bugs | Feedback and discussion

[0] I apologise for not having a better backwards-compatibility story, but it was very difficult to update all the tests to support the data being duplicated in two field names. I tried, and gave up as it was taking too long. And this is pre-1.0 software (just).

Time Tracking in bugzilla.mozilla.org

Recently, two groups (one from Adobe and one from Mozilla Mobile) have asked for better facilities to track people’s work time on bugzilla.mozilla.org. Bugzilla does have a time-tracking function; however, it is quite UI-heavy, so we did not want to enable it globally. Bugzilla’s UI is already complex enough :-)

So, we have decided to enable it only for members of a particular group, the “timetrackers” group. People at Adobe and members of the Fennec team are automatically members, but no-one else is. In this way, we keep the extra UI complexity contained to those who want it.

However, we don’t want Bugzilla to contain information not accessible to the community (other than by specific policy, such as our security policy, or company-internal information). So we want to make it clear that anyone with the “editbugs” privilege can ask to be made a member of the “timetrackers” Bugzilla group, without needing to give a reason. Then, they can get the extra time-tracking UI on all bug views, and see what the Tamarin and Mobile teams are doing with it. We don’t expect many people to want to do this, but we think it’s important that it’s possible :-)

Profanivore – A Bugzilla Extension Which Eats Bad Words

Recently, an unusually persistent troll came into Bugzilla and started swearing at contributors. This was obviously an unpleasant experience for them, and led me to thinking how we can balance Mozilla’s desire to be a place where everyone is welcome (some of our most awesome contributors have started when under 16, for example) and yet we support the general principle of freedom of speech (which is most important in government-related censorship contexts, but is also important in general).

So a couple of weeks ago I wrote Profanivore, a Bugzilla extension which ‘eats’ English profanities in Bugzilla comments, leaving behind instead a trail of droppings (‘****’). The profanity is only eaten where the comment was written by a user who does not have the global ‘editbugs’ privilege. The digestion happens at display time, so the comment in the database is unaltered.

I wrote this very carefully to walk the line between being over-censorious and being protective of our users having to read abuse from trolls. Hence the “you can swear if you have editbugs” feature.

Unless this meets with serious objections, we plan to enable it on bugzilla.mozilla.org some time in the reasonably near future. If we do, please resist the temptation to test it. It uses a standard Perl module for detection so I have reasonable confidence it’ll do the right thing most of the time, but do let me know if it censors your Scunthorpe.

Bugzilla API 0.8 Released

Version 0.8 of the Bugzilla REST API has been released.

People deploying installations of 0.8 against their own Bugzillas should take note of the “Upgrading” section below.

New In 0.8

  • Use quicksearch syntax on the /bug search call with the parameter “quicksearch=“.
  • Non-ASCII bug data now works in more places (hopefully everywhere)
  • /configuration data is now cached and you can asked for the cached version if performance is more important to you than freshness; see

    https://wiki.mozilla.org/Bugzilla:REST_API:Methods#Configuration_.28.2Fconfiguration_GET.29

  • Bugzilla 4.0rc1 now fully supported

Compatibility Notes

  • “is_url” support for attachments has been removed from Bugzilla, and so from BzAPI (including against older versions of Bugzilla). BzAPI will emulate the backwards-compatibility behaviour of new Bugzillas when it encounters attachments which are URLs.
  • On the Comment object, “author” is now “creator”, to match a change made in the Bugzilla XML-RPC API for version 4.0.
  • Versions 0.6.1 is now no longer available on the api-dev server.

Upgrading Your BzAPI

BzAPI now comes with the patches necessary to install it, in the /patches subdirectory.

Those upgrading from pre-0.8 to 0.8 will need to:

  • Update their Bugzilla to the latest point release of their stable
    branch

  • Update their copy of $BUGZILLA_HOME/template/en/default/config.json.tmpl from that in the /patches directory
  • Apply a patch (xmlrpc-fix.diff) to their copy of BZ::Client’s Client.pm if they want bug submission with non-ASCII values for e.g. summary to work.

(File bugs | Feedback and discussion)

bugzilla.mozilla.org: Status Field Changes

Update: this is sadly not as easy technically as it should be (e.g. Bugzilla 3.6 has several hard-coded status names lurking in the code). This is definitely not happening on Tuesday, and may have to wait until we upgrade to 4.0. <sigh>.

Next Tuesday, 9th November When we upgrade to Bugzilla 4.0, the values of the Status field in bugzilla.mozilla.org are changing, as discussed in mozilla.dev.planning. This is to match us up with upstream changes, make it more clear what each state means, and to clarify which bugs are the responsibility of QA and which of Dev.

The changes are as follows:

  • NEW will be rename to CONFIRMED, so all existing bugs change
  • ASSIGNED will be renamed to IN_PROGRESS, ditto
  • READY_TO_FIX is a new state (see below)
  • REOPENED will be removed (reopened bugs go straight to CONFIRMED or IN_PROGRESS)

So the Statuses and their meanings will be as follows:

Status Meaning Responsible
UNCONFIRMED not confirmed; filed by inexperienced reporter QA
CONFIRMED can reproduce, not an obvious dupe QA
READY_TO_FIX has appropriate QA/UX work complete, ready for a developer Dev
IN_PROGRESS developer is working on a patch Dev
RESOLVED patch has been committed to trunk QA
VERIFIED fix has been QAed in builds n/a

Bugzilla 600,000 Bug Sweepstake Results

Milestone bugzilla.mozilla.org bug 600,000 was filed on 2010-09-27 at 13:44 ZST by Frank Wein (mcsmurf).

The winner of the sweepstake to guess the date and time is Karsten Düsterloh, who guessed 2010-10-07 20:00:00 – 10 days, 6 hours and 16 minutes out. Accuracy is still decreasing over the years; it looks like this is geting harder to predict. Most entrants thought it would take longer than it did to get here.

Karsten writes:

I’m a SeaMonkey developer and author of the Mnenhy extension. I got drawn into Mozilla with milestone Msomething last century and even dared to file a five digit dupe a while later. MailNews development was slow even back then, so I started my own add-on in late 2002 – and it’s still alive!

My current focus is upon SeaMonkey MailNews and the future of open internet communication.

Karsten wins a Firefox backpack. The two runners-up are Dave Garrett (12d 20h 26m out) and Wladimir Palant (14d 20h 07m out) – in Wladimir’s case, for the second year running. They both win their choice of a t-shirt from the Mozilla store. Dave writes:

I got drawn into the Mozilla world in 2007 and it somehow went from filing a few bugs and improving an extension to testing and triaging Firefox and AMO bugs and being the developer of the Flagfox extension with over a million users. I’m not quite sure how that happened.

Wladimir writes:

I got involved with Mozilla in 2003 shortly after switching to Mozilla Suite – playing around with add-ons was just natural, it didn’t take long for me to start fixing issues in the ones I used. Soon I was writing my own extensions and eventually ended up creating Adblock Plus which turned into a very time-consuming hobby.

Bugzilla API 0.6.1 Released

Version 0.6.1 of the Bugzilla REST API has been released, and the main api-dev server should be back up and working fine. If it’s not, let me know.

New In 0.6.1

  • Compatibility with Bugzilla 3.6. If you are using the 0.6 version
    against bugzilla.mozilla.org or another installation of Bugzilla 3.6,
    upgrade, because some of your calls may no longer work.

Compatibility Notes

  • Two things are still not working: JSONP, and the new_since parameter to the /comments call.
  • Version 0.5 is now no longer available on the api-dev server.

File bugs | Feedback and discussion

BzAPI Unwell

Some parts of the bugzilla.mozilla.org BzAPI are a little unwell at the moment, following the upgrade to 3.6. I have fixes in hand, and hope to ship a new, working version tomorrow.

1000 Reported Bugzilla Installations

I just added the 1000th entry to the Bugzilla Installation List. Almost all of that list (a few were originally found by web trawling) is made up of companies who took the time to get in touch with us and thank us for writing Bugzilla. Which is really encouraging. :-)

I prune the list every few months to remove dead links and defunct companies, but of course we can’t tell if a company stops using Bugzilla and doesn’t tell us. Then again, Max tells me that, from update pings, he can tell that there are at least ten times that number of active installations out there. So the figures have uncertainty in both directions.

Still, it’s a pretty cool milestone.

Bugzilla API 0.6 Released

Version 0.6 of the Bugzilla REST API has been released.

New In 0.6

  • Include and exclude fields support, so you can define exactly what data you want each call to return;
  • Cookie auth – if you have access to a user’s existing Bugzilla login cookie information, you can use that instead of prompting them for a username and password;
  • Performance improvements for /configuration call, with recent Bugzillas (not yet available on bugzilla.mozilla.org);
  • Addition of comments with attachment updates now works properly.

Compatibility Notes

  • Some of the previous methods of including and excluding specific fields no longer work, although “attachmentdata=1″ is still supported as a shorthand;
  • Versions 0.4.1 and below had a security issue[0] which could, under some circumstances, allow one user to impersonate another. These versions are now disabled on the server, and the bug has been opened.

File bugs | Feedback and discussion

Bugzilla API 0.5 Released

I noticed I forgot to blog this…

Version 0.5 of the Bugzilla REST API has been released.

Note: versions 0.4.1 and below have a security issue[0] which could, under some circumstances, allow one user to impersonate another. If you are specifically using these versions (rather than the “latest” version) please update your client software to use version 0.5.

New In 0.5:

  • Performance instrumentation. I have added code which records how long requests take, so I can see whether the bottlenecks are, if any. If you are having performance issues with the API, please file bugs and tell me what is slow!
  • JSONP support. Use Accept: application/javascript and pass a callback=myfunction parameter, and you will get JSONP served as application/javascript.

Compatibility Notes:

  • The range of content types that the API can return is now restricted
    to:

    • YAML-HTML (for browsing) – text/html
    • JSON – application/json
    • JSONP – application/javascript
    • XML (utterly unfrozen!) – application/xml
  • The deprecated “count=1″ parameter to bug searches to get a count has been removed.

File bugs | Feedback and discussion

[0] This bug will be opened in a few days.

Community Size, and Bug Quality

I’ve just finished watching Diederik van Liere’s fascinating presentation (Ogg video, 20 mins), linked to by Mark, about his work with bugzilla.mozilla.org data. Download the full slide deck – you’ll need it to follow along with my comments, which should be of interest to the community-minded.

Slides 9 and 10, the community dynamics slides, are worrying. Diederik explains the decline in the size of the community as “increased product maturity”, but I’m not sure that’s true. If it were, we would expect to see the number of people entering the community decreasing. However, that number has stayed roughly constant, with peaks surrounding a release as we reach out to beta testers, and as new users report bugs. It’s the number of people leaving which seems to be increasing. Fewer people are hanging around once they get here. And I don’t know why.

Slide 11, about what the final resolutions ended up being of bugs filed in each year, is fascinating. It would be really interesting to see that graphed at a greater granularity.

We have made significant progress in reducing the percentage of duplicate bugs filed. People complain about Bugzilla’s search, but the percentage was a flat 25% from 2002-2004, and it’s now a flat 13% – basically half what it was. Also, from a low of 34% of bugs filed being FIXED, we are now up at 56%.

There are no doubt lots of factors involved in this effect, including the reduction in the size of the community mentioned in the earlier slides. I’m really interested in working out what we might have done which has improved the situation. I have an utterly open mind on this. here are some significant events I can think of which may be relevant:

Can anyone else think of other events or software deployments which may be relevant? I’m happy to do the research to find out when they happened.

From The Wind

It’s not possible to search Bugzilla for bugs where a particular flag has been set with no requestee. This means that people sometimes request e.g. review “from the wind” (as bugbot puts it) and no-one notices, so their patch sits there for ages, unloved.

Here is a Greasemonkey userscript which adds a “From the Wind” button at the bottom of buglist pages on bugzilla.mozilla.org. Clicking that button (after a few seconds) trims the list down to only those bugs where “review” (on an attachment) or “approval” has been requested from the wind. (It does this by fetching the bug data from the API and doing the check itself.)

So if you search for all bugs in the relevant component or product with “review?”, and then click the button, it’ll trim that list down to all bugs with “review?” and no requestee.

Why not take a moment to do this in a product or component for which you have review responsibilities?

Why didn’t I use Jetpack? Because at the moment (Firefox 3.6b4, Jetpack 0.7) the authoring experience is really not good. The API documentation doesn’t even mention that it uses jQuery (which led to me thinking for a long time that it was missing huge chunks of function), the Bespin code window is way too small and non-resizable, using Ctrl-X/C/V cut-copy-paste scrolls the page to the top, errors appear in the Firebug on the Jetpack page rather than the web page itself, you have to keep clicking “Try out this code” and “clear” on the Firebug on both pages, and I can’t see how to step through the code using the debugger.

Bugzilla API 0.4 Released

Version 0.4 of the Bugzilla REST API has been released. New in this version:

  • New /count call – get fast bug counts, either singly or in 1D, 2D or 3D tables. Great for stats.

Compatibility Notes:

  • The “count=1″ parameter to bug searches to get a count is now deprecated, and will be removed in a future release. Use the above API call instead.
  • The Configuration object’s “groups” hash will soon change to be keyed by ID rather than name (and so the “id” field has disappeared, to be replaced by a “name” field). Tracking bug.

File bugs | Give feedback.