We as an organisation made what I see as an unfortunate mistake in our Summer of Code project proposal process. We had the scratchpad for possible SoC projects be the same thing as the official list of projects – i.e. here, on the mozilla.org wiki, which is a public wiki on which anyone can have an account. The only clue as to whether an idea was “more official” was if a mentor’s name had been placed in the Mentor column; and even then, you had no way to tell whether that person had put their name there themselves.
This resulted in some unfortunate effects – primarily, students putting time and effort into applications which were bound to be unsuitable for a number of reasons.
I’m not going to be too specific about which projects I mean, as my point is not to criticise the people who put ideas on the wiki. It’s our fault that we didn’t make a clear separation between idea brainstorming and approved projects. But some of the problems with some projects in our list were:
- Project too small. There were a number of ideas that, while worthy, were clearly not worth $4500 to implement.
- Project too large. Conversely, some projects were quite clearly impossible to complete in eight weeks.
- Project too vague. “Improve foo” or “Provide a better experience for bar” leave the student with a lot of work to do to make their task concrete.
- Project premise incorrect. One project was to finish off something “started in last year’s SoC”, which actually wasn’t! Applicants for that proposal had clearly not done much research anyway but still, it would have been better if it was not there.
- Project pulled in multiple directions. Someone added a project idea, and someone added a related idea to it, until the project plan became ill-defined and near-contradictory.
- Project simply a bad idea. There was one particular project which had ten applicants but, on closer inspection, was clearly a terrible idea.
In summary, I think that next year we need a brainstorming page, and a separate “approved projects” page containing only those projects which have been assessed for size, scope and desireability and in which mentors have expressed an interest. In fact, it would be great if the Approved Projects system was part of the Google application webtool; that way, the mentors could easily sort the application list and assess the five people who want to do the same thing side by side. (Of course, you can apply to the SoC to do anything – and the tool would need to reflect that possibility too.)
I’d be interested in the experience other projects had in putting their lists together, and the methodology they used.