The introduction of Vector 2022 skin into en.wiki

[This started out as a response for [[Wikipedia:Requests for comment/Evaluation of Vector 2022]] but turned out to be expansive and too-broad-scope for there. It's intended for those familiar with the basics of en.wiki and WMF. ]

The introduction of Vector 2022 skin into en.wiki was a disaster by pretty much any metric.

Huge amounts of heat but very little light was generated on public-facing wikis (mainly en.wiki) and tools (mainly phabricator) and reading between the lines the process was very unpleasant for WMF staffers and the en.wiki admins. The end result was that a new skin containing modest improvements (mainly maintenance fixes) was adopted at the cost of huge ill-will.

Given that regular UI changes in web-based SaaS systems have been de rigueur for more than a decade, how did we get to the point where this change was so contentious?

1. It wasn’t about the technical content of the change. The changes were technically boring, competently implemented and worked reliably in the overwhelming proportion of situations for the overwhelming proportion of editors.
2. It wasn’t about the intention of the WMF staffers directly involved in the process. All the WMF staffers appeared to behave professionally and appropriately.
3. It wasn’t about the intention of the en.wiki admins. All the en.wiki admins appeared to behave appropriately.
4. It may have been partly about the huge pool of en.wiki editors who are deeply invested in the project, each of whom with their own point of view, set of priorities and fields of expertise. This, however, is a fundamental strength of the project (both Wikipedia as a whole and en.wiki specifically).

Systematic issues

en.wiki is a volunteer-edited information system running on systems provided by the professionally-staffed WMF. The volunteer side, while not explicitly a social-media forum, certainly shares aspects with social-media fora including, unfortunately, pile-ons. The en.wiki response to Vector 2022 was a classic pile-on: a community responded to a technical event in an emotionally charged manner with many people expressing very similar strongly-held views in such a way that emotive content completely obscured any informative content.

Indeed, the en.wiki policy WP:!VOTE policy encourages on-wiki pile-ons by explicitly prohibiting votes and vote-like processes unless each voter makes a substantive argument. Those substantive arguments can get very emotive.

Causes

1. The boundary between the volunteer-run and professionally-staffed portions of en.wiki is brittle, current processes and arrangements ensure that making technical changes to en.wiki is an all-or-nothing big-bang operation which is very costly to all concerned.
2. Technical changes to the en.wiki platform are seen by en.wiki editors as coming from “elsewhere” and being done to them, setting up an in-group and an out-group, with the WMF consistently being the out-group.
3. en.wiki continues to allow pile-ons.

Concrete ideas for WMF

Some of these ideas aim to 'soften' the boundary between the volunteer-run and professionally-staffed portions of en.wiki, increasing the proportion of editors the skills, knowledge and insight to better understand the underlying infrastructure and technologies. Other ideas aim to increase to availability of relevant academic studies in related areas.

1. Consider recasting wiki infrastructure updates to make WMF tech teams arbiters of technical quality rather than sources of disruption. This might be by funding (and providing infrastructure for) commercial or academic teams to build, debug, test and evaluate skins (and similar) which are then promoted to wikis by WMF based on quality.
2. Consider sponsoring academic research and a theme or track at a usability conference or journal on wikipedia usability (reading and editing; across language and culture textual practices; design for avoiding pile-ons; etc).
3. Consider sponsoring science communication in fields relevant to the wikipedia project: web UI; information systems; multilingual web usability; readability; etc.; etc. By promoting awareness of the academic consensuses in these fields there is hope that we steer discussion along evidence-based lines rather than “I don’t like X, don’t do it”
4. Consider sponsoring the creation and maintenance of wikibooks on each of the technologies wikipedia relies on, prioritising those usable by non-privileged wikimedians within the project (javascript, css, SQL, etc). Boosting access to such resources and aligning the versions and examples with the broader project would promote these skills across the project and enable motivated volunteers to engage with these technologies much more easily.
5. Consider using the volunteers who were actively involved in discussions related to one update as candidates for notification / testing of related updates. My participation in discussions related to Vector 2010 apparently didn’t qualify me for notification about Vector 2022; it should probably have. 12 years may seem like a long time to the WMF, but non-trivial numbers of active en.wiki users have been editing since before WMF was founded, embodying significant institutional knowledge. [Service awards can be used to find veteran editors.]
6. Consider processes to rollout changes to only portions of a wiki at once for testing purposes.
7. Consider moving to rolling updates of significant features as is common in SaaS. A new mainline skin appearing every January on all wikis, being made the default in May and marked as deprecated 48 months later. A new alternative skin appearing alongside it, with more innovative changes and more radical changes to the visual aesthetic might be deprecated earlier, with successful features appearing in a future mainline.
8. Consider publishing explicit design criteria for future wikimedia skins (and similar) built / commissioned by the WMF. 
9. Consider ‘introspection into the wikimedia system’ to be a design criteria for future wikimedia skins built / commissioned by the WMF. ‘Introspection into the wikimedia system’ in this context means enabling and encouraging users to reflect on the wikimedia install before them and might include: consistent visual differentiation between UI elements created by wikimedia core functionality, installed gadgets and /wiki/User:<user>/common.js; links from preference options to the respective tags in phabricator; etc.
10. Consider publishing formal technical evaluations of skins, to provide evidence and motivate change and progress. If editors can see that one skin fails on 25% of browsers used globally and one fails on 1% of browsers used globally, that's hard evidence the the second fulfills the WMF's mission better than the other. 

Concrete ideas for en.wiki

1. Consider better ways of handling contentious issues which don’t result in pile-ons and bordering-on-unclosable RFCs.
2. Consider a policy requiring complaints of specific technical issues in WMF infrastructure (broadly construed, but including skins) to be required to include a link to a relevant phabricator ticket (or a statement of why one can’t be created) if instructions for doing so are already on the page. Driving people who complain about WMF tech stuff to phabricator to create a bug report should be obvious, but it is apparently not.