Oxford e-democracy evaluation
From E-Democracy.org
Update - November 2007
In November 2007, Giles Moss the lead research assistant from Oxford, sent us the following communication, "Since that time, I have been following the Brighton & Hove and Newham Issues Forums closely, and think they remain one of the most successful examples of how local eDemocracy can work effectively in practice."
Related documents of note that feature Issues Forums, include the e-Democracy in Bristol report, among a group of e-participation case studies from Teledemocracy.Org (see page 62 in PDF), the Issues Forum section of the ICELE evaluation, and Issues Forum case study we wrote for the UK project.
Version 1.2 - 28 Oct 2005 - Steven Clift
Issues Forums received the honour of being chosen for independent evaluation by the Oxford Internet Institute. These evaluations were commissioned by the UK Local e-Democracy National Project. We encourage you to read their findings as we use them to help us improve our citizen-based model for local e-democracy.
We want to thank the Oxford Internet Institute, Prof. Stephen Coleman and Giles Moss in particular, for selecting Issues Forums from so many different pilot projects for closer evaluation. After over a decade of direct e-democracy practice, we note the lack of research that challenges or guides the real implementation of on the ground e-democracy initiatives that aspire to enhance democracy. Most research deals with "politics as usual" and their online attempts to help a candidate or political party win an election. The Oxford evaluation challenges other researchers to not just study "what is" online, but to also create knowledge that help those working for "what could be."
As you review the evaluation, we do want to clarify a some items before you read their full evaluation:
- 2 for 1, not 2 for 4 - E-Democracy.Org was required to launch one pilot local Issues Forum in the UK. We launched two forums - Brighton & Hove and Newham. Citizens in additional communities expressed interest, but were not ready to launch during the pilot phase. This process was not made clear to the evaluators. Allowing communities to start in a participatory process that might not result in an open forum in the end should not be viewed a failure to open a required number of forums. Why some communities are more ripe for Issues Forum is an important question to explore.
- Interview-based evaluation - The Oxford evaluation was based on interviews with steering committee leaders and councillor participants. Some of the ideals and goals articulated went beyond the baseline Issues Forum model as imported (for example, we require real names to build social trust because anonymity in local political discourse is mostly negative and drives away the public) from Minnesota. The interviews did bring out new ideas and goals that must be considered in future Issues Forum expansions. The evaluation did not evaluate some of the deliverables set out in our Project Initiation Document including the print training and multimedia materials which covered a third of the budget. The evaluation did mention the important open source technology advancement with GroupServer which we commend. A future evaluation exploring the cost-effectiveness of the model would provide real value. Are Issues Forums really as low cost as we suggest compared to other ideas?
- Social Inclusion - The most important challenge laid out by the Oxford evaluation is around social inclusion. While we pragmatically reach out to active citizens and community leaders in order to place the forum in the centre of real power - the "egg" - what about the "chicken" where the less engaged citizens act and join as well? How should we reach out to those less likely to participate? What resources will be required to do so successfully? We call this New Voices outreach. Our own case study notes that "The one short coming of our development process that must be shored up is the creation of a temporary recruitment position to assist with the launch of a local forum ... A key assignment of this person would be to lead outreach to diverse communities in order to launch a forum with a broad range of voices in the community." It is important to remember that the Internet is about choice - people choose what to click, what to read. Outreach to citizens that are not initially interested in local public issues or who don't see any kind local democratic experience as relevant to their lives, must be done carefully on a significant scale to bring results.
- Citizen-based by design - Unlike almost all of the other National project pilot projects, this effort was citizen-based and not led or controlled by the local Councils in Brighton & Hove or Newham. While being citizen-led was one reason we were asked to share our model, this difference (what we see as a strength) has led to some confusion and useful creative tension. (See the end of the guidebook for a full discussion of the potential role of Councils.)
- Training - Training materials built on the 60 page Issues Forum guidebook and multimedia E-Democracy Experience included many one-on-one meetings and telephone conversations were provided to Forum Managers and local steering committee members. We consider this type of training to be more effective than "formal" classroom style training.
- Legal ownership and liability - There is no question about who in the end "owns" these forums. They are part of E-Democracy.Org and governed by the local steering committees. This was established in our contract with the National Project. Further our universal rules make it clear that participants are liable for their own posts and we do not pre-moderate posts in order to limit our organisational liability as a potential "publisher." Ultimately E-Democracy.Org is responsible for the direction of these forums and for the support of the local steering committee/chapter governance structure now and after the national project closes. Should someone file a liability lawsuit against E-Democracy.Org in the UK for forum postings, this would present a challenge to our low cost model and potentially threaten the existence of a forum.
- English language - None of our Issues Forums have an official language. The topic was discussed at a Newham steering committee meeting, but no action was taken nor were statements made to forum participants or posted in the forum charters.
- Facilitated not Moderated - The evaluation suggests that our Issues Forums are "moderated." To some, facilitated or moderated may seem like different shades of grey. To us it is black and white. Not technically, editorially, or politically moderating posts before they are distributed is central to our facilitation model. We limit our power as hosts by not moderating and therefore avoiding "technical" opportunities to censor. Because members may be sanctioned for violating our rules, we give them the responsibility to be civil and treat people like adults. Participants, therefore have the power to engage in virtual acts of civil disobedience (and risk warnings, even temporary suspensions) in ways that all participants can see.
- Decision-making versus Agenda-setting - We suggest that how you evaluate a government-sponsored and controlled online consultation and its influence on decision-making should be different from a citizen-to-citizen Issues Forum where a goal is to influence public agenda-setting by fostering the creation of new public opinion through many-to-many online public spaces. Influencing the formal Council democratic process is important, but that has not been a core part of the Issues Forum model to date. Our discussions are more likely to directly influence individual elected officials and public leaders informally than to result in some quantifiable influence on a formal decision-making structure. With greater investment and professionalized (versus an all local volunteer) model, Issues Forum might become more like online consultations. However, we've focused on keeping the model low cost and oriented to agenda-setting. On that point, we are not specifically aware of local UK discussions that have led to local media coverage of issues. This "agenda-setting" leap happens frequently on our Minneapolis and St. Paul forums and it should be fostered in the UK Issues Forum communities.
- Numbers - The evaluation contains results from a March 2005 survey. These are useful numbers. In future evaluation we might recommend a full comparative survey of all our forum members (UK and US) building on the Oxford and these previously asked survey questions. Other chapters of the evaluation contain useful numbers on participants and postings in other pilots. While not a clarification of the Oxford evaluation, here are some useful numbers we compiled on July 5:
- As of July 5, 2005, the 184 "direct" members of the Brigton & Hove Issues Forum have produced 979 posts on over 100 topics (192 different subject lines). Via our web interface over 11,000 posts/topics have been viewed to date (including unregistered readers). In the week stretch from Jun 19-25, 29 different people, including three of the six posting councillors, shared messages on seven topics. (As of Oct. 12, there are now 202 members with 1215 posts total. Posting volume in Brighton slowed in August and is now regaining steam.)
- In Newham, with 109 members, postings volume has grown more slowly with 192 post on fewer than 50 topics. (As of Oct. 12, posting volume has grown to 486 posts on about 100 topics with 126 members.)
You may access the e-Democracy from the Ground Up evaluation from the UK Local e-Democracy National project web site.
This page was last modified on 3 December 2007, at 19:50. This page has been accessed 4,881 times. [1 watching user]