Civic Tech Monthly, March 2015

Welcome to the second edition of Civic Tech Monthly, our selection of what’s new and interesting in the last month of Civic Tech in Australia and around the world.

Don’t forget to share this newsletter with friends and colleagues. Forward this email to them or invite them to subscribe at http://oaf.us8.list-manage.com/subscribe?u=86c7f63554037aed044df4556&id=083b3ae43e

News & Notes

Update on the fate of the Office of Australian Information Commissioner

Peter Timmins has published an update on whether or not the OAIC, the independent Federal office overseeing Freedom of Information, Privacy and information policy, will be abolished. This follows discussions at Senate Estimates in late February.

New inequality data by location on KnowYourPlace

Rosie Williams reflects on World Open Data Day and announces the new inequality data on KnowYourPlace in the always interesting Croaky Heath Blog. Rosie spoke about working on this project in our interview last month.

Quickly find video of speeches in Parliament

Some exciting news about the Federal Parliamentary website. You can now quickly find the video footage for speeches in Federal Parliament. Hansard now has a link to ‘Watch ParlView Video’ next to each transcript. It appears that only speeches since the start of 2014 have this feature so far.

Wouldn’t it be wonderful to have this on OpenAustralia.org.au as well?

A local They Vote For You for Redlands, QLD

Residents of Redlands, Queensland are pushing for greater transparency in local council voting. Redlands 2030, a community group publishing civic information and news for Redland locals, present a case for open local democracy and explain their council’s current position. They collated all the votes of the current councillors and are now looking for help to publish them in a more accessible form.

Help open the NSW register of MP interests

Nick Evershed and The Guardian’s datablog are enticing their NSW readers to help investigate the state’s register of pecuniary interests. They’re using technology to crowdsource transcriptions , changing the form in which this data is available to the public, to one which is open to search, scrutiny and analysis. Neither parliament nor politicians have previously shown any interest in making this data accessible and usable.

This is another exciting development following January’s #nswvotes meetup!

This nifty combination of citizens, journalists and technology fundamentally changes the power balance. It’s a blend of political story, instruction, motivation and invitation to act.

Sobanukirwa launched in Rwanda

Rwandans can now request government information through Sobanukirwa. This is the 19th international deployment of Alaveteli, the software also running Australia’s Right To Know. There’s one partially successfully request so far, for the “Health care location details for facilities in Rwanda”.

UK Parliament Digital Democracy report

The UK Parliament’s Digital Democracy Commission released Open up!, a report into how “Parliament could use digital technology to work more effectively and in a way that people expect in the modern world”. The presentation of the report is a device agnostic website with lots of video and illustration. Check out the opening Key targets and recommendations.

Changes to our PlanningAlerts API

PlanningAlerts is introducing API keys for all users of the PlanningAlerts API. From the 1st of June 2015 it will become mandatory to use an API key. Never fear, it’s easy to get started. The blog post contains the instructions you need.

Scrape javascript heavy sites using morph.io

Matthew from the OpenAustralia Foundation came up against some tricky anti-scraping technology this month (which is additionally a big accessibility problem). To overcome these blockers he added PhantomJS support to Morph.io. Now everyone can scrape javascript heavy sites, and get more structured data from the web. Go forth and scrape!

The Poplus Component Integration toolkit

Associazione Openpolis (Italy) have launched an early version of Poplus Component Integration Toolkit to help python developers mash up data from different Poplus components. The project received a Poplus Grant in 2014.

Posted in Civic Tech Monthly | Tagged , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , | Leave a comment

Scraping javascript sites with morph.io

Just a quick post to let you know that it’s now possible to scrape javascript heavy sites easily using our scraping platform morph.io.

This is really useful with Microsoft .NET web sites that often use complicated states stored in javascript with links simulated via javascript posts.

Also, we recently discovered another more worrying example. The main website of the NSW Electoral commission, who oversee state elections in NSW, is “protected” by some anti-scraping technology that stops you from being able to download the contents of a web page without javascript. This is clearly terrible for accessibility and in our case for getting access to basic electoral information which is not available by any other means than scraping.

Thankfully…

phantomjs

PhantomJS is now installed for everyone using the experimental buildpack support.

PhantomJS is essentially a headless browser that you can control from your scraper using javascript or alternatively via wrapper libraries available for most major languages.

If you want to use PhantomJS but are not yet using the buildpack support, use this as a little bit of extra incentive to move over to it. All you need to do is ask us to enable it for you (letting us know which user or organisation you would like it for)

Posted in Announcement, Morph | Leave a comment

PlanningAlerts API changes

Starting today, we’re beginning to roll out a change to the way the PlanningAlerts API works. We’re introducing API keys for all users of the API from low-volume non-commercial users to high-volume commercial users. We are making this change so that we can better measure and understand the way people are using the API as well as give us the ability to contact all API users in case of any issues (or changes we make) that might affect them.

We’re phasing in the introduction of API keys. Right now they are optional, though as a new user of the API you will be guided into using them.

On 1st of June 2015 the use of API keys will become mandatory

3 months from now on 1st of June 2015 the use of API keys will become mandatory.

This means if you are a user of the API today, you will need to move over to using API keys before 1st of June 2015. If you don’t do it by then the API will stop working for you on that date.

We’ve made it very easy for you

Thankfully, we’ve made it very easy for you to get an API key.

  1. Simply register for an account on PlanningAlerts
  2. Then visit the normal API documentation page to see your API key
  3. Update your API calls adding &key=[your key]

We recommend you don’t use the PlanningAlerts api via client-side javascript

If you use the API in client-side javascript you will obviously be sharing your api key with the world which is a very bad thing. It is your responsibility to keep the api key safe and secure. If we discover any abuse of a key (e.g. someone else is using your key) we will cancel the key and you will have to get another one.

As a result we recommend you don’t use the PlanningAlerts api via client-side javascript.

Either

  • Switch to getting the planningalerts data server-side where you won’t expose the api key
  • Or proxy the client-side requests through your own server where you add the api key

If you have any questions on how to do this, of course don’t hesitate to get in touch with us.

Posted in Announcement, PlanningAlerts.org.au | Tagged , , , , , , , | Leave a comment

Civic Tech Monthly, February 2015

Sailing image by Steve Bennett, used under Creative Commons. See flickr.com/photos/63439615@N00/8000733006

Photo by Steve Bennett, used under Creative Commons

Welcome to the first edition of Civic Tech Monthly breaks champagne bottle.

Today we set sail around the world of civic tech, its peoples and their projects. Civic tech at the moment can mean ‘tech that’s all about citizens exerting and obtaining power’, as well as ‘tech that’s all about improving government services’. We’re following Tom Steinberg’s lead and including them both. We can all explore what we mean here together.

This voyage is an experiment, we’re in uncharted waters, so please jump aboard. Let us know if the links are interesting and what you think about them (better yet let everyone know on twitter). Don’t hold back if you think we’re sailing off course, if you want more—or less links, etc. .

You can also contribute to the newsletter on GitHub, like all our projects.

If you think this is interesting you’re sure to have friends who would too. Please pass on the subscribe link and make noise via the social media tubes.

For those who are able to make it, we’ll see you at the OpenAustralia Foundation Sydney Pub Meet next Tuesday.

Without further ado:

News & Notes

3 ways to keep They Vote For You tickety boo

For the past 18 months Micaela Ash has done an amazing job building up the division summaries and policies on our latest project They Vote For You—it is a vast new resource of how our laws are made. Micaela has now moved on to new, exciting work, so she left us some parting instructions on how we can keep the resource in shape. Surely there are people on this list who could take a few minutes to summarise a division once every few weeks?

Percs: searchable pecuniary interest disclosures

Following January’s #nswvotes meetup, Chris Nilsson whipped up a way to search the text of the previous very inaccessible 2013-2014 NSW Legislative Assembly Pecuniary Interest Disclosures. A huge step forward in the accessibility of this information, and there is more in the works—see the OAF Meetup’s discussion board.

Unlocking Australia’s public data: A catch up with Rosie Williams

An interview with Rosie Williams about the latest work on InfoAus, a site of ‘political and social research tools’ that make Australian government data more accessible. Rosie is currently adding new ways to access data that shows the differences in wealth and poverty across Australia.

Cuttlefish is now an official Poplus component

Cuttlefish is an easy-to-set-up transactional email server developed by OpenAustralia Foundation Founder Matthew Landauer. Last year the OpenAustralia Foundation was awarded a Poplus mini grant to create a hosted version of Cuttlefish which is freely available to the Poplus and international civic tech communities. Read Matthew’s post, Email is your secret weapon, on the Poplus blog to find out more. Now Cuttlefish has been accepted as an official component, more people will find out about and use it.

Poplus is a global federation for civic tech, developing components that can support projects everywhere. Join the Poplus maillist for updates and discussion.

Federal Government announces the Digital Transformation Office

The Government is establishing a “small team of developers, designers, researchers and content specialists working across government … to make services simpler, clearer and faster for Australian families and businesses.” It is an important project for the civic tech community to get involved with and help make it as effective as possible. You can read more about the DTO at the Communication Minister’s FAQ. The UK’s Government Digital Services have also written a blog post about the project.

Growstuff

Growstuff is ‘a community of food gardeners who are building an open source platform to help you learn about growing food, track what you plant and harvest, and swap seeds and produce with other gardeners near you.’

We love this one because it’s a practical open source project helping people grow food and community. It helps keep information held in plants and seeds open by making it easy to share. Growstuff releases the data it collects under open licenses, giving power to it’s developers as well as growing a community of gardeners. It promotes pair programming as the norm.

This is a very active project. Volunteer to help test, write for the Growstuff blog, contribute code to the application or just dig in, grow some food and share your seeds and data.

Nick Evershed injects some context into the news

After Tony Abbott apologised for saying Labor created a “holocaust” in defence industry jobs, Nick Evershed used theOpenAustralia API to provide more context about the use of the term in Parliament.

Hackfarming Blood Buddies

Jeremy Keith takes us through the process of planning, building and deploying a hack to help more people donate blood at Clearleft’s Hackfarm getaway this year.

Their 2012 Hackfarm resulted in a prototype for a personal political opinion tracker—a few people at the OpenAustralia Foundation meetup have suggested similar concepts for projects they want to work on.

Local events for International Open Data Day

This Saturday, 21st Feb, is International Open Data Day. The Open Knowledge Foundation Australia have a list of events for you. Code for Australia’s day of workshops in Sydney tomorrow caught our eye. It is a free event and registrations are still open.

Learning to contribute with Git

Contributing through the Git version control system is an important skill for people working on civic tech and open source generally—but it ain’t that easy to learn and it’s not always clear where to start. We’ve observed that it can be a barrier to people getting involved.

Once you’re rolling, it really is easier than you’d think. You’ll quickly see why it’s so popular. If you learn well with audio, try the Web Ahead Podcast’s Git episode. If video, interactive tutorials or reading works better, try one of the many useful tutorials on Github’s resources list. Once you’ve learned it you’ll have superpowers, so don’t be afraid to ask if you need help.

12 steps toward a better neighbourhood

Henare and Luke visited the Friends of Erskineville community group in Sydney last week. We introduced the group toPlanningAlerts and had a great time finding out about how they’re working to improve their area. In return we picked up this interesting list of 12 steps that they were discussing. These are practical ideas for getting engaged in your local community. Maybe these steps will spark some ideas for your next civic tech project?

Civic Hall launched in NYC

Civic Hall is an enormous space now allocated for civic tech hacking and to house civic tech organisations. If anyone is in NYC, drop in and let us know what it’s like!

Posted in Civic Tech Monthly | Tagged , , , , , , , , , , , , , , , | 1 Response

Unlocking Australia’s public data: A catch up with Rosie Williams

Photo of Rosie Williams

Rosie Williams has independently made a huge contributions to budget transparency and the accessibility of public data in Australia through her project InfoAus.

It’s been a while since we were able to catch up at the last pub meet so I sent through a few questions.

 

What are you working on at the moment? and what was the path to this point?

I am re-developing my open data project KnowYourPlace to include data visualisations of ABS SEIFA data. The idea is to provide this information on social and economic inequality to the public in ways we can interact with and make use of. KnowYourPlace is the part of my website (InfoAus) that displays political information based on geographical and political boundaries. 

I implemented the original version of this data over the Australia Day long weekend in 2013. This redevelopment is aimed at giving meaning to the data- providing visual context to the data so that users can look at the pages and see what the data means for them.

Another aspect of my work is to push the boundaries on data provided for re-use by the government which forms the basis of open data projects like those on InfoAus. For example, after speaking with the ABS last week about the information needed to filter data by federal electorate, the ABS agreed to publish electorate concordances along with locality information. Previously, federal electorate was only published with postcode information rather than suburb-level information. This is a great step forward.

With the projects I work on it is often the first time data has been presented in ways that allow grouping by political boundaries so it is only with such use-cases that the government can get the feedback that it needs to provide data in ways that provide meaningful analysis for web applications.

Why should people be interested in the ABS’s Socio-Economic Indicators For Areas data? Has that data shown you anything surprising?

The most interesting and sad thing about the SEIFA data is that the lowest percentile of socio economic localities all appear to be Indigenous communities. I got a bit of a shock when I first mapped the top and bottom percentile of advantage and saw Australia’s racial inequality so starkly illustrated.

How have you found working independently on projects?

Working independently is difficult in that I can often go for long time periods without any feedback on what I’m doing. However in the past couple of weeks I’ve really enjoyed creating original solutions to data visualisations that are entirely my own code. Being responsible for both the front and back end of my projects means I have been able to provide custom solutions for the entire project, designing visualisations that provide data in ways that have meaning and that people can interact with. Being a sociologist with coding skills has enabled me to provide this custom solution.

For people wanting to contribute to your projects, or find out more, what is the best way for them to get involved?

InfoAus and it’s projects are the result of what is now years of hands-on work and the university education that preceded them. My passion regarding inequality in Australia began with my own experience of disadvantage and through a degree in Sociology, programming and the years of work and activism that has gone into building the projects I have made real progress in improving political and budget transparency in Australia.

One of the problems facing open data projects is the difficulty in finding a way to support the people who work on them.

In order to continue working on budget/political transparency through my projects, I will have to begin earning money to support myself. I will look be looking at crowd funding and other options for the survival of my projects.

I’ve introduced Google ads to the site and I’ll be looking at doing iphone apps and marketing the databases to libraries. This will probably mean the databases will eventually be put behind a paywall.

What has impressed or interested you from the worlds of civic tech or open data recently?

The world of civic tech had a really important win last year that I became aware of in December. Grants.gov.au is a site being created by the federal government that- when available- will host information from all federal agencies on the community grants they provide. Currently, agencies publish this information in different ways, in different places with no consistency in terms of the information or format. A new specification has been developed for this grants data that requires agencies to provide the data in ways that allow it to be cross-referenced with the Portfolio Budget Statements released on budget night. This is consistent with the requests that I have made to the Department of Social Services regarding their grants funding data so I was very pleased to see it be implemented across the board.

The upshot of this is that grants funding (which will be published with geographical information) can then be matched with budget data and made searchable by geographical or political boundaries. The first implementation of this new specification made searchable by political and geographic input is part of my budget transparency project BudgetAus at http://infoaus.net/budget/social/index.php

Contacts that I work with within the government are also making in-roads in providing avenues for representatives of projects like mine to give feedback to for example, the Joint Committee of Public Accounts and Audit which sets out the specifications for agencies to provide information in their Annual Reports. Annual Reports data have implications for budget transparency so this is an important matter to pursue. Most of this current progress in budget transparency is being ignored by the media so it is good to be have this publication in which I can report it.

Posted in Uncategorized | Tagged , , , | Leave a comment

3 ways to keep They Vote For You tickety boo

Keeping They Vote For You up to date can seem a little challenging. So, I’ve prioritised the three most important jobs to make it easier for people to get involved.

If these three tasks are done regularly by enough people, the site will remain tickety boo (which is urban slang for ‘useful and current’).

1. Manage the policies

Firstly, mark any problematic policies you see on the site as draft policies. Problematic policies are, for example:

  • Biased (i.e. too ideological or a case of political point scoring)
  • Impractical (i.e. include too many aspects that should be broken into separate policies)
  • Redundant (i.e. have no relevant divisions)
  • Nasty (i.e. a swipe at a particular party or politician)
  • Silly (i.e. clearly a joke)

Note: if existing policies are turned into problematic policies by cheeky editing, reverse the change.

Secondly, combine any duplicate policies.

2. Edit at least one division

Try to do this weekly, particularly when Parliament is sitting (see the sitting calendar).

This will keep the site nice and relevant.

For a reminder on how to edit divisions, see our blog post.

3. Advertise the policies

We need to keep reminding each other what policies we have – there are a lot of them!

For example: when an article appears in the media that is relevant to a policy like For live animal export, share it on Facebook or tweet it on Twitter along with a link to that policy.

Happy editing!

 

Posted in Projects, They Vote For You | 1 Response

How to find what you’re looking for on Right to Know

Following on from some recent queries on Twitter, I thought it was a good idea to sit down and provide a guide on some of the ways you can keep in touch with a request on Right to Know.

Right to Know is a simple, free service provided by the OpenAustralia Foundation that takes the confusion out of asking the Federal Government for information. It enhances accountability by showing you all requests made using the site, and the responses from the Government.

There are a few ways you can find requests, some include:

  • Searching by authority
  • Searching by status
  • Searching by user

You can also “follow” almost anything on Right to Know, which allows you to receive a daily digest when something matches what you’ve followed. Most of the time people follow single (or multiple) requests, but what you may not know is that you can follow an authority, a user, a search term or even the entire site!

With Right to Know growing (we have over 570 authorities, and over 830 users who have made 880 requests so far!) I thought it would be good to describe some of the ways you can keep up to date with requests and topics on Right to Know.

Before you begin

Anyone can search for requests, but you’ll need to register to be able to make a request, comment on a request, or follow a request.

Registering is quick and easy, all we need is your name (you can use a pseudonym if you want) and email address. We believe in transparency, so we describe how we will use your information on our Privacy Page.

Searching and following requests by authority

If you’re interested in a particular authority (such as the Department of Immigration, or ASADA) you can see all requests made to that authority, and also follow it to be updated whenever a request is made or updated.

To do this, click View Authorities on the top. Here you see a list of all authorities, and ways to group them (on the left hand side). You can also search for authorities using the search bar just above the list:

0001-allauthorities-resized

 

When you find the authority you’re interested in, just click it and you’ll be taken to the page for that Authority (I’ve chosen the Immigration Department as an example):

0002 - dibpauthpage

From this page, you can:

  • Find information on what the authority does
  • Follow (for email) or subscribe to an RSS feed to receive updates on requests
  • Find out how many requests have been made in total
  • Browse a list of all requests, or look at requests which are successful, unsuccessful, or unresolved.
  • Search for requests made to that authority

Searching and following requests by keywords

Some people might be interested in a specific topic which could apply to multiple authorities (such as “Technology”, “Open Government Partnership” or even “Religion”).

Right to Know makes it easy to find exactly what you’re looking for, using the search bar at the top of the page, or the Advanced Search feature.

If you want to follow anything about, for example, Religion, you simply search for it:

0003-searchreligion

Then on the search page, simply click the Follow button on the right hand side:

0004-religion

That’s it! Right to Know will automatically email you once a day with any new requests or comments related to that search.

Searching and following people on Right to Know

Right to Know has some amazing contributors, and sometimes it can be great to see what some of these contributors say.

To follow a contributor, simply search for the user or click their name, and you’ll be taken to their page (I’ve chosen a new user to Right to Know, Vera Lystich):

vera

Simply click Follow, and that’s it! You receive an email with that person’s comments and requests every 24 hours.

 What if I want to stop following something?

While we only send you one email per day (we combine everything you follow into a helpful daily digest) we don’t want to send you emails about things you’re no longer interested in. That’s why we include a link to change your preferences. Simply click the link to be taken to a list of the things you follow, where you can unsubscribe with the click of a button.

More Information

You can find more information on how to search and follow requests on the MySociety blog – https://www.mysociety.org/2014/09/19/how-to-keep-up-with-hot-topics-in-your-local-area/ and https://www.mysociety.org/2014/09/29/subscribe-to-foi-requests-on-any-topic/

You can also get in touch with the team at Right to Know, either via email (contact@righttoknow.org.au) or on Twitter (https://twitter.com/RighttoKnowAu).

If you want to support Right to Know, please consider Volunteering or making a small donation to the project.

 

 

Posted in Presentation, RightToKnow.org.au | Tagged , , | 1 Response

Getting to know you

A mess of stickies, collecting and sorting observations we made while speaking with our supporters.

A mess of post-its: collecting and sorting observations made while speaking with our supporters.

Over the last couple of weeks we’ve been reviewing and trying to improve the way the OpenAustralia Foundation handles donations. While there are tens of thousands of people using our projects, over the past few years we’ve only received a very small amount of funding from small donations by individuals.

We know that many people are enthusiastic about our projects and so contribute their time and skills. Last year we launched a monthly meetup for people like this and it’s been a great success. There are also other people who aren’t able to contribute code or are busy. They can still support us with financial donations. We’d like to make this process a lot simpler and rewarding for those people, and of course increase this kind of donation.

Research aims and process

Reaching out and getting to know the individuals that have donated to us has been an one key goal of this mini-project. We’ve been follow the process for design research set out in Erika Hall’s excellent book, Just Enough Research:

  • Define the problem
  • Select the approach
  • Plan and prepare
  • Collect the data
  • Analyse the data
  • Report the results

Our goal, or problem statement is:

Describe the kinds of people who donate to the OpenAustralia Foundation and why they do it.

Because the number of number of people who had donated to the Foundation is quite small, we decided to contact them with a small survey and then conduct short semi-structured interviews with as many as possible. This would give us a chance to thank these awesome people, establish a connection with them and find out how they see the world. To quote Hall:

The goal of interviewing users is to learn about everything that might influence how the users might use what you’re creating.

Finally we would collect and analyse our data and then present the findings in a blog post (here it is).

Survey

We sent a very short Google Forms survey to the 160 people who have donated to the OpenAustralia Foundation.

The questions were:

  • Roughly, how old are you? (Under 20, 20s, 30s, 40s, 50s, 60s, 70s, Above 80)
  • What is your occupation?
  • Which OpenAustralia Foundation projects are you familiar with?
  • Do you regularly donate money to organisations or projects?
  • Why did you donate to the OpenAustralia Foundation?
  • Would you be happy to have a brief chat to help us learn more?

The idea was to try and keep the survey as short as possible so that people would fill it out and be happy to chat, whilst prompting some useful insights.

So far we’ve received 38 responses. Here’s a basic summary of the responses:

  • Over 90% of respondents were over 30 years old, with 75% being evenly distributed between 30s, 40s and 50s.
  • After categorising the occupation responses, IT/Tech was the most group, followed by retired and professional services. There were two smaller groups working in government and in civil society organisations.
  • 75% of respondents said they were familiar with PlanningAlerts, 65% OpenAustralia.org and 50% Election Leaflets. 45% were familiar with They Vote For You, 40% Right To Know and 10% morph.io.
  • 65% of people said they regularly donated money to organisations or projects.

While we are dealing with a small amount of data, this information is helping turn some of our assumptions into knowledge.

Twenty-one people were happy to be interviewed and so far we’ve spoken with eleven. It’s been a pleasure to learn about the people who use our projects. A few very generous people even made donations after speaking with us.

While the survey was a few basic questions, the interviews were a deep dive. We wanted to get to know these people and find out about their relationships with NGOs, their needs and concerns.

So what did we learn?

Passionate and knowledgeable in their area of focus

The people that we spoke to were passionately interested and knowledgeable in their chosen area of focus. For a number of them, that was local planning and they were either working professionally in the area or were members of resident action groups.

These people are using one of our projects as part of a wider project or campaign they were active in.

Here are some of our observations about who these donors are:

  • These donors are generally quite self directed. Many of them were self-employed, involved in community action groups, or had established independent projects of some kind.
  • A number of people were involved with local community organisations.
  • The people we spoke to who were using PlanningAlerts (most people), had acted upon information they had received in an alert by making a submission to council.
  • People who donated made little use, if any, of more than one OpenAustralia Foundation project. They were often aware of a number of them, but were only an active user of one project. In the survey, a quarter of people selected just one project that they were familiar with.
  • Most people we spoke to had discovered their first OpenAustralia Foundation project by being directly introduced to it, often by someone from the foundation, rather than finding it through Google or social media.
  • Everyone we spoke to had found the OpenAustralia Foundation through one of the projects, not the other way round.
  • Everyone we spoke to was computer and web savvy, a number of them having experience with programming or data management.

People donated because they use the projects and support OAF’s broad work towards better access to civic information

In the survey there were two very clear themes in the responses to the question “Why did you donate to the OpenAustralia Foundation?”

  • Because they support the values of the OpenAustralia Foundation
  • Because they get value from an OpenAustralia Foundation project and want it to continue

These two themes came through strongly in the interviews—in particular it was clear that these people were successfully using one of the projects.

“I believe in freedom of information, open and transparent government, open data”

In the survey the majority of people mentioned their support for the general mission of the OpenAustralia Foundation to create better access to civic information. Some examples:

  • “Because I believe in OpenAustralia and the ideals therein”
  • “I believe public, open access to parliamentary records is important, and that politicians need to be held to greater account. They Vote For You is a step in the right direction to holding politicians accountable.”
  • “Believe in freedom of information, open and transparent government, open data – that governments should be more transparent, should publish the information they produce – briefings, data, revenue and expenditure detail – as a matter of course, without the need for FOI requests from the public or journalists.”
  • “I believe in your open data/technology agenda whole heartedly. Democracy is a closed broken old boys club and needs to be re-designed and re-built.”

“Because it provides a service I use!”

Many people also said they donated because they were getting value from one of the projects and wanted to support it. Examples:

  • “Because of the free service offered and that donations are what keep [organisations] like this going.”
  • “As a small thank you for the emails (about my MP’s speeches in Parliament) that you make it possible for me to receive without having to read/search all of Hansard!”
  • “I use the PlanningAlerts service and I value it. I also highly regard OpenAustralia Foundation and the sense of transparency and freedom it brings into the society”
  • “Because it does provide a useful service, making me aware of up coming proposed planning applications in my immediate neighbourhood that I might otherwise have not been aware of. Due to this early notification I have had some success with council and even VCAT in influencing the resulting plans to maintain respectful scale and privacy within my immediate neighbourhood.”
  • “i use it”

People don’t need to know much about OAF to donate to it, but appreciate a clear explanation when they ask

The majority of people that we spoke to knew very little about the OpenAustralia Foundation. Those who did know about the Foundation had an accurate picture of our size and structure. No-one thought the OpenAustralia Foundation was a large, well funded institution.

A number of people who had donated also donated to other non-profit websites. Wikipedia, Getup, Mozilla and Black Box Voting were mentioned.

There were quite a few questions about how the OpenAustralia Foundation is funded. A majority of people mentioned that they were concerned by the amount of money NGOs put into marketing rather than their core mission. They appreciated it when they received a clear answer from the NGO they were questioning.

People like to receive updates but appreciate brevity

Donors were generally positive about receiving regular email updates from the organisations or campaigns they supported. There were strong bad reactions to being signed up to campaigns they did not support.

People who receive organisation update emails from liked concise messages that were quick to read and scan.

A few people mentioned that the email newsletters they liked were ones that gave them some actionable information: TV shows to note down to watch or facts about the issues they were interested in to discuss with friends.

What now

In designing the way we handle donations, we don’t want to just follow the standard path. Everything we do should be focused on making our projects more useful and effective to more citizens.

The insights gained through this initial research will provide a basis for more experiments to see how we can help new groups of people get use from and support our work. We’ve already got some on the boil.

Posted in OpenAustralia Foundation | Tagged , , | Leave a comment

Email is your secret weapon

(Cross posted from the Poplus.org blog)

Earlier this year I was lucky enough to attend PoplusCon in Santiago, Chile. It was a remarkable experience spending two very intense days working through problems, sharing experience and knowledge, and making connections with incredibly talented civic tech coders and activists from around the world.

If you were lucky enough to be there as well, I hope we met and I also hope you remember the lightning talk I gave on a tool I had been working on for some time. Hopefully it’s something that you’ll find useful.

Lightning talk at Poplus Con

The tool is called Cuttlefish.

Cuttlefish

I have a cool new announcement to make about Cuttlefish.

That comes a little later. First I want to tell you a little bit about the motivations behind Cuttlefish and of course what it actually does.

What is transactional email and why should I care?

Transactional email is email sent from applications in “transactions”. This could be for example, the email you get when you sign up for a new service; it could be the email alert that notifies you when a politician has spoken in parliament. In simple terms it’s anything that’s generated automatically by you doing something in an application.

Email is your secret weapon. Email is what reminds people your website exists and keeps them engaged. It’s what allows useful information to come direct to people.

As an example at the OpenAustralia Foundation we run a project called PlanningAlerts which is a really simple service that collects up applications from around Australia for people and companies wanting to build or knock down buildings. This is information that’s published on local government websites.

Now, who in their right mind goes and looks at their local government website once a week to see if there are any new planning applications that might be of interest? Maybe you would do it if it was your job but ordinary people are hardly going to do that.

What instead we do with PlanningAlerts is allow people to sign up for email alerts when there is a new planning application in their area. They don’t have to find or navigate their local government website and most importantly they don’t have to remember to regularly check.

When there’s something new it arrives in their inbox. At the moment around 50,000 people per week receive an email alert from PlanningAlerts in their inbox.

If you’re anything like us you probably send a lot of email too.

Here’s the problem – Sending email and doing it properly is actually surprisingly really hard and it gets even harder the more email you send.

Commercial route

One option is to use one of many commercially available transactional email services, such as Sendgrid, Mandrill or Amazon SES. They are great and incredibly reliable. They make sending email so much easier.

However, it does come at a financial cost. It can be inexpensive or even free if you’re not sending a lot of email. However, as a civic technology organisation you likely send a lot of email and you don’t have much money.

Then, there’s the tricky issue of working in civic tech fundamentally depending on a piece of closed-source technology. What if the service disappears or changes? What if I can’t afford to keep paying the bills?

It’s a practical limitation as well as an ethical issue.

Can open-source help?

The open-source route

Alternatively you could build the email smarts into your open-source application.

That is what has tended to happen so far in the open-source world. Every application that sends a lot of email has had to figure out for itself how to do bounce detection, link tracking and all those tricky things.

For example, Alavateli from mySociety has lots of smarts for figuring whether an email has been delivered (so that government agencies have a hard time claiming that they never received an freedom of information request when it can be shown that it was delivered to their email server).

CiviCRM can send emails to large lists of people. It has all this built-in smarts for link tracking, figuring out whether people have opened emails and detecting bounced emails. It’s not exactly easy to set up though and frankly is crazy buggy.

FixMyStreet and WriteToThem also by mySociety similarly have their own code for handling bounce detection.

The goal behind Cuttlefish is to create a platform and language agnostic open-source service for sending email so that every application doesn’t have to figure it out for itself.

Let’s solve the hard (and boring) problem once and share the winnings.

Sounds a bit like a Poplus component doesn’t it?

So here are some of things that Cuttlefish can do for you, your organisation and your civic application:

  • Send email from your application using smtp in the usual way and get all sorts of added benefits for no effort
  • A lovely web UI to browse what’s happening
  • Monitor in real time which emails arrive at their destination and which bounce
  • Works with any web framework and language
  • Automatically not send emails to destinations that have hard bounced in the past
  • Track which emails are opened and which links are clicked
  • Statistics on emails sent, soft/hard bounced and held back
  • View the full email content for recently sent emails
  • Multiple applications can each have their own SMTP authentication
  • Built in, super easy to set up, automatic DKIM signing
  • Postfix, which you know and trust, handles email delivery
  • Open source, so no vendor lock in.

Cuttlefish and Poplus

Fast forward to October, two months ago, the OpenAustralia Foundation was awarded one of the Poplus mini grants to create a hosted version of Cuttlefish which will be freely available to the Poplus community for all your email sending needs.

This involves several related parts

  • Allow you and your team to use a hosted Cuttlefish server as if it’s your own. You will be able to use a shared server without seeing any of the emails being sent by other teams because we’re adding multi-tenanting to the Cuttlefish software.
  • Making Cuttlefish server provisioning completely automatic using Ansible.
  • Providing free access to the Cuttlefish server to the Poplus community for a minimum of 1 year (until at least the end of 2015).

Also we will be applying shortly for Cuttlefish to become a Poplus component.

The big announcement

The work is done and you now have access to a hosted version of Cuttlefish

  • Completely free to members of the Poplus community
  • Send emails from your own domains
  • No restrictions on volume of emails
  • Hosted and maintained by the OpenAustralia Foundation
  • Available at https://cuttlefish.oaf.org.au now.

How can I get started?

The plan is to add a simple form to https://cuttlefish.oaf.org.au from which you can apply for access. We haven’t quite done that yet.

In the meantime, if you want access please email us at contact@oaf.org.au and include the following information:

  • Your name
  • Your email address — where you will receive the Cuttlefish invitation
  • Organisation name & website — If you’re not part of an official organisation, that’s fine too. Simply include something about the people you’re working with.
  • Describe the emails you would like to send with Cuttlefish and the applications they come from (e.g. email alerts for our parliamentary monitoring site …)

The obvious disclaimer here is don’t use Cuttlefish to send anything that could be mistaken as spam. No unsolicited emails. No mass marketing emails. Only things related to civic technology. If you have any doubts or questions just ask.

If you’d like to look at the code or would like to try installing it on your own server go tohttps://github.com/mlandauer/cuttlefish.

IMAGE CREDIT

Posted in Announcement | Leave a comment