Developing Taarifa; London Water Hackathon 2011


On Friday 21st of October I attended my first hackathon. For those unfamiliar with hackathons they’re events with the idea of sticking a load of intelligent and driven computer hackers and programmers in a room for a period of time and see what drops out. The format of the London Hackathon was that for 48 hours at UCL groups would produce technology demonstrators and designs to solve global water problems.

The execution follows an introduction by the organisers framing the exercise with problem statements and presentations by experts wanting to solve different problems, people skyped in or did them in person. The problems presented all came form the Random Hacks of Kindness website and ranged from a water trading platform to my own issue of public service infrastructure and community mapping – my PhD.

Once the ideas where presented it was up to the developers to choose their projects. I had some awesome and – as my friend Josh Goldstein would say – ‘pumped’ developers with skills ranging from PHP to Python. The idea is essentially ‘Fix My Street’/Open311 for less developed countries. From this I started to hoover up whiteboards like they were going out of fashion. On them trying to get a design spec on workflow through the potential system, how triaging of reports would work. We were left with some outstanding design to be done, but nothing too complicated; or so we thought at the time. The idea was to create this system so it can be used as a technology demonstrator to the Tanzanian Commission of Science and Technology (COSTECH).

When I was last in Tanzania, COSTECH was being bounced around by the IT community. In Nairobi, the iHub acts as a lightning rod for technology and IT supporting local and foreign developers. The iHub model is excellent and really drives technological innovation in East Africa, other initiatives include the Hive Colab in Kampala and Bantalabs in Senegal. The people that run the iHub also run Ushahidi. Ushahidi is a platform/CMS designed for the crowdsourced reporting of issues, its inception was due to the Kenyan election crisis of 2008 and since has been used to report on Flamingos to the recent ‘occupy’ movements. Under the bonnet it’s a PHP website developed with the Kohana framework. It was here where we hit our first issue; none of our developers had worked with Kohana before. We split into two groups, one figuring out Kohana, the other designing workflow.

The first issue faced by the newly developed team was developing environment and server access. While most of the developers (myself included) were familiar with the programming languages needed, it wasn’t our primary choice. Personally I haven’t hacked about with PHP since October – December 2005, for others the experience seemed the same. Installing Ushahidi also proved problematic. Issues with mod_rewrite and other PHP extensions were experienced, but were eventually fixed. This wasn’t a problem which Ushahidi per se, though due to some very strange address rewriting configuring mod_rewrite was necessary for all. As we didn’t have server access we remote hosted, using one of the developers personal servers.

Once the workflow was sketched out we presented back to the developers, who had been exploring the Ushahidi plugin ecosystem. We integrated ‘actionable’, ‘simple groups’. Actionable was used to ‘action’ reports, and place them in a triage system.  Simple groups was used to simulate a ‘team’ of fixers. Fixers was used generically as the people fixing the reported problems, however the dynamic of how this would be worked wasn’t considered at this stage. Currently in Tanzania, in Dar Es Salaam and Zanzibar a number of reporting/monitoring projects are in the pipeline. The next steps are to interface with local government, private companies and citizens to develop public services.

We started to have a good interface, with tasks and problems being received and triaged. The workflow started to come together with reports being verified, on verification being triaged, going to the imaginary team of fixers and finally reaching conclusion or dispute resolution. The tabs to accommodate these were integrated into Ushahidi.

Now reports were able to be triaged we focused on expanding the reporting mechanism. Out of the box Ushahidi supports reporting through a web-based form, twitter and through its mobile applications (iOS, Android, Java and Windows Mobile 6). It can interface with SMS gateways like FrontlineSMS, we wanted to use SMS due to the ubiquitous nature of feature phones in Africa that realistically can only use SMS as a form of reporting. Using SMS presented problems of geo-locating the messages. Theoretically it’s possible for the telecos to triangulate the position of the sender and supply a latitude and longitude but this isn’t practical over a 48 hour hackathon notwithstanding the ethical and privacy concerns.

The solution we came up with, kudos to Dirk Goriseen, was to create a 100m2 grid and have a 10 digit reference for each grid square that people could text in. This would be prefixed by a hash (#) then found through a regular expression in the submitted message. Obviously questions remain when implementing this on a large scale namely ensuring local people know what their code is and creating a reference system that conforms to the human geography not just the physical. Integrating the SMS into the system was more problematic. We found that FrontlineSMS on the face of it doesn’t work in the cloud – if that is wrong please correct me! So Caroline Glassberg-Powell (Caz, our PHP guru) phoned a friend of hers, Daniel Fligg to hack an SMS gateway together.

Now it’s about 2100. We’ve been working for roughly nine hours and out project and code is starting to take shape. Teutonic reinforcements then arrived, all of them with Android mobile phones, none of them came with any Android development experience but did come with a willingness to learn about developing the Android application. This process entailed downloading Eclipse, setting up the Android SDK etc. It was a process that I feel should have been simpler, some issues with devices not being recognised, Eclipse’s idiosyncrasies (like interfacing with GitHub) were encountered.

About two hours later, once an Android development environment had been setup on each of the laptops we forked the Ushahidi Android app on GitHub and started development. While the stock app is very good we wanted to add the functionality of being notified when your report changed its status. To do this, avoiding login and passwords we wanted to use a unique identifier. Android, within its SDK supports a unique identifier and has specific calls for it. We later figured out that it’s not implemented across all versions and devices, with some choosing to not make that part of the SDK available. Quite why this is the case, the rationale doesn’t seem clear and from a design aspect it seems quite stupid, but the situation is as it is so we hack!

Our solution was to create a 256bit random hash assigned when the user opens the application for the first time. This would then be included in the reports to identify which reports come from which device (with the potentially flawed logic of one person uses one device/one device is used by one person). So instead of seeing all the reports in the system you just see the progress of reports that you have submitted. Unfortunately this process went on into the early hours, at which point sleep was necessary.

None of us wanted to use the night buses to go home so we slept in the working space. We fashioned little cubicles from the whiteboards, with chairs providing a bed. The sleep came easy, but when the three that remained woke we were all quite cold, in need of some more sleep and shower. However that doesn’t detract from more hacking!

On waking up the majority of the web-based system was already completed. During the night the SMS gateway had been magically integrated during the night. What remained was obvious bug-fixing, tidying up code and starting documentation.

The Android app had started to encounter real difficulties,  it wasn’t compiling, even when reverting back to code which was known to be ‘fine’. This process continued from 0930 – 1400. The issues with Android turned out to be problems within the Eclipse development environment. The code compiled and ran – with our adjustments – for the first time during the presentation. It was good timing.

We presented the slides above to the judging committee. We won. The credit to this victory goes to the people involved in the project, because of the numbers I guess we had covered the most amount of ground. However this isn’t trying to detract from the Herculean efforts of the programming team and other participants in the hackathon. Participation in our project wasn’t strict, people flitted in and out on the basis of their skills, in all about 4/5 people could be considered to be core/’never say die, I’m here till the end’ members. But the entire project had around 10-12 people that scribbled on the board, gave advice or rolled up their sleeves and hacked a little. This blog is dedicated to all of them.

Photos from the event can be found on Flickr here: http://www.flickr.com/photos/markiliffe/sets/72157628083354348/

Written and submitted from the Nottingham Geospatial Building (52.953, -1.18405)

Advertisements

Author: Mark Iliffe

Traveller, Programmer, Geospatialist and Motorcyclist

6 thoughts on “Developing Taarifa; London Water Hackathon 2011”

  1. For a 48 hour exercise, you guys managed to get a lot done. But I have real issues with the model of Taarifa, particularly for a Tanzanian context.

    UN Habitat and Google.org (with others) created a “Human Sensor Web” in Zanzibar trying something very similar, and nobody sent any messages (apart from a few people who were paid to do so), and nobody at the relevant authorities ever used the information made available to them through the system. We at Daraja have tried something similar with our Maji Matone programme, and though we have a good record at turning messages received into problems solved, we didn’t receive that many messages (despite serious spending on promotion and marketing).

    I guess what I’m trying to say is that the technology side of projects like this is the easy bit. You can pretty much use off-the-shelf software like Ushahidi and FrontlineSMS to do that, without needing to create anything new. So while it may be impressive that a system like Taarifa can be put together in a very short time, it seems to me to miss the point. Getting people to engage with systems like this is the real challenge.

    1. What we created was the workflow of the reports so they could be triaged and sent to the relevant team of ‘fixers’. We did use off the shelf software like Ushahidi, just hacking it about to add a few more screens.

      Within the Tanzanian context, in collaboration with other actors we’re hoping this can be used as a technology demonstrator to show what is possible. Thanks for pointing out the ‘human sensor web’ links, they look pretty interesting. Out of question where did you launch Maji Matone and how did you advertise it?

      1. I see what you’re saying about adding an extra layer – the triage system – which is potentially useful.

        But I’m concerned if you see the human sensor webs as promising. Even the project team see that as a failure. Without major efforts either in promotion or finding ways of turning information into action, their elegant software was useless.

        With Maji Matone, we’ve done slightly better. We struggled to get people to engage, but we did very well at ensuring district water departments responded to the information received. For our six-month pilot in three districts – Njombe, Mbozi and Morogoro Rural – we had weekly radio programmes and daily adverts on local stations in all three districts and we distributed around 10,000 leaflets, posters and flyers per district.

        You can get more details from http://blog.daraja.org/search/label/Maji%20Matone

  2. Thanks for the information. I regard it as being useful as regardless of its success or failure; It provides lessons learned for future projects and developments framing future design considerations. Hence my later post around the ergonomics and sustainability of such projects. There is also an opportunity with such projects to not just engage community members but also local government, CBOs and NGOs.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s