The work and mumblings of Mark Iliffe


Community -> Hackathons -> Software Shipped -> Repeat?

Taarifa inception as hackathon project makes it special through it’s continued existence. From speaking to peers it seems that many hackathon projects are to create ideas not code. Fortunately we’ve stuck together, widened the community and now have members of the Taarifa community working on it for their dissertations, myself with my job within the World Bank deploying an instance of it to Uganda.

Within our own community we’ve got developers across the world working on little bug fixes and functionality. One member has taken on improving mobile UI upon himself and now the mobile web app looks brilliant on a mobile phone screen, decluttered from unnecessary components with things like inline labels ensuring things are ‘better’ on a phone. This is building upon a hackathon held at Shoreditch Design’s office where we shipped Taarifa 1.0.

With the forthcoming Uganda deployment (and potentially other applications) we have the opportunity to test and refine certain components. However building capacity in places where this is deployed is important. We hope by demonstrating Taarifa in developing innovation centres we can seed the ideas and principles of Taarifa (and by extension open source software) with them joining our growing community. This in turn would provide different perspectives on the requirements and ultimately filter down into software releases improving it.

The community is currently in the process of writing a Taarifa API consisting of entirely new code. We’ve just deployed a Q&A site on making it easier for users of Taarifa to ask questions and be a knowledge base for future users, avoiding the ‘goto github’ mentality which – in my opinion – should be for developers. Currently Nico and I are working on screencasts to give short introductions into installing and using the Taarifa system. Onwards!

Written and submitted from the Leicester, UK (52.6352614,-1.1378884)

Developing Taarifa; London Water Hackathon 2011

On Friday 21st of October I attended my first hackathon. For those unfamiliar with hackathons they’re events with the idea of sticking a load of intelligent and driven computer hackers and programmers in a room for a period of time and see what drops out. The format of the London Hackathon was that for 48 hours at UCL groups would produce technology demonstrators and designs to solve global water problems.

The execution follows an introduction by the organisers framing the exercise with problem statements and presentations by experts wanting to solve different problems, people skyped in or did them in person. The problems presented all came form the Random Hacks of Kindness website and ranged from a water trading platform to my own issue of public service infrastructure and community mapping – my PhD.

Once the ideas where presented it was up to the developers to choose their projects. I had some awesome and – as my friend Josh Goldstein would say – ‘pumped’ developers with skills ranging from PHP to Python. The idea is essentially ‘Fix My Street’/Open311 for less developed countries. From this I started to hoover up whiteboards like they were going out of fashion. On them trying to get a design spec on workflow through the potential system, how triaging of reports would work. We were left with some outstanding design to be done, but nothing too complicated; or so we thought at the time. The idea was to create this system so it can be used as a technology demonstrator to the Tanzanian Commission of Science and Technology (COSTECH).

When I was last in Tanzania, COSTECH was being bounced around by the IT community. In Nairobi, the iHub acts as a lightning rod for technology and IT supporting local and foreign developers. The iHub model is excellent and really drives technological innovation in East Africa, other initiatives include the Hive Colab in Kampala and Bantalabs in Senegal. The people that run the iHub also run Ushahidi. Ushahidi is a platform/CMS designed for the crowdsourced reporting of issues, its inception was due to the Kenyan election crisis of 2008 and since has been used to report on Flamingos to the recent ‘occupy’ movements. Under the bonnet it’s a PHP website developed with the Kohana framework. It was here where we hit our first issue; none of our developers had worked with Kohana before. We split into two groups, one figuring out Kohana, the other designing workflow.

The first issue faced by the newly developed team was developing environment and server access. While most of the developers (myself included) were familiar with the programming languages needed, it wasn’t our primary choice. Personally I haven’t hacked about with PHP since October – December 2005, for others the experience seemed the same. Installing Ushahidi also proved problematic. Issues with mod_rewrite and other PHP extensions were experienced, but were eventually fixed. This wasn’t a problem which Ushahidi per se, though due to some very strange address rewriting configuring mod_rewrite was necessary for all. As we didn’t have server access we remote hosted, using one of the developers personal servers.

Once the workflow was sketched out we presented back to the developers, who had been exploring the Ushahidi plugin ecosystem. We integrated ‘actionable’, ‘simple groups’. Actionable was used to ‘action’ reports, and place them in a triage system.  Simple groups was used to simulate a ‘team’ of fixers. Fixers was used generically as the people fixing the reported problems, however the dynamic of how this would be worked wasn’t considered at this stage. Currently in Tanzania, in Dar Es Salaam and Zanzibar a number of reporting/monitoring projects are in the pipeline. The next steps are to interface with local government, private companies and citizens to develop public services.

We started to have a good interface, with tasks and problems being received and triaged. The workflow started to come together with reports being verified, on verification being triaged, going to the imaginary team of fixers and finally reaching conclusion or dispute resolution. The tabs to accommodate these were integrated into Ushahidi.

Now reports were able to be triaged we focused on expanding the reporting mechanism. Out of the box Ushahidi supports reporting through a web-based form, twitter and through its mobile applications (iOS, Android, Java and Windows Mobile 6). It can interface with SMS gateways like FrontlineSMS, we wanted to use SMS due to the ubiquitous nature of feature phones in Africa that realistically can only use SMS as a form of reporting. Using SMS presented problems of geo-locating the messages. Theoretically it’s possible for the telecos to triangulate the position of the sender and supply a latitude and longitude but this isn’t practical over a 48 hour hackathon notwithstanding the ethical and privacy concerns.

The solution we came up with, kudos to Dirk Goriseen, was to create a 100m2 grid and have a 10 digit reference for each grid square that people could text in. This would be prefixed by a hash (#) then found through a regular expression in the submitted message. Obviously questions remain when implementing this on a large scale namely ensuring local people know what their code is and creating a reference system that conforms to the human geography not just the physical. Integrating the SMS into the system was more problematic. We found that FrontlineSMS on the face of it doesn’t work in the cloud – if that is wrong please correct me! So Caroline Glassberg-Powell (Caz, our PHP guru) phoned a friend of hers, Daniel Fligg to hack an SMS gateway together.

Now it’s about 2100. We’ve been working for roughly nine hours and out project and code is starting to take shape. Teutonic reinforcements then arrived, all of them with Android mobile phones, none of them came with any Android development experience but did come with a willingness to learn about developing the Android application. This process entailed downloading Eclipse, setting up the Android SDK etc. It was a process that I feel should have been simpler, some issues with devices not being recognised, Eclipse’s idiosyncrasies (like interfacing with GitHub) were encountered.

About two hours later, once an Android development environment had been setup on each of the laptops we forked the Ushahidi Android app on GitHub and started development. While the stock app is very good we wanted to add the functionality of being notified when your report changed its status. To do this, avoiding login and passwords we wanted to use a unique identifier. Android, within its SDK supports a unique identifier and has specific calls for it. We later figured out that it’s not implemented across all versions and devices, with some choosing to not make that part of the SDK available. Quite why this is the case, the rationale doesn’t seem clear and from a design aspect it seems quite stupid, but the situation is as it is so we hack!

Our solution was to create a 256bit random hash assigned when the user opens the application for the first time. This would then be included in the reports to identify which reports come from which device (with the potentially flawed logic of one person uses one device/one device is used by one person). So instead of seeing all the reports in the system you just see the progress of reports that you have submitted. Unfortunately this process went on into the early hours, at which point sleep was necessary.

None of us wanted to use the night buses to go home so we slept in the working space. We fashioned little cubicles from the whiteboards, with chairs providing a bed. The sleep came easy, but when the three that remained woke we were all quite cold, in need of some more sleep and shower. However that doesn’t detract from more hacking!

On waking up the majority of the web-based system was already completed. During the night the SMS gateway had been magically integrated during the night. What remained was obvious bug-fixing, tidying up code and starting documentation.

The Android app had started to encounter real difficulties,  it wasn’t compiling, even when reverting back to code which was known to be ‘fine’. This process continued from 0930 – 1400. The issues with Android turned out to be problems within the Eclipse development environment. The code compiled and ran – with our adjustments – for the first time during the presentation. It was good timing.

We presented the slides above to the judging committee. We won. The credit to this victory goes to the people involved in the project, because of the numbers I guess we had covered the most amount of ground. However this isn’t trying to detract from the Herculean efforts of the programming team and other participants in the hackathon. Participation in our project wasn’t strict, people flitted in and out on the basis of their skills, in all about 4/5 people could be considered to be core/’never say die, I’m here till the end’ members. But the entire project had around 10-12 people that scribbled on the board, gave advice or rolled up their sleeves and hacked a little. This blog is dedicated to all of them.

Photos from the event can be found on Flickr here:

Written and submitted from the Nottingham Geospatial Building (52.953, -1.18405)

Involvement of Community Organisations

Community Based Organisations Learning About Our Project

Community Based Organisations Learning About Our Project

The first community forum went well. It was deal breaker for the project in the sense that if we didn’t get the community to share our ideals and objectives, making them their own, then the project would fail. Now the map is basically complete for first draft. We have enough of a basemap so we can now support platforms like Ushahidi and enable blogs to be geolocated.

Collecting the data and producing the map, as I’ve previously mentioned is only a first step. The same can be said for involving the students and community members. By creating a small nucleus of highly engaged people, proficient in mapping and storytelling techniques understand the project, they can evangelise the project to others in their community. This ‘infects’ the community from the inside allowing for more people to interact and share the project without ‘outside’ involvement. This will in time hopefully reach a plateau where the entire process of updating the map, reporting and blogging becomes self-sustaining using only the initial equipment and investment.

With this in mind, in the build up to the final community forum of the project (where presentations will be made to the community as a whole ie. interested citizens, civil servants and politicians) we gave a ‘pre-release’ talk to ten community based organisations. The format for this was quite simple, the students introduced the map with it’s features and intended functionality and the community members introduced the storytelling elements.

Within this process I spent most of it being a photographer and an observer. It wasn’t quite seeing the monster you created evolve but when presenting both students and the community are owning the process. The community organisations engaged in a Q&A session then participated in reporting using Ushahidi.

During the Q&A many questions dropped out regarding the future of the project and how the map can be used further. Because we are still formulating the future strategy it is difficult to say what the next step will be, but it will be along the lines of franchising to other areas overseen by community, NGO/CBO and Ground Truth, constructing this framework will be taking place for much of the coming week.

We are also printing the map and distributing it. This is key; in using Open Street Map, the collected data is freely available for viewing and data analysis without restrictions like a prohibitive licence. However accessibility to computers and internet understandably is a problem in communities like Tandale. To enable the community to view their map we will be printing A2 maps for placement in the sub-ward offices and printing A4 sub-ward handouts.

By placing it in a communal areas for each of the communities, we aim to reduce the barrier of people using the map by making it accessible. This process has started with our small nucleus of students and community, expanded by involving community based organisations and will be expanded further by integrating the map into governmental offices at the sub-ward level. Having the map built into the fabric of the community from the beginning of the project should make further incremental additions easier.

Community organisations being fully involved in the project is the next step, the process has been started and the ball is rolling. However Eid is coming in the next week, so everything is going pole pole, Swahili for slowly slowly. However tangible results are starting to become very clear, on all levels with all stakeholders.

Written and submitted from Slipway, Dar Es Salaam (-6.75174,39.27117)

Water, Sanitation and Geography

I have been surprised with during the Tandale project with how community members are familiar with the geographic boundaries and extent of their community. On touring areas with community members, it was clear how they used geographic features to navigate. Also the administrative boundaries were formed through natural features like rivers, without being imposed upon by an outside force.

Previously when facilitating mapping (essentially “There isn’t anything here, go have a look”) the mappers would have a difficultly in collating the map and their own mental model. Here in Tandale reading of seemed to be a lot easier than when I have previously experienced.

Map reading is a difficult skill, essentially it starts with understanding that the map is an abstract representation of space. As a map is a representation of space; a visualisation of various elements is at the whim of the cartographer (in our case the esteemed people which write the map styles for OSM) and the data that the cartographer/surveyor collects.

The people of Tandale seem to have a spatial awareness down to a very precise art, using landmarks and features with which to demarcate areas. This also relates to official and unofficial landuse within Tandale. Because of the lack of formal solid waste collection, most of the waste is dumped in swampland or wasteland. Unfortunately these waste areas have no buffer with residential homes, further illustrating the potential for disease.

On a tour of the Sokoni sub-ward executive he spoke at length on sanitation and water security. The conversation turned to common diseases and illnesses within Tandale with Malaria and HIV unsurprising common. Also mentioned in the same breath was Typhoid and Cholera, which, according to the officer,  outbreaks are common. Looking at the state of sanitation and drainage this is very believable.

We believe that the first step to solving these problems is to have a map. Hopefully have enough data to give evidence of the problems, to both inside and outside the community. We have also mapped dumping grounds, formal and informal medical facilities, toilets, water points among many other things. Using the map as a basemap in Ushahidi instance allows for the community members to use the map that they have created. Now our focus turns to completing the feedback loop, so there is an interface for the reports. Funnily enough that’s where my PhD comes in…

Written and submitted from the City Style Hotel, Sinza, Dar Es Salaam (-6.47319,39.13199)

Ownership Of Community Reported Problems In Tandale

The Sokoni Sub-Ward Officer (right) and Community Members

The Sokoni Sub-Ward Officer (right) and Community Members

Within the project we have effectively split into two groups along the lines of students and community members. The students are effectively bug hunting, filling in missed tracks and POIs. The community members are focusing on the submission of data into the Ushahidi instance,

Today was quite special in that the Sokoni sub-ward officer joined one of our community teams, reporting the problems within the community. They engaged fully in the process, finally seeing the problems they had found on the map.

I could feel the sense of achievement from the community members and students. The community members were animated at showing their work to the representatives of the local government, especially those who were on friendly terms with the officials.

Initially I feared that the dynamic of student and community would be one that would be difficult. The students are from Ardhi University and are the best and brightest urban planners that East Africa has the offer (the students come from Zambia, Kenya, Rwanda and Congo). They are on average technology literate and have, with some guidance, become power-users of JOSM and other OSM tools. Understandably the community took time getting to grips with the software and methodology.

We ran a Saturday session, now they are quite proficient with computers, GPS’ and cameras. Closing the gulf the community can help the students and vice-versa. Because we wish for the project to be sustainable the drive must come from the community members, and it has. They’ve really taken ownership of the project. Before they would be querying the OSM Tags, or how to pull tracks and waypoints off a GPS. Now they navigate all tasks with ease.

We start each mapping session at 0900 sharp. To be ready for this time, we arrive at 0830 to set the projector, plug laptops etc. Now I set the projector up, once I’ve handed out the laptops. Nearly all members (community and student) are present, ready and waiting. They, and by extension, Tandale, now own the project.

Written and submitted in the World Bank Offices, Dar Es Salaam, Tanzania (-6.81298, 39.29194)

Ramani Tandale: Work In Progress

Tandale as of the 19th of August 2011

Two weeks or so ago I posted about the impending Map Tandale project. In it I spoke about the broad aims of the project and the methodology. It was accompanied by an almost blank slate; the map below. This is compared with the map above taken on the 19th of August 2011.

Tandale as of the 9th of August 2011

Quite a lot of progress here! One thing that should be considered is the schedule upto this point. We held the community forum on Tuesday 9th of August. Wednesday was about the community and students working together and getting familiarity with the GPS;mapping with your feet. Thursday and Friday downloading GPS tracks and editing with JOSM was covered. Because of enthusiastic community members and students we ran an additional session on Saturday, attended by a few, to further explore the editing process.

This meant on Monday we could really get started with mapping with everyone at the same skill level, and we did. I now think we’ve got around 90-95% of the major tracks done, the rest of the tracks should be finished on Monday/Tuesday. POIs like water tanks and toilets are in, again we have some areas that need to be mapped and these will be picked up by Tuesday.

Community Members Using Ushahidi

Because of us nearing completion we’ve entered phase two of the project; using the map we’ve created to tell stories and highlight issues. To do this we’re using Ushahidi, categorising reports and issues along Water, Health,  Education, Accessibility and Security. These themes were identified, not through previous experience, but from community members themselves.

In the community forum they provided areas where they believe improvement is needed. With us delivering on a system that monitors what they want monitoring we hope to give the community ownership of the project. Once the taxonomy exists within Ushahidi we requested that the community enter some of the issues that they face along the identified themes. After familiarity with the system was gained they then went out into their sub-wards, their communities, and gathered stories and issues that were happening then. The website for the Ushahidi instance is, please do have a look at it, though you will probably have to translate the reports!

One point of note is the name of the project; also chosen by the community. Ramani is Swahili for map. Ramani Tanzania, Map Tanzania. I believe that the community really feel part of something and are anxious to contribute further.

Understandably this is only the first step, realistically we are barely two weeks into the project, however it is shaping up well. In the coming weeks we will focus on using the map to tell the stories and issues in Tandale and looking how the project can become sustainable with a tangible benefit.

So far the tangible benefits encompasses the urban planning students who have gained surveying knowledge and the community members who have improved their own skills with technology, however we want the project to succeed and be a continuing success. To do this involvement needs to occur between Community Based Organisations, Non-Governmental Organisations, Charities and the City Council. We have already engaged these actors on varying levels but will ramp up interaction in the coming weeks.

As a parting note I would like to personally thank Lucy Fondo and Hassan Abdalla from Map Kibera. They engaged the community members and students with equal aplomb and along with Simon Kokoyo, from Map Mathare, smoothed the implementation of the project removing obstacles like language, making it child’s play.

Written and submitted in the Tandale Ward Office, Dar Es Salaam, Tanzania (-6.797164,39.242736)


Get every new post delivered to your Inbox.

Join 1,301 other followers