Wednesday, 8 July 2015

Ark/PEAS SchoolTool Project (Part 2): System Evaluation

In my previous blog post I discussed progress in our project to get SchoolTool working across the PEAS network in Uganda. In this post I'm reflecting on the system more generally. But before I get to specifics, I'm going to digress for a few paragraphs.

Before I joined Ark, a friend and I set up a company called Unlocked Guides producing children's travel books. Here are the two main lessons I learnt from the experience:
  1. Make your customers part of the design process. They know what they want, and aren't afraid to tell you. Especially nine year olds.
  2. People massively overvalue ideas, and massively undervalue execution.
The background to point 1 is that the first draft of our first book (London Unlocked) was, with hindsight, terrible. Lots of pastel colours. No real structure to the pages. Not enough jokes. But mercifully, we workshopped the design with two hundred children before launching. And they told us that the first version was rubbish. So we iterated and iterated and in the end created a children's board of directors to gain regular input from our target audience. We ended up with a great product which still sells a few thousand copies a year, six years on.

The context for point 2 is that several times before and after launch, we got nervous about other people attempting the same thing as us. Lonely Planet launched a rival book. Sandi Toksvig wrote a children's travel guide. And, adorably, a furious parent approached us one of our book launches and told us she had been thinking of doing the exact same thing, but that her (as yet unlaunched) concept was much better than ours. But the thing is, because we'd worked so hard to make our product great, we never really suffered too much from competition. Rivals either fell by the wayside or ended up helping us by creating a new, browsable bookshop category (before we launched, children's travel wasn't really a thing). And we never did hear again from the furious "better idea" lady.

Which brings me back to SchoolTool.

If we're being super-picky, we'd say that SchoolTool hasn't had enough customer input over the years - particularly when it comes to school improvement functionality. For example, assessment was a rather thin module when we first started working with the system - it only tracked absolute scores, which made performing analysis really hard. And while the attendance module had some neat features for fast data entry, the standard reports available to track attendance changes over time were either limited or non-existent.

So we've become the nine year olds. As well as implementing the system in Uganda, we're speccing features we think will make the system work for school improvement, and paying for their development. (SchoolTool may be a free, open-source system, but it doesn't have an active volunteer developer community, so new features come at a price). Since we're requesting features based on both input from PEAS users during the product rollout AND our own experience of supporting SIS in the UK, we're comfortable we're asking for the right things.

At the moment these features sit in our own project branch on the SchoolTool code respository, but they should all make it into the main project code in the coming months. The features we've funded include:
  • upgraded assessment module (making it much more flexible so it can handle a wide range of assessment structures)
  • upgraded attendance module (added group filtering and colour coding for easier data entry)
  • new assessment and attendance reports
  • central data sync and dashboard to allow a central office to see data from multiple schools
When it comes to to ideas vs execution, in contrast, SchoolTool has a fabulous and unique story to tell. I've lost count of the number of people who have conceived of an open-source SIS. We reviewed six before selecting SchoolTool. Scholarpack (which is now in over 300 schools in the UK) started out as a free, open-source product. We even considered developing our own.

But to our knowledge, the Schootool team are still the only ones that have actually done it at any scale and stuck with it. And it's still free. And it works. The problems mentioned above - and the ones covered on the previous blog post (like the fact that it only works on Ubuntu, and requires some technical knowledge to install) have no doubt constrained uptake. But our experience has been that if you can overcome these fairly minor hurdles and get the thing installed, it'll do a great job for your school at my favourite pricepoint. So sure, it could be better. But give the guys credit - they executed well on their vision of a free SIS where others have changed tack or given up.

What's next
For the next phase, Fran and Sunesh are looking at how we could reduce the barriers to uptake of SchoolTool further. For example, what if we could pre-install the software on cheap, compact hardware so that a school just needs to plug it in and bingo, there's a SchoolTool server up and running? And how could we produce better user guides to make adoption possible without requiring training sessions from experts?

We're also planning to test alternative ultra-low-cost SIS products in different contexts - we'll talk more about that in a few months time. But for now, if you're reading this and considering whether to use SchoolTool in your school, we encourage you to give it a go.

Ark/PEAS SchoolTool Project (Part 1): Implementation Report

Ark/PEAS SchoolTool Project (Part 1): Implementation Report

As we wrote last year, some colleagues of mine (Fran and Sunesh) have been working with the  PEAS network in Uganda over the past year to get SchoolTool up and running as their School Information System (SIS). It's been a fascinating project, and by helping PEAS deploy SchoolTool in nine schools we've really got to see the product up close. In this blog post, I'll update on progress over the past twelve months. In my next post, I'll reflect on SchoolTool more broadly.

Pleasingly, after a year of hands-on experience, our view of SchoolTool remains the same as at the outset: it's an intuitive and easy to use system. Teachers have little trouble getting started, even if they've never used an SIS before. Thanks to its usability, nine PEAS schools are already using it effectively to track student and teacher attendance, and a few trailblazers are also using it to manage assessment. That's big progress, and a testament to all involved (with special mentions to Abraham and Jakub from the PEAS team).

I was out in Uganda in the Eastern province in late May, and visited four PEAS schools in Malongo, Mukongoro, Nyero and Ngora. Malongo has had the system live since February and the impact on performance is already evident. Teacher and student attendance has increased, and crucially, the school are thinking through how to take action based on the messages contained in key system reports. Fran facilitated a fabulous conversation in the school around how to improve teacher attendance further. One teacher suggested putting the main teacher attendance report on the staff room wall to encourage shared ownership of performance on this key metric. That's the kind of conversation that makes you know that something good is happening.

Group discussion at Malongo about how to use SchoolTool effectively

Mukongoro, Nyero and Ngora all had the system installed during our visit, and broadly speaking, it was a smooth process in each case. The SchoolTool setup process isn't completely intuitive, so we still find that we need technical expertise on hand to be sure that things are established in the right way. That said, we've become accustomed to the potential pitfalls now, so troubleshooting takes minutes / hours, rather than days. Also, the schools had all done a great job of preparing spreadsheets for data upload, which is in many ways the part of the setup process where things are most likely to go wrong. Also, between our team and PEAS, all schools got a solid set of training on how to get started.

It was particularly heartening to hear school leaders from these three schools talk about their colleagues in other schools who were already using the system, and how they plan to share knowledge and expertise. Another nice touch was that Fran and Abraham had planned joint training sessions. Since there will only be a few users in each location, shared training works well - as with all systems, everywhere, the trick to high quality adoption is to become part of a community of users sharing experiences. PEAS are well on the way to making this a reality.

Training at Nyero 

A final thought: I think it's important to empahasise that in all the schools I visited, the value of data was well understood BEFORE SchoolTool was implemented. See the colourful notice boards in the above picture from Nyero? Well, several of the displays were filled with well-presented tables and charts showing key school metrics. I mention this because it's important to acknowledge that part of the success of our project to date is that we're working with people who already "get" the importance of data. So it's worth remembering that a good deal can be achieved with school data even without a School Information System. The value of SchoolTool is that it streamlines data collection and enhances analysis. Crucially, that then leads to more impactful data action.

Ark/PEAS SchoolTool Project (Part 2): System Evaluation