or Scrum for disparate projects

Scrum's iterative approach to development consists of four main sections:

• Project planning
  Collecting stories from project owners, prioritisation and planning for the upcoming sprint.
  
  
• Development sprint
  The actual development work, directed by the prioritised backlog created in the project planning stage
  
  
• The Demo
  A demonstration of the completed stories provided by the development sprint
  
  
• Feedback
  A retrospective on the good, bad and straight up ugly parts of the most recent iteration.
  
  

This is an excellent framework for rapid, focused software development. In the scientific software realm maintaining focus on the problem (and hopefully, a possible solution) whilst keeping momentum and allowing for rapid change is essential. These main milestones in the iteractive process are usually held together by a daily 'stand up' meeting: a "scrum", from which the framework takes its name. This ensures that all the developers on the project are on the same stage as to the state of progress, that focus is maintained and that any road blocks to progress are identified and removed.

Disparate projects require different approach
Within the scientific software realm, however, things operate on a subtly different level. You are less likely to find yourself in a larger team focussed toward a single project (although, certainly, such projects exist), and much more likely to find a team working on a wide range of scientific problems.

This is not to say that the team wouldn't have a collection of libraries shared between projects, but that the projects themselves are geared towards very different users in very different environments. Some examples, currently in play at Sanger:

• Automatic trace release, as required by our funding source, and used by project leaders
• Illumina sequencing tracking, used by faculty members to organise and track sequencing requests
• Laboratory information management, used by lab staff to direct experiments at the bench
• Repository for short read traces, used by all to query and recover short read traces

Such a wide range of multi faceted projects can be seen in informatics departments all around the world. Whilst the physical implementation may be similar, the demands and prioritisation from each set of users can be very different, framing and directing the projects differently. Can a 10 minute daily meeting, limited to answers to only three questions, really hope to pull together such threads?

The Good

Communication within teams is absolutely key to progress, and a daily stand up meeting provides a regular point of contact for information exchange. This is a Good Thing, but doubly so with disparate informatics projects where regular contact also provides a fulcrum for the cross pollination of ideas.

The solutions to problems encountered in one project can often seed an innovative change in direction elsewhere.

The Bad

Even short meetings use up valuable time. With larger teams (6+), this can become considerable, especially if the focus of the meeting is lost. A steady, guiding hand is essential to make sure that the meeting stays on track and adds value.

Editors have a role to play in development meetings as much as they do in other areas.

The Ugly

As recently posted, I've been concerned with promoting Flow recently. A regular, daily meeting is pretty much guaranteed to break the flow of work, no matter when it occurs during the day, but this interruption can be damaging to both the work, and the meeting.

Beforehand, developers are required to break their process and collect their thoughts for the meeting, swapping back to their previous task when the meeting concludes. Because the meeting is deliberately short, developers can find themselved in limbo: focused on neither their work nor the meeting, often forgoing the opportunity to collaborate in favour of getting back to their broken task.

Improving value

The often rapid reduction of 'real' roadblocks to progress, and the lack of overlap between projects on scientific software teams leaves the daily stand up meeting a little redundant. It is reduced to a simple status update, often adding little value towards progress.

An alternative approach could be a weekly 'State of the Union' discussion in which each team member takes a few minutes (no more than three) to present last week's work and outline the challenges for the next.

• A slightly longer, less frequent meeting removes interruptions and promotes Flow. First thing on a Monday morning is probably a good time to get started.
  
  
• Less swapping between tasks, since the meeting is less frequent
  
  
• A simple agenda (which can be the same each week, as happens at Apple) promotes focus
  
  
• Provides an opportunity to highlight new libraries, patterns or plugins which can be shared across the team.
  
  
• Limiting each team members time is equivalent to limiting what questions are answered at the daily scrum meeting: it promotes brevity and prevents show stealing.
  
  
• Can for the basis of a regular team retrospective on the most recent sprint.

If your daily meeting is suffering, or if the idea of it is putting you off implementing Scrum in other areas, it may be worth a shot.