Thursday, September 9, 2010

Apache Release Management

I originally wrote this article shortly after performing Apache Karaf's first release as top level Apache project. Since then I have participated in numerous releases, and have come to view release management as both an art and a science - the roles a Release Manager (RM) takes on with a project are many and varied. I hope the below article helps introduce some of the issues a RM tackles, and provides some insights in to how Apache projects reach release status.

I recently had the privilege to perform the release management duties for Apache Karaf's first release as a top level project. The process went relatively smooth considering we were creating our release guide as we worked. So for this blog entry I'd like to talk a little about the experience in the hopes that other release managers may gain some more insight into the goings on in Apache projects.

The duties of a release manager (RM) are diverse, it includes serving as a facilitator, gatekeeper, architect, support engineer, and overall as a coordinator. In these roles the RM will deal with software defects, issues, risks, change requests, feature requests, deployment and packaging, and community involvement. Sometimes these duties are liken to the practice of herding cats, however it's so far been my experience that most parties involved in Apache projects just want things to go smoothly towards each release.
The facilitator acts as a go-between different parties.
As a facilitator the RM's goal is to provide the drive towards a timely release. The RM works here as a liaison between different contributors, committers, Project Management Committee (PMC), and other projects to guarantee the smooth delivery of a release. Normally this process starts by the RM sending out a proposal to the project developers requesting that a release be cut at some up coming date. When a release candidate is prepared the RM will then hold a vote for project members to ratify the release.

Apache Karaf build
and release infrastructure.
The gatekeeper role is the practice of having someone held responsible for the resources required for production of the release. In Apache projects this responsibility is somewhat modified in that the RM typically has committer status on the project (quite often part of the PMC), has a signed PGP key for the project, and access granted to production support systems. The actual systems used to produce a release candidate are often the personal property of the RM, so proper documentation of how to build the release environment must be maintained in order to not create a magic build machine. Pictured to the right is the Apache Karaf build and release infrastructure. When not in use I keep the machines physically disconnected from the internet, no development or testing use is permitted on these machines (I have other resources available for those purposes). There is an excellent article on the measures some RMs go to in regards to protecting their PGP keys here. I rely on a secure physical location in a remote part of Newfoundland, up to date security patches, and disconnection from the internet when not in use, as my peace of mind from having the release system I use becoming exploited.

RMs build processes like
fortifications. - protecting
build quality.
The architect aspect of the RM role is in building processes or tools to manage the project release. In Apache Karaf some of these tasks can be seen in the release guide. Having this release guide published helps not only the RM during the release process, it promotes the open process by which the project operates. Decisions and practices are clear throughout the release process, and are open to discussion. An example of the type of tools used by RMs include code signature verification scripts (test all pgp signatures, MD5s, etc) and the Apache release audit tool. In relation to these tools are the continuous integration system Jenkins, and automated analysis tool Sonar, which RMs use in the lead up to a release candidate. Reviewing these tools status reports the RM can gain confidence in the quality of the build they are promoting towards release statue.

As supporting systems
increase, the RM has to
ensure they're maintained.
The support engineer role of the RM is to resolve issues encountered with the build and release infrastructure. In setting up for the first Apache Karaf release a lot of my time was spent on creating accounts or correcting generated permissions. The next release should go smoother since a lot of little details have been taken care of now. Some of the infrastructure that the RM has to work with includes (but not limited too) the source control management system, issue tracker, project wiki, project website, Continuous Integration builds, Sonar reports, and Nexus repositories.

Finally the coordinator role is to balance the needs of the contributing developers with that of the communities requests. In the Apache world this means attempting to align contributors efforts with that of user and other projects needs towards making the release date. This may mean communicating with developers to attain status' on particular issues in the project tracking system, or assigning issues to available resources to get required work resolved.
The co-ordinator role can feel like balancing on the edge of a 200 ft cliff.
In conclusion the Release Manager is an indispensable role in the process of producing Apache software. They handle the many competing demands of parties involved in their project and help keep each release on time.

No comments: