History of Project Flow & Tracker

Early versions of Project Flow & Tracker used at http://awebfactory.com.ar

Project Flow & Tracker Washington: The Making of (early 2009)

This is a log of how Project Flow & Tracker was built.

 

First steps

I had a blast right from the start:

Created SVN repository
Installed Basic Drupal
Installing the spanking new Drush
Re-read the bibliography
Reviewed existing toolsets

Created SVN repository

  • Created the repository itself
  • Created repository basic structure
  • Imported basic drupal install (from ./sites on down)
  • Committed at every important point
  • Finally tagged basic Drupal install

Installed Basic Drupal

I installed Drupal 6.9, the current release of its moment, from CVS, of course. Then I configured it into a basic install: from the SVN log:

@7   	 [7]   	02/06/09 14:58:58  	victorkanebasic Drupal 6-9 install (tag)
@6 	[6] 	02/06/09 14:58:39 	victorkane 	Basic install tag structure 	(edit)
@5 	[5] 	02/06/09 14:54:43 	victorkane 	Drupal 6-9 plus cck, views, devel and admin_menu modules, plus a 	(edit)
@4 	[4] 	02/06/09 14:45:41 	victorkane 	drush install devel module 	(edit)
@3 	[3] 	02/06/09 14:42:05 	victorkane 	added db backup to version control 	(edit)
@2 	[2] 	02/06/09 14:21:29 	victorkane 	initial import drupal 6-9 plus cck, views, and admin_menu modules, plus acquia_marina theme  	(add)
@1 	[1] 	02/06/09 14:15:38 	victorkane 	Initial structure

So, just cck, views, devel and admin_menu modules, plus the acquia marina theme.

I used the amaaaaazing new Drush 2 (6.x-2.0-alpha1) and Drush Extras which live completely outside of any one Drupal instance!

Installing the spanking new Drush

Many ways to do it. What I did on my development workstation:

  • I disabled then uninstalled then removed completely from my file system all traces of Drush in any given Drupal installation.
  • I took Drush 6.x-2.0-alpha1 and untarred it  under my home directory.
  • I edited my .bash_aliases file and included the following line (I baptized it drushcli so I could drum it into my head through dint of praxis that this was not your grandmother's drush):
    alias drushcli='php /home/victorkane/bin/drush/drush.php
  • Smiling to myself, I download Drush extras 6.x-2.0-alpha1 and (whaaaaa??) untarred the contents into a .drush subdirectory in my HOME directory (/home/victorkane/.drush). I really trust this guy moshe weitzman.

Then it worked just the same as always, from any Drupal instance document root directory. OMG.

Re-read the bibliography

Reviewed existing toolsets

General Analysis

Analysis is not so effective if it is not  intimately associated with a code sprint itself (that is, itself done in an incremental and iterative manner). However, I did want to get a "total immersion" mindset before starting out. To do so I reviewed:

Feedback from previous PFT users

Roadmap from DrupalCon Barcelona 2007 presentation

Input from bibliography

Process toolsets

Drupal architectures

Feedback from previous PFT users

  • Usability
    • Be able to follow simple workflows in entering roles, their associated user stories, their acceptance tests, iterations and releases.
    • Be able to visualize the progress made in a project (i.e. burndown graphs.
  • Link up to source code version control repositories. Be able to trace version control (changesets, or builds) to user stories and back.
  • Link up to running demos, test sites, production sites associated with iterations, releases, monitor those sites and Drupal instances.
  • More support for estimates, planning and teamwork.
  • My own ideas, among them:
    • embed all of this inside the project itself, in the context of prototyping with Drupal itself, and to do so kick-start that project with a number of useful modules, etc.
    • Package as distribution or installation profile
    • Internationalized and localized

Roadmap from previous DrupalCon Barcelona 2007 presentation

  • Multi-lingual support!
  • Quantitative tracking and statistics
  • Linkup with SVN for traceability
  • Linkup between requirements and architecture
  • Linkup with automated testing
  • Multisite install for self-contained projects with top-level overview
  • Workflow wizard as well as dashboard for each user when they login

Input from bibliography

Go back to the roots, of Agile, Scrum, etc. But also... KanBan, JIT, systems originally created to super-exploit workers but which could somehow cast light on how workers could cooperate efficiently and democratically in many social contexts.

Process toolsets

Everything seemed to be leading back to physical cards in plastic envelopes, on walls, empty trolley carts in the factory, something easily visual.

Why software? Why not colored plastic and cardboard... And then it hit me: shoot, everything is a card! A card that can be stuck on a wall next to other cards, so the overall picture is always clear; and I can pluck a card off the wall and get down to it, in comfortable but non-overwhelming detail. That's got to be in there!

Drupal architectures

  • jQuery will be the usability candy once we get it done!
  • If everything is a card, how do I have content types which can inherit easily from other content types?
    • Share fields among content types, with one class (card) being the "abstract" content type never to be used, except for the place where all the fields are housed (never deleted).
    • Use the Content Type Inheritance Module, with its automatic theming template polymorphism into the bargain?
  • How do I show ad-hoc and non-exclusive relationships between cards (nodes) of different kinds: show features containing user stories, show projects containing releases, releases containing iterations, iterations containing user stories, tasks, defects, change requests...

OK, enough "general analysis", time to get to work...

Using PFT to build PFT from scratch: project

I've decided not to use Trac (which comes installed automatically with my repository instance) but PFT itself to build PFT. That way, its functionality will be based on its own genesis.

You know: project -> roles -> user stories -> releases and iterations -> planning -> tasks/defects/change requests -> build -> test and all the viewing and editing that goes on all the while...

Create PFT user roles on the system

Create the project content type

Create a couple of projects

Support each user specifying a  current projects list

Build info

Create PFT user roles on the system

  • product owner

  • team leader
  • team member
  • create some example users

Create the project content type

So, ...must ...create ...project PFT said to itself.

  • Create content type project with just Title and Body, called Project
  • No, wait, import Project type from legacy PFT! Should do splendidly for now.
    • You do realize you are going to import a CCK Drupal 5.x into a Drupal 6.x instance, don't you? Just do it! Hmm, would help if you installed Date modules...
      $ drushcli pm install date --svnsync --svncommit --svnmessage="drush installed date"
      Project date successfully installed (version 6.x-2.0-rc6).
      Project committed to Subversion successfully
      victorkane@victorkane:~/Work/AWebFactory/projects/PFT/Washington/www$
    • Yeah, but CCK for Drupal 6 chokes on CCK exported from Drupal 5. But there is still a way out! Upgrade the old legacy PFT to Drupal 6 then you can easily harvest everything... OK, later
  • Created very simple project content type:

    Label Field Field type
    Name title
    Short name

    field_project_short_name Text
    Owner

    field_project_owner User reference
    Team leader

    field_project_team_leader User reference
    Estimated start and ending dates

    field_project_est_date Date
    Notes

    field_project_notes Text

    Note: The team members will be the sum of all those signed up to work on user stories and tasks.

  • Set up content type and field permissions for  team leader / project owner.
  • Set up content permissions so anonymous users can see the demo site. Later on OG access will be used so there can be private (premium?) projects.

Create a couple of projects

  • Log in as team leader
  • Created first project
  • Log in as product owner
  • Create PTF project
  • Create second project
  • Make simple river-of-projects view with url "projects" as default front page (installed advanced help module while I was at it).

Support each user specifying a current projects list

Make PFT a current project

  • Need variable "current project" and make Project Flow & Tracker current project?
  • Flag? added token
  • Each authenticated user can now flag a project to include it in her navigation menu item "My current projects"
  • Changed river of projects (rop) view, by adding relationship with current user current projects flag, to sort descending (differently for each user) according to most recently selected current project.

Build: Changeset #12 - initial implementation of project business object; basis of cck, views, date, advanced_help, token and flag modules

Using PFT to build PFT from scratch: project documents

This is so cool!

I want node2node to help me drill downn to project documents included in the project. So I want those project documents to be a content type in their own right. But I also want to be able to have them be hierarchical books too in their own right! I don't want to use up the book content type for project doucments only!

 

So:

  1. Mark the project as the target.
  2. Create a project_document, checking the box to include it as a child of the project.
  3. Before saving, go to the Book Outline section and start a new book with this document as root. Even though the content type remains "project_document", it acts like a hierarchical book. Awesome!

Using PFT to build PFT from scratch: roles

Again, functionality of PFT will be based on its own genesis. That means that I will create the functionality of PFT according to my concrete needs in wanting to use an agile approach to its building.

So far, I have implemented the project business object, and have used that to specify the project and added functionality for each authenticated user to be able to flag the projects she is working on and to be able to see them towards the top on the homepage.

According to the process we are following for website application development, based on Chapter 1 of my book Leveraging Drupal: Getting your site done right (available for free download here), I now need to specify the roles: the actual users who will be interacting with the website application.

Of course, in order to actually create the project content, I already created Drupal authenticated user roles for them. Since Drupal is its own best prototyper, it is also in this case.

However, since roles need to be defined and constitute the actor in each of the user stories we will need to specify the functionality of the webapp, we do need to have a content type created for them.

Here I did the following:

Decide upon architecture for drilling down node trees

Support drilling down from the project to the roles associated with it

Create the role content type

Create the actual roles for the PFT project

Build info

Support drilling down from the project to the roles associated with it.

Here it is necessary to determine the architectural baseline of how we are going to support relationships between nodes. I have used the Node Family module in the past. But no version is listed as available for Drupal 6.x, and no date is set for that.

Alternatives:

With Node Hierarchy, it would be possible to integrate with pathauto for hierarchical and intuitive URLs reflecting the hierarchy.

i.e. project/projectname/user-stories would show all the user stories.

Testing Node Hierarchy

I installed it and decided to try to make two pages and four stories and see if I could assign stories to pages, especially if a given story could be the child of more than one page.

Nope, no go; a given node can be the child of one and only one parent. As seen in the next section, we need a node to be able to be the child of multiple parents. I regretfully ripped out the module and restored the database image to how it was before I started testing.

Testing NAT

How would NAT work anyway?

To make a hierarchy of Story's "in" a page, one way might be the following:

  • Create a Pages vocabulary (like a Features, an Iteration, a Release, where Iterations and Releases are going).
  • Configure the Page vocabulary to categorize Story nodes
  • Configure NAT so that whenever a Page is created, a taxonomy term is created in Pages with its Title (and removed when the node is deleted).
  • Create a page "test one", and another "test two".
  • Confirm that the appropriate terms are automatically created.
  • Create several Story nodes "assigning them" (or not) to the appropriate page container (i.e. taxonomy term of vocabulary Page).
  • Make a view to show the relation
  • Yes! Sweet! light! clean! Click on taxonomy term "Test two" and you get a river of stories associated with that term, of course.

 

To implement a set of Roles belonging to a Project:

  • Install pathauto+global redirect to get taxonomy terms with cool configurable urls
  • Install NAT module

Create the role content type

  • Create a simple Role content type (Name and Description (title + body)).
  • Create a Project Vocabulary to contain roles
  • Configure NAT to relate project nodes to the Projects vocabulary.
  • Sync so that the existing projects appear in the proper vocabulary.

Create the actual roles for the PFT project

  • Create the roles for this project, assigning them to the appropriate project in the Project vocabulary.
    (Later on releases can also be assigned to the same project, with the Projects vocabulary set to contain releases, and features for that matter in the same way, as well as roles now.)
    • Product owner
    • Team leader
    • Team member
  • With the Acquia Marina theme this works out really cool because you can configure each vocabulary to appear on its own separate line, so you can see which project, etc. and... when you click on the project you get a list of the roles in that project! Cool!

TODO: some kind of tree display (taxonomy tree, term tree...?)

We've practically got our "drilling down" architecture all sewn up, and without those pesky node reference thingies also!!!

Build info

State: Changeset #16 (see http://awebfactory.svnrepository.com/pft/trac.cgi/log/ changesets 13-16)

 

Edit: 2009-02-25

  • Node Relativity: could be cool, but stalled in all its 6.x branches.
  • Node 2 node (yes: allows many to many relationships AND semantic role names)

Turns out to be exactly what I am looking for here. Have to write a views handler, tho; but light, simple as sin and dead easy to use. And shows really cool relationships, plus navigational node to node links...

 

Using PFT to build PFT from scratch: user stories

Background: Different disciplines lead to different relationships between the user stories and other elements

At this point we just want to be able to get the user stories into the PFT system

Create user story content type

Edit 2009-02-25: Using node2node module for drilling down and zooming out because it allows many to many relationships AND semantic role names, so not using NAT anymore.

 

Create Vocabulary Role to contain user stories, configure NAT to add roles to role and sync

Create Vocabulary User story to contain, for now, user stories (recursive splitting of a single user story into child user stories).

Configure NAT to add user stories to user stories

Enter user stories

Some kind of drilling down from project to user stories

Build info

ToDo

Background: Different disciplines lead to different relationships between the user stories and other elements:

The business model (meme map) gives rise to a list of features.

Just as we can drill down:

  • project
    • roles
      • user stories

And in planning and tracking we can drill down:

  • project
    • release
      • iteration
        • user story
          • task / defect / change request
          • tasks allow multiple developers to work on a single story
          • tasks encapsulate bursts of analysis and design (including the user story analysis and design that gives rise to the disaggregation into tasks)
          • tasks are estimated and then tracked; they are the unit of estimation and tracking.

So we can drill down in business modelling:

  • project
    • meme map
      • features
        • user stories

And in architecture we can drill down:

  • project
    • roles
      • user stories
        • task
          • domain analytical classes
          • map to Drupal modules and semantics

Create user story content type

Label Name Type
Type

field_user_story_type Text
Owner

field_user_story_owner User reference (multiple)
User Story Points

field_user_story_points Integer
Conversation

group_conversation Standard group
Conversation

field_user_story_conversation Textarea (multiple)
Confirmation

group_confirmation Standard group
Confirmation

field_user_story_confirmation Textarea (multiple)

Much use could be made later of field level permissions.

By the way: remember to always set Create new revision in the workflow settings for all of these, so versions never get lost.

Added permissions for team leaders and team members (we will have organic groups manage access later on: projects, and all artifacts will be either public or organization-wide.

Create Vocabulary Role to contain user stories, configure NAT to add roles to Role and sync, configure in theme

Done

Create Vocabulary User story to contain, for now, user stories (recursive splitting of a single user story into child user stories), configure in theme.

Done

Configure NAT to add user stories to user stories

Done

Enter user stories

Done

Some kind of drilling down from project to user stories

Made two views (thanks to NAT! but it was very simple, did not use NAT views integration) and node templates to match.

View roles_in_project lists the roles containted in a given project by listing all roles containing a taxonomy term equal to the title of the project.

View user_stories_in_roles lists all the user stories containing a taxonomy term equal to the title of the role.

These views are then embedded into the corresponding node template in the theme directory. For example, here is the content div of node-role.tpl.php:

  <div class="content">
<?php print $content ?>
<?php
$view_name = 'user_stories_in_roles';
$args = array(0 => $node->title);
$view = views_get_view($view_name);
$view->set_arguments($args);
print $view->execute_display(0, $args);
?>
</div>

Build info: Changeset #17

ToDo

  • Drill down to tasks (see issue tracking and change management); at least task disaggregation.

  • Add diagrams to user stories and tasks

Using PFT to build PFT from scratch: release and iteration planning

According to Chapter 20 of User Stories Applied, the following steps are necessary for release planning:

1. Select an iteration length.
2. Estimate the velocity.
3. Prioritize the stories.
4. Allocate stories to the one or more iterations.

This means, actually: create a release, with a date, then based on how much time separates then and now, and how long you want your iterations to be, decide how many iterations there will be.

 

Once the iterations are there, user stories can be allocated to them.

There is no allocation of user stories to releases; they are either allocated to an iteration or not implemented.

Releases should have a number (like 8.10 according to an arbitrary scheme) and a name (like Gutsy). The Drupal title of a release should be automatically formed from these.

 

Note that thanks to the many to many character of node 2 node relations, a user story can be allocated to more than one iterations.

 

So we first install Automatic Nodetitles. Then create the release and iteration content types.

[field_project_short_name-raw] [field_release_version-raw] [field_release_name-raw] [field_release_date-yyyy][field_release_date-mm][field_release_date-dd]

So project short name has to be required; we need it anyway.

 

To prioritize the user stories, just create an initial release and iteration, allocate all the user stories to the first iteration, then use the ordering of children found on the edit form of the iteration to order the user stories.

Using PFT to build PFT from scratch: risk management

Every project runs a series of risks (usually a top ten, but there is no limit). Each risk has a title, a descripition, a degree of severity , and a mitigation plan. They can be added, modified and have their state changed at any time.

 

Risk Ranking (sorted by magnitude)

Magnitude   (

Description

Impact

Indicators

Mitigation Strategy

Contingency Plan

 

The following risk list uses these conventions:
Magnitude:                  Risks are ranked from 1 to 10. 1 is the lowest risk and 10 is the highest
risk. Ranking is based upon the criticality of the risk and the probability
of the risk occurring.
Description:                Brief description of the identified risk
Impact:                     C – Critical (Affects all project functionalities and baselines.)
H – High (Affects stakeholders needs and major product functionalities)
M – Medium (These risks are subject to contingency but most of the times,
a mitigation plan will be established in order to avoid the risk.)
L – Low (Generally these are risks for which Risk Acceptance strategies
will be held or quick mitigation plan will be implemented. Usually the
team will decide to live with the risk as a contingency)
Indicator:                  Metrics concerned.
Mitigation/Contingency: Plan to live with or avoid/transfer the risk

 

Select Project as relationship target and add risks, specifying that the project runs the risk.

Using PFT to build PFT from scratch: acceptance testing

Using PFT to build PFT from scratch: issue tracking and change management

Using PFT to build PFT from scratch: build, version control and bi-directional traceability

A natural: Version Control API module.

Also, it will be necessary to integrate with Trac (via RSS / feed api? strong parser?)

Using PFT to build PFT from scratch: business modeling meme maps and features

Actually going to do this last...

Where's the wireframe sitemap?

Make one if you feel it is necessary.

But:

Drupal is its own sitemap

Prototype with Drupal itself!

Project Flow & Tracker Barcelona (2007)

So over the past couple of years I have been working on web applications using the Drupal CMS as a web application framework, and I have used a lot of project and configuration management tools in the past. But excited about a year ago with using Drupal itself as a project management tool and a rapid prototyping tool at the same time, I developed the Project Flow and Tracker, designed to be a Drupal installation profile (in case that doesn't translate instantly and intuitively into something for you, it is a downloadable Drupal, which, when you download it and install it, will become a Project Flow and Tracker instead of just a plain vanilla Drupal installation).

So I went to Barcelona DrupalCon in late 2007 and gave a presentation on it and got a lot of really cool feedback. And, I actually used it with some clients, and got a lot of really useful feedback there.

Enough time has passed for a couple of things to happen:

Drupal itself is much more powerful (panels 2, nodequeue, jquery ...) and the ideas have had some time to mature pretty well.

So now I want to take the Project Flow and Tracker apart and put it back together again. I need to reverse engineer it and implement it again, into something super usable and which I hope will constitute a cool contribution to the Drupal community.

So there will be three big parts to this treatise, dear reader: An Intro, dealing with how to get involved in helping the shiny new Project Flow and Tracker become a reality soon enough for people to actually use it; an ongoing report on that process of development; and a section on how to use it, perhaps including a case study. And a couple of appendices thrown in on process engineering background, and so on and so forth.

Project Flow and Tracker Barcelona

Process Flow and Tracker is an agile process tracker built on Drupal.

Please visit the PFT Sandbox (reset to sanity once every 24 hours).

User: drupal
Password: rocks

 

2007 Barcelona DrupalCon Presentation

Somewhat outdated, but kind of fun, just to get your head wrapped around the why's and wherefore's, here it is.