Upload
pragmaticweb
View
1.225
Download
0
Tags:
Embed Size (px)
Citation preview
Intelligent Systems Laboratory
An evaluation of the pragmatics ofweb-based bug tracking tools
Proc. 2nd Intl. Conf. on thePragmatic WebTilburg, Netherlands23 Oct 2007
Harry S. DelugachIntelligent Systems LaboratoryComputer Science Department
Univ. of Alabama in Huntsville, USA
Intelligent Systems Laboratory
Context of inquiry
● Why web-based tools to support softwaredevelopment?– Distributed environments, or at least where the
participants might have difficulty meetingregularly face-to-face
– Web-based tools allow them to collaborate in agenerally cost-effective way
– Large number of activities and artifacts to becreated and maintained
Intelligent Systems Laboratory
Difficulty with tools● Effectiveness is determined by how well
participants understand how to use them.● Mere use of tools is not sufficient to support an
effective process.● Much current work is focused on characteristics of
artifacts or products● This work focuses on the roles and purposes of the
participants
Intelligent Systems Laboratory
Purpose in developing models● To better characterize and describe the
processes themselves● To formally analyze and evaluate tools with
respect to generally accepted processmodels
● To formally compare and contrast themodels with each other
● To arrive at a shared process model amongdevelopers
Intelligent Systems Laboratory
Techniques● Workflow modeling● Software problem tracking
– sometimes called “bug tracking”
● Conceptual graphs:– a convenient formalism– easily understood visualization aid to represent
the models
Intelligent Systems Laboratory
Workflow modeling● Aim is to provide framework for describing and
evaluating software development processes● Neutral with respect to being either normative or
descriptive.● Being “informal” does not render a model incapable
of being modeled● We assume that developing a model of a process
is generally more useful than not modeling it at all.
Intelligent Systems Laboratory
Workflow step● Starting with RENISYS
– An organizational role has particular obligationsto an activity, and an intention to accomplish
Intelligent Systems Laboratory
Beyond RENISYS● Include concept “Intention” with respect to a role in
the process.– Previous models show only the obligations (required,
allowed, prohibited) of a particular role.– No indication as to why a particular role would be given a
particular assignment.– For example, why would a program manager be required
to review a change, or why would a developer beallowed (but not required) to make a change?
● Include status with respect to that intention
Intelligent Systems Laboratory
Software problem tracking process
● Part of software configuration management (SCM)● Many people involved● Issues to consider:
– Who is involved?– What are stakeholders’ roles in the process’s success?– What responsibilities do they hold with respect to the
system?– What are their communication needs with other
stakeholders?
Intelligent Systems Laboratory
Modeling problem resolution processes
● Bug-tracking is one kind of problemresolution process.
● Standard ISO/IEC 12207.0 clause 6.8● Processes supported/implied by tools
– Bugzilla– Trac– sourceforge
Intelligent Systems Laboratory
Existing tools and processes● No explicit set of roles which are defined in the
process● Most tools seem implicitly geared toward a
particular change control process● Some of them appear to imply certain role● Others appear role-neutral.● Generic models for software engineering
processes
Intelligent Systems Laboratory
Ontology for bug tracking
Intelligent Systems Laboratory
Model for reporting a bug
Intelligent Systems Laboratory
Model for fixing a bug
Intelligent Systems Laboratory
Validation of model graphs● Data mining
– Use conceptual graph tools to scan the wealth ofexisting data (e.g., Ripoche and Sansonnet)
– Emails, forum posts, program source codecomments identifier names in programs
● Human subjects– Paraphrasing graphs in natural language– Analyze comparison graphs
Intelligent Systems Laboratory
ISO/IEC 12207 process
● Described by text in passive voice– E.g., reported by whom, to whom, who initiates
action, etc.
“… all detected problems are promptly reported andentered into the Problem Resolution Process; action isinitiated on them; relevant parties are advised of theexistence of the problem as appropriate; causes areidentified, analyzed, and, where possible, eliminated;resolution and disposition are achieved; status is trackedand reported…”
Intelligent Systems Laboratory
ISO/IEC 12207 process model
Intelligent Systems Laboratory
ISO/IEC 12207 process model● What’s missing?
Intelligent Systems Laboratory
Bugzilla bug tracking
Intelligent Systems Laboratory
Bugzilla’s model● Nothing missing!
Intelligent Systems Laboratory
Sourceforge bug tracking● Assignee - The project administrator to
which a tracker item is assigned. Can bechosen from one of the administratorsregistered in this project.
● Status - This is the (potentially changing)current status of a bug.
● Priority – a nine-level scale.● Category – project-specific.● Group – project-specific.
Intelligent Systems Laboratory
Status?●The online help says:
●Suggests definitions and process
You can set the status to 'Pending' if you arewaiting for a response from the tracker itemauthor. When the author responds the status isautomatically reset to that of 'Open'. Otherwise, ifthe author doesn't respond within an admin-defined amount of time (default is 14 days) thenthe item is given a status of 'Deleted'.
Intelligent Systems Laboratory
What does “support” a process mean?
● Consider the attributes of “category” and“group”– Each project administrator can choose them
based on their own preferences.– Automated bug tracker has no capability to:
• Relate categories or groups to each other, or• To accommodate constraints between particular
categories, groups or values of the other attributes(except for the ability to search the bug list by value). For example, are “category” and “group” orthogonal to each
other, or is a group a sub-category, etc.?
Intelligent Systems Laboratory
Modeling “status”● One educator reports:
– “if the phrases describing subtask status are not defined,different student teams often give different meanings tothe same phrase. Even worse, sometimes, differentmembers in the same team would interpret the samephrase differently.”
● Need to define status phrases indicating which roleand process are involved– e.g., the status “Ready for Review” meant ready to be
reviewed by the quality assurance (QA) role on the team.A better way to name this would be an explicit “Ready forReview by QA” status.
Intelligent Systems Laboratory
Status as a derived concept● Process perspective: item’s status reflects
the process that produced it
Intelligent Systems Laboratory
Summary● Many current tools are incomplete with respect to
organizational needs– Because tools do not address the issues of why
someone is authorized (or not) to make a change to anitem’s status, it is likely for a bug’s status to beinconsistent with a process
● The advantages of using conceptual graphs torepresent the processes– Have the potential to be formally manipulated and
compared– Are well-defined as in ISO/IEC 24707 (Common Logic)– Provide an easily understood visual description of the
process for developers and analysts.
Intelligent Systems Laboratory
Next steps● Empirical validation of the approach.
– compare the performance and experience of softwareengineering project teams
• some teams using conventional approaches and others usingthe pragmatically supported approach.
– incorporate the ideas into an existing production-levelsoftware organization (assuming they already performsufficient metrics on their process)
– perform some linguistic analyses of various naturallanguage used or produced in software development,especially in distributed communities where a largeportion of the interaction takes place through electronicmeans.
Intelligent Systems Laboratory
Further steps● Study the subsequent process of actually
correcting the bugs identified during the process,with duties and responsibilities assigned toappropriate roles– Involves a superset of the same roles involved in
problem tracking.
● Compare different organizations’ processes to findcommon elements, and (likely) missing elements;this is a natural next step.
● Identify where processes actually conflict with eachother.