Memo - Nokia Networkssablime.alcatel-lucent.com/C-SabConcepts.doc · Web viewSubject: Sablime Concurrency Feature Date: 6/25/01. From: Paul Riffle

Subject: Sablime Concurrency Feature Date: 5/10/2023

From: Paul Riffle STC-Sablime Development CB 1d383s +1 614 860-4836

This document is a concept exploration of a major feature enhancement to Sablime. This feature permits Concurrent Development using Sablime. In order to contrast the enhanced behavior with the current behavior, the document refers to this enhanced version as “C-Sab”. It is, in fact, upwardly compatible with the current (v5.2) version of Sablime and is envisioned as the next version of Sablime (most likely v6.0).

Version History: Version 1.0 – distributed week of 1/15/01Version 1.1 – Includes revisions in response to customer review comments and Bell-Labs China team inputVersion 1.2 - Current – Includes changes in response to internal review, including also the “Branch MR” concept

NOTE: Please note that the concepts defined herein have been refined and revised during the subsequent requirements and design phases of C-Sab development. This document is not expected to remain up-to-date.

C-Sab...................................................................................................................................3C-Sab definitions..............................................................................................................4

Delta...........................................................................................................................4Branch........................................................................................................................4Merge.........................................................................................................................4

Current Sablime and SCCS..............................................................................................4C-Sab and extended branching.........................................................................................6

SCCS IDs in C-Sab....................................................................................................7SCCS file compatibility.............................................................................................7

Dealing with the extended branches.................................................................................7Version Ancestry.....................................................................................................10Branch Ordering and Branch Path...........................................................................10

Lucent Proprietary - 1 - Version 1.1

Dependencies...........................................................................................................11Concurrency Models and Branch Types........................................................................12

Public Branches.......................................................................................................12Private Branches......................................................................................................13Temporary Branches................................................................................................13Branches as Generic Level Items............................................................................13Feature Level Concurrency (Public Branching)......................................................13MR or PTS Level Concurrency (Private Branching)..............................................14Minimal Concurrency (Temporary Branching).......................................................15Optional Restrictions on Branches..........................................................................15

C-Sab Database Changes................................................................................................16Branch Relation.......................................................................................................16Source Branch Relation...........................................................................................17Source Merge Relation............................................................................................17MD relation Change: Inheritance............................................................................18ADM Relation Change: Branch Flags.....................................................................19

Merging..........................................................................................................................19Change Reconciliation.............................................................................................19Changed File Management......................................................................................19Merge Process Management....................................................................................20

Command Processing in C-Sab......................................................................................21Common..................................................................................................................21Getversion and sget..................................................................................................21Source......................................................................................................................23

Miscellaneous Topics.....................................................................................................23Multi-Sab.................................................................................................................23SBCS........................................................................................................................24Featuring, labeled deltas and labeled versions........................................................24

Conclusion and Acknowledgments................................................................................24


v2.0

chg23

spcl

main C-Sab Concurrent Sablime

Within Lucent, Sablime is well established as the Configuration Management system of choice for parallel development. It is, however, thought of as being incapable of supporting concurrent development.

In the context of Configuration Management systems, concurrent development is most

often characterized as the ability to have unreserved checkouts of a file, and the ability to do arbitrary branching and deferred merging. The above drawing is a typical illustration of concurrent development’s branching and merging approach as it applies to a particular source file.

Serial development, on the other extreme, generally has a single change stream for each source file, arranged sequentially. When a file is checked out, it is locked against other checkouts, and it contains the net effect of previous changes.

Parallel development expands on the serial methodology by establishing multiple development versions (i.e. branches) of a file, each of which is developed serially. As with Concurrent

development, the files share a common ancestor, but with Parallel development, there is no merge operation.

Sablime implements parallel development by permitting the independent development of files in multiple “generics”.

It is not simply the lack of a merge capability that prevents Sablime from being effective for concurrent development, though. The whole MR process layer that is Sablime is built around the generic as a permanent branch – as a “release” indicator. To build a concurrency capability into Sablime based on casual merging between generics (generics-as-branches) would be a change so fundamental that one might as well start from scratch and design a new system altogether.1

What is needed, and what this proposal supplies, is a method for having arbitrary branching and merging within a generic.

1 Note that this does not preclude the use of the C-Sab merge tool as a aid to the merging of any arbitrary files, included those from different generics.


chg23

main

This concept of Concurrent Sablime – C-Sab, for short – makes greater use of the true branching capabilities of the underlying SCCS2, along with some new and modified Sablime database relations, to provide a version of Sablime that is recognizably Sablime, with its MR progression and Generic concepts intact, and is compatible with current Sablime, but that also permits and facilitates concurrent development.

C-Sab definitionsFirst, let us establish a few of the basic definitions:

DeltaThe set of recorded changes made to a file in order to produce a new version of the file. The change set may be empty. That is, the creation of the new version may

for procedural or administrative differentiation rather than representing actual changed content.

BranchWhenever a particular version of a file is “cloned”, so that multiple tracks of independent development may be followed, this is a branch. In the figure at left, the branch is shown encircled. The term branch also refers to the series of file deltas that follow from the branching operation. Thus the branch “chg23” was created from the “main” branch. Further deltas applied to that creation are considered part of the “chg23” branch. While a branch typically creates two or more new versions, it is sometimes desirable to create a single new branch from an existing one. The “v2.0” branch in the original illustration is an example.

Merge When two branches of a file are brought back together – that is, they have their changes reconciled - the operation is called a merge. In the figure at the right, the merge operation is shown encircled. One of the two branches is seen as receiving the changes that are required to reconcile the two versions.The merge may mark the end of development of the other branch, or development may continue there without regard to the reconciliation changes.

Current Sablime and SCCSAs mentioned before, Sablime uses SCCS for its version control engine. SCCS maintains the version tree of each file, permitting a properly formatted request to extract practically any version of the file that ever existed. In fact, it is quite possible – using the SCCS capability to skip intermediate deltas - to extract versions that probably shouldn’t exist.

2 SCCS - Source Code Control System - is a multi-vendor supported version control engine, distributed with most Unix platforms.


Sample SID in Current Sablime14.1.2.7

Gen

eric

Off

icia

l or M

R B

ranc

hSe

quen

ce

ofc intl

mr

Each delta made to an SCCS controlled file is assigned an SID (SCCS ID) composed of four integers, and formatted Release . Level . Branch . Sequence. SCCS is restrictive about the manner of branching permitted at the “Level” level3. It permits unlimited branching only at the “Branch” level.

In current Sablime, the Release and Level, together, constitute the “generic”. That is, the generic identifier is n.1, where the first digit increments with each release, and the second is always 1. Neither the current Sablime nor the proposed C-Sab makes any real use of the second digit (Level).

Current Sablime uses the third SID component to maintain an “ofc” (official) and an “mr” branch of each file. The value is always either “1” (for ofc) or “2” (for mr). All deltas created against files in the current Sablime are created on the mr branch. Later, when MRs get approved, Sablime incorporates the deltas onto the ofc branch.

The official branch doesn’t have any original deltas, it has only these approved deltas from the mr branch. The illustration at left portrays this branching concept. When MRs get approved – regardless of the order in which they get approved - the associated deltas get incorporated into the ofc branch in the same relative position as they were created on the mr branch.

This “merge” onto the official branch is mechanical, since without original deltas on the ofc branch, there can be no conflicting changes.4

As mentioned before, this “mr” and “ofc” branch concept is implemented as the first two branches on the SCCS “Branch” level. C-Sab retains this usage and uses it as a foundation for

more extensive branching.

Note: While it is convenient to illustrate the approval of an MR as a merge onto the ofc branch, it should be noted that the actual SCCS operation is an “include”. This means that rather than actually having a duplicate of the mr branch delta, the ofc branch just contains an indicator that the particular mr branch delta is to be included when extractions are made from the ofc branch. Those not interested in the details of SCCS operations can ignore this distinction.

3 Once version 2.1 is created, for example, you can no longer create a 1.anything.4 Due to the missing deltas, the official branch version may not be logically correct or meaningful.


chg23

spcl

ofc

v2.0

mr

C-Sab and extended branchingTo illustrate how C-Sab retains the current usage of the “mr” and “ofc” branch, but adds “extended” branching, we start by taking the original drawing from the title page and flopping all the branches onto one side.

One should be able to see that on this (admittedly less attractive) drawing the delta relationships remain the same. Now if we re-name the “main” branch to “mr” and add an official branch, the C-Sab branching strategy becomes apparent.

These principles describe the relation between the traditional branches of Sablime and the extended branches of C-Sab:

1) Extended branches can be created from the “mr” branch, or from another extended branch, but not from the ofc branch.

2) MR approvals cause “mr” branch deltas to be incorporated onto the official branch (same as always).


intl

original “flopped”

chg23

spcl

ofc

v2.0

mr

1.1

1.3

1.4

1.5

2.2

2.1

2.3

2.4

2.5

3.1

3.2

4.2

4.1

1.2

5.1

5.2

3) Deltas present on extended branches do not get incorporated onto the official branch upon MR approval.

4) Merges may happen from a branch onto another extended branch or onto the mr branch. The only merges that are permitted onto the ofc branch are those performed by the “approve” command (i.e. directly from the “mr” branch).

SCCS IDs in C-SabAs mentioned before, Sablime uses the first two components of the SID to specify the generic. The third component – the Branch ID - Sablime already uses for ofc and mr branch (ID 1 and 2), and the fourth – the Sequence ID - for the delta number on that branch. The C-Sab extended branches make use of those Branch IDs beyond number 2 and, of course, the Sequence IDs.

The illustration at the right has the Branch and Sequence IDs added. Recall that this whole delta tree is within a generic, so the first two components of the SID are constant.5

SCCS file compatibilityThe combined effect of this branching strategy and of the principles listed above is that C-Sab retains full compatibility with existing SCCS files. If extended branching is not used, the resultant “s.” files would be indistinguishable from those created with current Sablime. 6

Furthermore, the strategy and principles combine to permit the Sablime commands themselves to be compatible. Commands that address the mr and ofc branches continue to behave the same and produce the same results.

Dealing with the extended branchesThe C-Sab branching scheme builds upon the existing ofc and mr branches. Similarly, the C-Sab source commands extend the existing “br” keyword to permit selection of versions from the extended branches.

Just as the latest version from the “mr” branch is selected by “br=mr”, so is the latest version of an extended branch selected by “br=<branch_name>”. The following table is similar to the one found in the Sablime documentation for the getversion command, but expanded to cover the new keyword usage. The behavior of the sget command is similar

5 The illustrations show the mr branch coming off the ofc branch at 1.1. In reality, both of these branches come from the same point in the next higher SCCS level (the “Level”).6 C-Sab will have a configuration parameter to disallow extended branches, and thereby force persistent compatibility with current Sablime.


to getversion, except that there are no “MRs for File Selection”; in “sget” the files are directly specified.


Branch

MRs for File Selection

MRs for Additional Changes Retrieves

ofc

None specified

None specified Latest ofc branch version of all files

Unapproved MRsLatest ofc branch version of all files- with -additional mr branch deltas from MRs for Additional Changes

Approved MRs

None specified Latest ofc branch version of files touched by MRs for File Selection

Unapproved MRs

Latest ofc branch version of files touched by MRs for File Selection- with -additional mr branch deltas from MRs for Additional Changes

Unapproved MRs

None specifiedLatest ofc branch version of files touched by MRs for File Selection- with -additional mr branch deltas from MRs for File Selection

Unapproved MRs

Latest ofc branch version of files touched by MRs for File Selection- with -additional mr branch deltas from MRs for File Selection and from MRs for Additional Changes

mr

None specified None specified Latest mr branch version of all files

Approved and/or Unapproved MRs

None specifiedInitial mr branch version of files touched by MRs for File Selection- with -additional mr branch deltas from MRs for File Selection


Initial mr branch version of files touched by MRs for File Selection- with -additional mr branch deltas from MRs for File Selection and from MRs for Additional Changes

<branch name>

None specified None specified Latest <branch name> version of all files, or latest mr branch version

for files without <branch name> deltas7


None specified

Initial mr branch version of files touched by MRs for File Selection -with-additional <branch name> or mr branch deltas from MRs for File Selection


Initial mr branch version of files touched by MRs for File Selection -with-additional <branch name> or mr branch deltas from MRs for File Selection and from MRs for Additional Changes

The last two table rows permit some ambiguity in the delta selection. Since an MR may have touched a file on both the mr branch and on an extended branch, or it may have touched the file both before and after it branched off the mr branch, it is unclear which of these deltas belong in the extracted file.

7 It may be desirable to permit an option here that would limit the file set to those that have deltas on the specified branch.


C-Sab addresses this issue by using the concept of Version Ancestry.


chg23

spcl

ofc

v2.0

mr

1.1

1.3

1.4

1.5

2.2

2.1

2.3

2.4

2.5

3.1

3.2

4.2

4.1

1.2

5.1

5.2

Version AncestryVersion ancestry is defined as the set of deltas in the path from the latest delta on a branch to the mr branch root. Merge arrows are ignored when tracing the ancestry. The illustration at right displays the version ancestry of the “v2.0” branch of our sample file.

Whenever a file is extracted from an extended branch, the delta selections can only include those present in the version ancestry.

Returning to the ambiguity from the table, we can now say that even though the same MR may have been responsible for deltas 2.1, 2.3, 4.2 and 5.2 (for example), if “br=v2.0” is specified, only 5.2 and 2.1 should show up in the extracted file (along with, of course, any other deltas from the version ancestry that are selected by the MR specifications).

The “edget” and “edput” commands, which in traditional Sablime do not permit the “br” keyword, will now need to do so. In traditional Sablime, a file extracted for edit always came from the “mr” branch. In C-Sab, though, one can edit a file version from any of the branches. Like always, though, edget will only permit the extraction of the latest version on the specified branch. It also (as before) does not accept “mr” or “umr” arguments as a means to selectively include deltas.

Branch Ordering and Branch PathWith the potential for multiple versions of a file within a generic, and for files with multiple branches, we may need to go beyond specifying one branch for the extraction. We may want to select the “v2.2” branch version of a file if it has one, but the “v2.1” version if it does not.

Within C-Sab, for the source management commands that accept “br=<branch>” arguments, the <branch> may be a colon-separated series of branches. An example might be “v2.2:v2.1”. For each file that a C-Sab source command retrieves, it attempts to extract the version of the file from each branch in the series (reading left to right). Once it finds a version, it looks no further. (See below for exceptions with regard to edget and edput). If the variable BPATH is defined in the runtime environment, C-Sab will use its value as the default for the “br” value.


In all cases, the mr branch is implied as the final branch in the series. That is, if the file has no presence on any of the specified branches, then the mr branch version will be selected.8 The ofc branch may also be in the BPATH, but this may generate an error in circumstances (“edget” for example) where the “ofc” branch is not permitted as a “br=<branch>” argument.

Edget and Edput As mentioned earlier, since there are now multiple editable versions of a file within a generic, edget and edput in C-Sab accept br=<branch> arguments. The handling of the br argument is somewhat different for these commands:

The “edget” will, by default, only consider the first component of the “br” value (as derived from the br=<branch> or from the BPATH). As elsewhere, the “br” value defaults to “mr”. Edget will only consider the extended BPATH when a command line flag (bpath=y) forces it. This is intended to reduce the likelihood of editing the wrong version.

If the determined branch is not yet created for that file, the edget command might create it, depending of the settings of the “branch flags” (defined later), and depending on permissions (also defined later). The user may be required to specify the “parent” branch and delta.

Note that since “mr” is implied as the final component of BPATH, using the “bpath=y” argument precludes the creation of new branches. There will always be an existing branch from which to edget (either that or an error if, for example, “ofc” was placed in the BPATH).

Further development may continue on the parent branch without regard to and without affecting the newly created branch.

The edput command uses the same logic as edget for determining the value for “br”. That determined branch must, of course, turn out to be one on which the current user and MR have done an “edget”, or else an error will be generated. The unedget and unedput commands have obviously similar behavior.

Dependencies No discussion of file extraction with Sablime would be complete without considering dependencies. Sablime supports three kinds of dependencies: 1) automatic file-level, whereby any MR that changes the same file as an earlier (unapproved) MR is made dependant on that earlier MR; 2) automatic line-level whereby any MR that changes the same lines as an earlier (unapproved) MR is declared dependant on that earlier MR; and 3) manual (logical), where an MR is manually declared to be dependant on some other MR.

8 As mentioned before, it may be desirable to permit this behavior to be overridden with a command option (bpath=n, for example).


Certain Sablime commands, chiefly getversion, sget, and approve, will not permit operations on dependant MRs unless their depended-upon MRs are also in the set of MRs to be considered.

Introducing independent development branches to files adds a new dimension to this issue. An MR may make changes on one branch of a file, while another MR may make changes on an unrelated branch. Or that second MR may have made changes on the parent branch of the current branch.

Dependency calculations, then, need to be re-evaluated throughout the C-Sab command set to properly consider the branches. The basic difference is that the candidate MR set for depended upon MRs is no longer the full set of unapproved MRs that touched the file, it is only the set of unapproved MRs that created the deltas in the version ancestry.

Physical dependencies (i.e. file or line level) are recorded into the database when a file is “edput”. C-Sab will take into account the version ancestry and only record those physical dependencies where the potential depended upon MR is present in the ancestry.

File extraction commands will also need to consider whether a physical dependency listed in the PDEP applies to the version of the file being extracted. They can do this by verifying that the delta (listed in the PDEP) that made the dependent change is in the version ancestry.

A sample “getversion” procedure is presented later, after the description of the database changes used for such dependency and version ancestry calculation in C-Sab.

Concurrency Models and Branch TypesUp until now, this document has concentrated on defining branching and merging and how it relates to SCCS file structure. File extraction commands have been described in order to illustrate how C-Sab would choose which of the several versions of a file would be extracted.

Of course, there must also be a process for using the branches. Creating them, merging them, putting deltas on them, etc.C-Sab presents three different concurrency usage models, corresponding to three branch types: public, private, and temporary. The types have different handling requirements and differing usage restrictions. Projects can use each of the types (or not) as they prefer depending on their concurrency preferences.

Public BranchesPublic branches are of the type used for the most liberal concurrency mode pictured in the diagrams. They have any number of deltas, created by any number of users with any number of MRs.


Public Branching is for feature level concurrency, where extended development and multiple MRs and developers may be involved, and where other branches may need to be created from and merged to this branch.

Private BranchesPrivate branching is used for a more restricted concurrency process. Private branches contain multiple deltas, all created by the same MR.

Private Branching is for PTS or MR level concurrency, where a developer may perform regular check-ins of his/her work while still keeping the activity apart from the mainstream branch(es).

Temporary BranchesTemporary Branches are used in the most restricted concurrency model, what might be called “minimal concurrency”.

Temporary branching permits a user to work with a file is already out for edit by another user and/or MR. The branch only exists for the duration of the edit. When the user attempts to check the file back in, the system will force a merge operation if other check-ins have happened since the check-out.

Branches as Generic Level ItemsThe actual Sablime Database changes to support C-Sab are described later. In order to properly describe the usage models, though, it is necessary to introduce the fact that C-Sab maintains a generic level record of each branch name.

This record permits branches to be managed independently of the files within which the branch deltas are made. Branch “v5.2”, for example, can be thought of as the complete collection of files in which that branch name has been used.

This ability to manage branches independently of the files gives us the tools to create the usage models supporting the three types of concurrency.

Feature Level Concurrency (Public Branching)There are few restrictions on Public Branches. They can be created from any other public branch or from the “mr” branch. They may have other branches created from them. Any number of users or MRs may apply deltas onto a public branch.

Public branches are created using the (new) “branch” command. The branch command can be executed by the Generic Administrator or anyone else designated by the Command Permissions feature. The branch command requires an MR to be specified as the “branch MR” for that branch.

Note that this “branch creation” only creates the generic-wide record of the branch. It does nothing to any source files. The Assigned Developer (AD) can then proceed to use


“unmerged” delta

the branch MR or some other MR to “edget” files. Only then does the branch become associated with the individual source files.

The “branch MR” is also the MR that must be used to authorize any merging from that branch. Branching and merging can only be done when the branch MR is in the assigned state. Note: This stipulation is currently under review. It may be desirable to allow other MRs assigned to same developer to perform merge. That way, those other MRs can get submitted and further promoted, while there continues to be work performed on the named branch. Eventually, when the original Branch MR is submitted, then work on the branch is discontinued.

Notice that the effect of these process restriction is that one can manage a branch generic-wide by managing an individual MR. That is, if MR0001 is the “branch MR” for branch “v5.2”, then submitting that MR has the effect of declaring the work on branch “v5.2” as completed.

Analogous to doing multiple edputs prior to submitting, you can merge a branch repeatedly before you eventually submit the branch MR.

It is important to point out again that public branches also permit other MRs to be used to make deltas on the branch. While there is certainly an implication that merging a branch is intended to bring all that branch’s changes onto the target branch, there is no enforcement, and submitting the branch MR does not have any effect on the status of any other MRs that made changes on the branch.

The submit command will issue a warning if a user tries to submit a branch MR that has unmerged deltas (that is, deltas that were created after the most recent merge operation).Once the branch MR is submitted, no further deltas are permitted to be made on that branch.

Since feature level concurrency implies an extended duration, and may in fact be used for permanent separation of parallel-developed features, public branches have an indefinite lifespan, meaning that they exist until explicitly removed by the source administrator.

MR or PTS Level Concurrency (Private Branching)With private branching, the branch MR is the only MR permitted to make deltas on the branch. Private branches are intended to permit a user to checkpoint his or her code by edput’ing on a regular basis, while still maintaining isolation from the main stream.

Like public branches, the private branch name exists independently of the individual files, so it can be used for development against multiple files. Within each file, though, the merging of a private branch causes C-Sab to remove the individual deltas created on the branch.


If it is desirable to continue with further deltas, then the AD can simply branch off again from the merge delta.

The submit command will not permit a private branch “branch MR” to be submitted if there are unmerged deltas on the branch. Once a private branch MR is submitted, the name can be re-used with another MR.

Since only the branch MR is permitted to make deltas on a private branch, it follows that the only branches that can be created from or merged to a private branch are those created with the same branch MR. Public branches may not be created from private branches.

Minimal Concurrency (Temporary Branching)Minimal concurrency allows users to check out a file that is already checked out by another PTS/MR. When the user attempts to perform a check-in, C-Sab will see whether other check-ins have happened since the check-out. If so, a merge will be required.

On the other hand, if there have not been other check-ins, the temporary branch check-in will succeed, and the original check-out will be forced to merge upon check-in.

C-Sab will need to adjust the SID assignments in the “p.” file if a temporary branch gets checked back in while the original check-out is still in effect. Likewise, it will need to adjust the original check-out’s MD record.

It is interesting to note that temporary branches do not even exist within the SCCS files. The actual delta is always made on the branch from which the file was checked out. The “temporary” branch only exists as a reserved SID in SCCS, which then is discarded when the check-in is accomplished.

Temporary branches do not have a “branch MR” or a generic-wide branch record, and if temporary branches are enabled, they can be created with any MR on the “mr” or any public branch.


Mr0001

Mr0002

priv

ate

bran

ch

mr b

ranc

h

priv

ate

bran

ch

mr b

ranc

h

Mr0003

Mr0003

Mr0003

mr b

ranc

h

Mr0001

Mr0002

Mr0003

Private Branch during development merging after merging

Optional Restrictions on BranchesWhen a branch is created (with the “branch” command), there are some further restrictions that can be invoked to allow tighter control of their usage.

Public branches may have an assigned “owner”. Only this PTS (or PTS group) may create deltas on the branch.

Private branches may also be assigned an owner. For private branches, the owner is permitted to change the branch MR name. This permits a private branch name to be used repeatedly by the same PTS, without needing to involve the GA.

Public branches may have an assigned “authorized MR”. Only that MR (or members of that MR group) may make deltas on the public branch. (This may exclude the “branch MR”, by the way).

For private branches, the “authorized MR” is not used (only the “branch MR” can make deltas).

The branch command permits public branches to be made private, if no deltas have been made by other than the branch MR. It also permits private branches to be made public.

C-Sab Database ChangesReaders familiar with the existing Sablime database will have noticed that there are a couple of new entities in C-Sab that will need to be stored somewhere. C-Sab introduces three new relations to the Sablime Active Database: Branch relation (BR); the Source Branch Relation (SB), and the Source Merge Relation (SM).

Note: Database relation specifics are suggestions only, and are here only for concept completeness. Performance or other factors may cause these details to be implemented differently.

Branch RelationThis relation contains the general information about the extended branches.

bname g brmr browner authmr brsite brabst brtype

These are bname for branch name, g for generic, brmr for the Branch MR, browner for branch owner, authmr for authorized MR, brsite for site owner (future use), brabst for branch abstract, and brtype for branch type.

This implies (correctly) that the branch name (bname) must be unique within the generic (g). These two are the relation keys. Bname is the key for hashing the BR relation tuples.

The brmr is the name of the MR used to create the branch; only that MR may be used to merge the branch. The browner field can be used to indicate the owner of the branch. On a public branch, only the owner of the branch is permitted to put deltas onto the


bug74

intl

ofc

v2.0

mr

1.1

1.3

1.4

1.5

2.2

2.1

2.3

2.4

2.5

3.1

3.2

4.2

4.1

1.2

5.1

5.2

branch. If left empty, then all PTS ID are permitted. On a private branch, the owner is permitted to change the Branch MR (brmr).The authmr field is only used for public branches, and can specify that only the indicated MR (or MR group) is permitted to modify the branch. Like browner, it can be left empty to avoid this restriction. The brabst is a textual description of the purpose of the branch. The brtype indicates whether the branch is of type public, private, or temporary.

A BR record is created whenever the branch command is used to create a public or private branch. A BR record is automatically created for temporary branches, although it only contains information in the bname, g, brmr, and brtype fields.

Source Branch RelationThis relation contains extended branch information specific to its usage for a particular source file.

sfile dir g bname bid pid sbstat

These are sfile for source file, dir for directory, g for generic, bname for branch name, bid for branch ID, pid for parent ID, and sbstat for busy/free indicator.

The sbstat field serves the same purpose as the gsstat in the GS relation: that is, to indicate whether or not this particular version of the file has already been checked out. C-Sab does not permit multiple checkouts of the same branch of the same file within the same generic.

Source Merge RelationThis relation contains the record of merge actions.

sfile dir g tid fid mr

These are sfile, dir, and g as in the SB relation, and tid for “to ID”; fid for “from ID”, and mr for the mr name that authorized the merge.

This storage of the merge to-and-from delta SID permits tools to calculate the complete version tree of a file, including the merge arrows.

Referring once again to the illustration, we can build a complete BR , SB, and SM relation for the sample file.

bname g browner brmr brsite brabstbug74 g1.0 Some text

intl g1.0 A different versionv2.0 g1.0 pmr My branch

sfile dir g bid bname pid sbstat


SB

BR

bug74

intl

ofc

v2.0

mr

1.1

1.3

1.4

1.5

2.2

2.1

2.3

2.4

2.5

3.1

3.2

4.2

4.1

1.2

5.1

5.2

src.c mydir g1.0 1.1.3 bug74 1.1.2.2 freesrc.c mydir g1.0 1.1.4 intl 1.1.2.3 freesrc.c mydir g1.0 1.1.5 V2.0 1.1.3.2 free

sfile dir g tid fid mrsrc.c mydir g1.0 1.1.2.5 1.1.4.2 mr000001src.c mydir g1.0 1.1.2.4 1.1.3.2 mr000002

Recall that the Branch and Sequence IDs displayed in the illustration map to the third and fourth digits of the SCCS ID. The first two digits specify the generic. Thus, the “generic” field in the SB and SM relation is a bit redundant, since it is apparent in the “bid” and other “ids”, but it is likely to be useful here for ease of indexing.

MD relation Change: InheritanceC-Sab adds one field to the MD relation to record the “merge state” of the delta. The value is either y or n, depending on whether the delta has been merged to the MR branch. More precisely, it is set to yes (y) if

1) the delta is on the mr branch itself--or--

2) the delta’s branch has been merged to the mr branch directly, or onto a branch that has itself been merged onto the mr branch (directly or indirectly).

Whenever a branch receives a merge, it is said to have inherited the deltas created on the parent branch prior to the merge operation. Note that deltas created on the parent branch

after the merge are not included in the inheritance.

We can restate the rule above to say that the merge state is set to yes if the delta is on the mr branch or has been inherited by the mr branch.

In the figure, all the deltas except for 5.1 and 5.2 have been inherited by the mr branch.

If there were a merge from 5.1 to 4.2, for example, then 5.1 would also have been inherited by the mr branch (by virtue of the

subsequent merge of 4.2 to 2.5), but 5.2 would not.

This concept of mr branch inheritance permits C-Sab to warn the user if they attempt to submit or approve MRs that have deltas that appear not to be represented on the mr branch.


SM

Change Reconciliation

Changed File Management

Merge Process Management

Note that a delta’s inheritance to the mr branch does not necessarily mean that the change is logically present on the mr branch. Any intervening delta, or the merge operation itself, may have undone or negated the original change.

ADM Relation Change: Branch FlagsC-Sab adds a new field to the ADM relation to hold the branch flags. There will be four flags, each with y or n as possible values. The four flags are: public – whether or not public branches are permitted private – whether or not private branches are permitted temporary – whether or not temporary branches are permitted mrmerge – whether or not a “special” MR must be used for the merge of public

branches. If the first three are all “n”, then all C-Sab functionality is disabled.

MergingThe overall merge process may be conveniently divided into three components: The inner-most process – Change Reconciliation – is the process of comparing the two or more changed files and selecting which changes are to be brought through to the reconciled (merged) version. Outside of that process is the Changed File Management process which extracts the files from Sablime for the benefit of the reconciliation process, and stores the resultant merged file back to Sablime. The outermost process is the Configuration Management process that permits, controls, and records the fact of that merge operation

Change ReconciliationThis component accomplishes the actual merging of the files, and while it is crucial to the effectiveness of the system, it is an almost wholly self-contained problem whose net result is a delta for the receiving branch. This document leaves the evaluation and selection of this physical merging process and/or tool for an independent study.

Changed File ManagementThis second layer of the merge process is a fairly mechanical operation: running sget on the sending branch(es) and edget on the receiving branch; feeding the files to the Change Reconciliation process; accepting the changed file back from that process; and running edput to store that updated file back onto the changed branch.

Much of this operation will be driven by the user interface of the new command (presumably named merge) that would be invoked to perform a merge.


Note: Some revisions are still required in this and following sections to account for the Branch MR concept and other v1.2 changes.

Merge Process ManagementThe outermost, and in the author’s opinion, most interesting aspect of merging is the Merge Process Management, i.e. the Configuration Management of the merge.

Some of the questions involved are these:3) Must a merge operation be authorized by an MR?4) May that MR be the same MR that applied changes to the branches, or must it

exist for the merge operation only?5) Does the merge operation require its own role?6) Does a merged MR have a different state, or some other “query-able”

attribute.7) Can an MR with changes that have not been inherited by the mr branch be

submitted? Promoted? Approved?

Must a merge operation be authorized by an MR? Some systems that support concurrent development treat the merge operation as if it were of no engineering consequence – strictly the administrative joining of two or more development streams. C-Sab believes that any merge where an overlap conflict is resolved requires technical and engineering choices to be made. In fact, due to logical conflicts, many non-overlapping changes also require such choices.In any case, though, the result of a merge is a change to the code base. This change may or may not have been done correctly and should be subject to the same review process that applies to other code changes. C-Sab, therefore, requires that an MR be used to authorize the merge operation. The MR must be accepted and assigned in the generic that contains the receiving file/branch of the merge operation.

May that MR be the same MR that applied changes to the branches, or must it exist for the merge operation only? For private and temporary branches, the MR that makes the deltas must also make the merge. However, for the public branches, where a branch may contain multiple (maybe unrelated) changes, made by multiple MRs and users, it might be desirable to require that a separate MR – one not responsible for any of the individual changes – authorize the merge.

C-Sab, then, has a configuration parameter in the ADM relation that declares whether or not an MR used for merging can be used for other purposes as well. See ADM Relation Change: Branch Flags on page 20.

Does the merge operation require its own role? The “branch owner” field of the SB can be used to restrict the ability of PTS IDs to write to a branch (which would be required in order to merge). Furthermore, the “merge” command can be restricted by using the Command Permissions features of Sablime. At the time of this proposal, then, C-Sab does not recommend any special role or permissions feature targeting the merge operation.


Does a merged MR have a different state, or some other “query-able” attribute? C-Sab considers the branch as the item that is merged, not the MR. It does record, though, whether a not a delta has been “inherited” by the mr branch. See MD relation Change: Inheritance on page 19.

C-Sab will supply a merge-status utility to report whether a particular MRG has file deltas that have not been inherited by the mr branch.

Can an MR with deltas that have not been inherited by the mr branch be submitted? Promoted? Approved? C-Sab will warn the user if an MR uninherited changes. Each of these operations will be permitted if the user either responds to the warning with a “go-ahead”, or if they override the warning from the command line.

Note that the promotion or approval of an MR doesn’t affect the ability to make further modification to the file and public branch with other, assigned-state, MRs.

Command Processing in C-SabCertain commands in Sablime will require substantial logic enhancements to support C-Sab. Some of these are mentioned below.

CommonThe common command permits two files that were previously not common to be made so if their current (latest mr branch) version is identical. C-Sab retains this concept, but also must consider the extended branches.

C-Sab declares that the latest version on all extended branches must also be identical, and if not, the branch is “frozen”, meaning that further changes are prohibited. This “frozen” indicator can be stored as a secondary value in the SB relation, sbstat field (free-frozen or busy-frozen). Furthermore, since there may have been any number of named extended branches on one or more of the common candidate files, newly created branches can only be given names that are not already in use. A further implication of this is that the now-common files could have different Branch IDs on new extended branches.

After a file is successfully declared to be common (or in the more normal case where the file has been common from the start), the operations are consistent with current Sablime practice, in that all deltas including those on branches, are made in each of the common generics.

Getversion and sgetThe getversion command uses its various option settings (including MR state and list specifications) to calculate the file set to be retrieved. The sget command starts with the files already specified.

In both cases, though, the command may be required to bring in additional files and/or file deltas based on discovered dependencies. In current Sablime, this is a straightforward


matter of checking the DEP and PDEP relations to see if the file or MR in question is dependent. If so, additional MRs (and thus files in the case of getversion) may need to be considered, or additional deltas may need to be included for the existing file set.

When the “br” setting contains extended branches, though, the command must also determine if the dependent delta is within the version ancestry. If it is not, then the command behaves as if the PDEP record did not exist. Note that this only applies to physical dependencies. All logical dependencies are honored, since these exist among MRs without regard to the associated file deltas.

Version Ancestry Calculation ProcedureThe following procedure is a sample of how a version ancestry might be determined:

Note that this outline is intended to illustrate the calculation of version ancestry in the Sablime command set. It is not necessarily the processes that the final C-Sab implementation should use. That is left to the discretion and expertise of the implementers.

1) For each component of “br”1. Find record in SB relation where sfile;dir;g;bname is current sfile;dir;g;<branch component>2. If found set BID to SB/bid and break out of loop2) If no SB record was found, set BID to n.n.2 (where n.n is generic ID from G relation)3) Collect records from the MD relation where the MD/sid is from file’s branch (MD/sid is the

same first three n.n.n as BID)4) Do this loop1. If BID is n.n.2, break from loop 2. Set PID to SB/pid3. Find record in SB for current file;dir where SB/bid is same n.n.n as PID4. Set BID to SB/bid5. Collect records from MD where MD/sid is same first three n.n.n as BID and where fourth

(n.n.n.n) is less than or equal to fourth of PID

Getversion Dependency HandlingGetversion starts with a specified or derived MR and UMR list. A process for handling dependencies is outlined below:

Note that the following outline does not attempt to describe all the various behaviors that can be imposed on getversion by option settings. It is an exercise by the author to explore some of the less obvious implications of extended branching.

1) Generate MRLIST and UMRLIST based on supplied parameters.2) Generate APPMRS – The list of approved MRs in this generic.3) Generate DEPMRS – A list of those MRs from MRLIST and UMRLIST that are dependent4) Generate DEPUMRS – The list of MRs depended upon by DEPMRS5) Generate L-MISSING – The DEP records of this generic for DEPMRS where DEP/logical is

“y” and DEP/dep is not in MRLIST, UMRLIST, or APPMRS.6) Generate P-MISSING – The PDEP records of this generic for DEPMRS where PDEP/dep is

not in MRLIST, UMRLIST, or APPMRS.7) Generate FILELIST – The list of files touched by MRLIST8) Process MISSINGS 9) While BADLIST is non-empty1. Permit editing of list as with current Sablime


2. Generate new L-MISSING and P-MISSING with (only) newly specified MR/UMRs as DEP or PDEP/mr

3. Process MISSINGS 10) For each file in FILELIST1. Generate version ancestry for this file (or remember it if stored)2. For each MD record in version ancestry

1. If MD/mr is in MRLIST or UMRLIST or APPMRS, mark delta as included 3. Fetch file using include list derived from marked deltas

Process MISSINGS1) For each MR in L-MISSING1. Update Lists (MR) 2) For each record in P-MISSING1. If PDEP/dep is in P-FOUND, continue with next P-MISSING2. If PDEP/dir;file is not in FILELIST, continue with next P-MISSING3. Determine file ancestry of PDEP/dir;file4. If PDEP/sid is not in version ancestry, continue with next P-MISSING5. Update Lists (PDEP/dep)

Update Lists1) If incldep is “inumrs”, Update UMRLIST 2) If incldep is “inmrs”, Update MRLIST 3) If incldep is “no”, add MR to BADLIST

Update UMRLIST1) Add MR to UMRLIST, add MR to P-FOUND2) For any DEP records of this generic where DEP/mr is this MR and DEP/logical is “y” and

DEP/dep is not in MRLIST, UMRLIST, or APPMRS1. Update Lists (DEP/dep)

Update MRLIST Add MR to MRLIST, add MR to P-FOUND Update FILELIST with any new files touched by this MR Append to P-MISSING any PDEP records of this generic where PDEP/mr is MR and PDEP/dep is not

in MRLIST, UMRLIST, or APPMRS For any DEP records of this generic where DEP/mr is this MR and DEP/logical is “y” and DEP/dep is

not in MRLIST, UMRLIST or APPMRS1. Update Lists (DEP/dep)

In its basic form, sget simply fetches the latest version of the files in the specified file list from the specified branch. If “mr=<mrlist>” is specified as an option though (the equivalent of getversion’s UMR list), then sget must consider dependencies and file ancestry as well.

SourceThe source command should be enhanced to perform branch maintenance operations. It should, for example, be able to rename a branch, or delete a branch.


Miscellaneous Topics

Multi-SabThe infrastructure put in place by C-Sab greatly facilitates the development of a potential Multiple Site Sablime. Granted that the following is somewhat off-topic, but is documented here for lack of a better place at the moment.

Given the following rules and using C-Sab infrastructure, Sablime multiple site development can proceed with extremely few possibilities for database conflicts. Thus the synchronization of databases would be an almost strictly mechanical operation.

Multi-Sab Usage Rules: MR name prefixes are unique to the site MRs are owned by a site (ownership can be moved) MRGs are owned by a site (ownership can be moved) Commands that create or close MRGs only permitted at the ownership site Commands that preclude the creation or closing of MRGs only permitted at the ownership site C-Sab branch name prefix is unique to a site C-Sab branches are owned by a site (ownership can be moved) Sites all use a common Time Zone setting

SBCSIndications are that the C-Sab concept can be carried over to SBCS files as well. That is, SBCS supports the arbitrary branching used in C-Sab. Merges, of course, are thoroughly different when the files involved are “binary”. While C-Sab may recommend or support the use of some utilities to merge common formatted binary files (MS-Word, Framemaker) if such things can be found, C-Sab is not – in general – responsible for such binary file merges.

Featuring, labeled deltas and labeled versions.The C-Sab branch mechanism lends itself to a featuring-like capability. Since there would be multiple active versions of a file within a generic, and the version to be used for a particular operation is selected by BPATH, it facilitates the maintenance and usage of multiple variations of a product within the same generic.

Additionally, since C-Sab already evaluates each delta on a file version’s ancestry, it would be a fairly short step to include the capability of having delta labels [stored in MD] that could cause deltas to be included/excluded based on external input.

Similar labels might be used to tag a delta [and by implication, the version ancestry to that point], as a labeled version, and file selection might be enhanced to consider that.

Labeled versions, in particular, present some other complications, though. If a version label can be used to extract a file, then is it assumed that the use of a particular label with a particular file will always produce the identical extraction? Even if some of the MRs in the version ancestry have been approved, for example, since the labeling?

Labeled versions and deltas would also, most likely, require further additions or changes to the Active Database.


Conclusion and AcknowledgmentsThis document presented a proposal for an upgrade to Sablime that would maintain current Sablime look-and-feel while permitting concurrent development. The document goes into some detail in order to address the first level technical issues, and to explore some of the areas most affected. It should not be seen, though, as a full requirements or design document for the development of C-Sab.

There have been contributions to this document from many of the Sablime team, both in the USA and in Bell Labs, China for which the author is thankful. Special thanks, though, goes to John Snively, who supplied or helped refine many of the concepts, and who – more than once – helped me get past seemingly hopeless roadblocks.


publ

ic b

ranc

h

“unmerged” delta

Documents

Memo - Nokia Networkssablime.alcatel-lucent.com/C-SabConcepts.doc · Web viewSubject: Sablime Concurrency Feature Date: 6/25/01. From: Paul Riffle