Getting Information Out Of Subversion

We will begin in a few moments

Getting Information Out Of Subversion

3

Outline

•Log information

•Property information

▫ Log and Properties demonstration

•Difference information

•Using Project and User information

•Using Hook scripts to log information

▫ Hook script demonstration

This training is done with the latest TortoiseSVN Version

http://www.wandisco.com/subversion/os/downloads

Extracting information out of the repository

Use the log command

Use a hook script

Use the Subversion API’s

Third party tools

Repository

Subversion API

6

svn_client_proplist3

Output from Log Command

7

Revision: 29Author: MikeDate: 3:53:04 PM, Sunday, May 08, 2011Message:QA Level:Change User Doc:----Modified : /BankDocSystem/trunk/auditbuild.aspModified : /BankDocSystem/trunk/borrowerdeleteconfirm.asp

Revision: 28Author: MikeDate: 10:44:41 AM, Wednesday, May 04, 2011Message:QA Level:unitChange User Doc:No----Modified : /BankDocSystem/trunkModified : /BankDocSystem/trunk/Stored ProceduresModified : /BankDocSystem/trunk/imagesModified : /BankDocSystem/trunk/includeModified : /BankDocSystem/trunk/include/javascriptModified : /BankDocSystem/trunk/reports

Revision: 27Author: MikeDate: 10:42:08 AM, Wednesday, May 04, 2011Message:added copyright and owner----Modified : /BankDocSystem/trunkModified : /BankDocSystem/trunk/Stored ProceduresModified : /BankDocSystem/trunk/images

Revision: 29Author: MikeDate: 3:53:04 PM, Sunday, May 08, 2011Message:QA Level:Change User Doc:----Modified : /BankDocSystem/trunk/auditbuild.aspModified : /BankDocSystem/trunk/borrowerdeleteconfirm.asp

Revision: 28Author: MikeDate: 10:44:41 AM, Wednesday, May 04, 2011Message:QA Level:unitChange User Doc:No----Modified : /BankDocSystem/trunkModified : /BankDocSystem/trunk/Stored ProceduresModified : /BankDocSystem/trunk/imagesModified : /BankDocSystem/trunk/includeModified : /BankDocSystem/trunk/include/javascriptModified : /BankDocSystem/trunk/reports

Revision: 27Author: MikeDate: 10:42:08 AM, Wednesday, May 04, 2011Message:added copyright and owner----Modified : /BankDocSystem/trunkModified : /BankDocSystem/trunk/Stored ProceduresModified : /BankDocSystem/trunk/images

Cntrl C

Copy and place in a text file

C:\source\BankDocSystem>svn log -v------------------------------------------------------------------------r28 | Mike | 2011-05-04 10:44:41 -0600 (Wed, 04 May 2011) | 2 linesChanged paths: M /BankDocSystem/trunk M /BankDocSystem/trunk/Stored Procedures M /BankDocSystem/trunk/images M /BankDocSystem/trunk/include M /BankDocSystem/trunk/include/javascript M /BankDocSystem/trunk/reports

QA Level:unitChange User Doc:No------------------------------------------------------------------------r27 | Mike | 2011-05-04 10:42:08 -0600 (Wed, 04 May 2011) | 1 lineChanged paths: M /BankDocSystem/trunk M /BankDocSystem/trunk/Stored Procedures M /BankDocSystem/trunk/images M /BankDocSystem/trunk/include M /BankDocSystem/trunk/include/javascript M /BankDocSystem/trunk/reports

added copyright and owner------------------------------------------------------------------------r26 | Mike | 2011-05-04 10:40:26 -0600 (Wed, 04 May 2011) | 1 lineChanged paths: M /BankDocSystem/trunk M /BankDocSystem/trunk/Stored Procedures M /BankDocSystem/trunk/images M /BankDocSystem/trunk/include M /BankDocSystem/trunk/include/javascript A /BankDocSystem/trunk/privacyPolicy.asp M /BankDocSystem/trunk/reports

added a new file------------------------------------------------------------------------r25 | Mike | 2011-03-13 07:00:04 -0600 (Sun, 13 Mar 2011) | 1 lineChanged paths: M /BankDocSystem/trunk/exceptioncommentedit.asp

Fixed Bug #284------------------------------------------------------------------------r24 | Mike | 2011-03-13 06:59:11 -0600 (Sun, 13 Mar 2011) | 1 lineChanged paths: M /BankDocSystem/trunk/auditmaint.asp M /BankDocSystem/trunk/borrowerdeleteconfirm.asp

Fixed Bug #2214------------------------------------------------------------------------r19 | Mike | 2011-03-13 06:52:09 -0600 (Sun, 13 Mar 2011) | 1 lineChanged paths: M /BankDocSystem/trunk/documentdeletemaint.asp

Fixed Bug #321------------------------------------------------------------------------r18 | Frank | 2011-03-12 09:55:32 -0700 (Sat, 12 Mar 2011) | 1 lineChanged paths: M /BankDocSystem/trunk/default.asp

Working on Feature #123------------------------------------------------------------------------r16 | Mike | 2011-03-12 08:52:59 -0700 (Sat, 12 Mar 2011) | 1 lineChanged paths: M /BankDocSystem/trunk/addcomments.asp D /BankDocSystem/trunk/bank_list.asp

Fixed Buf #11282

source\BankDocSystem>svn log -v

Output from Command Line Log Command

8

svn log -v

source\BankDocSystem>svn log -v

Output from Command Line Log Command

9

svn log file:///c:/bcd/trunk -v

The log command can use the option: --with-no-revprops --with-all-revprops --with-revprop AUTHOR

svn log file:///c:/bcd/trunk/sort.asp -v

svn log file:///c:/bcd/trunk/ sort.asp tune.cpp -v

svn log –v 25:18 svn log –v 18:25

source\BankDocSystem>svn log -v

Filtering the log command by date range

10

You can specify a date range

C:\Users\Mike>svn log file:///c:/devvc -r {2011-05-11}:{2011-03-13}------------------------------------------------------------------------r25 | Mike | 2011-03-13 07:00:04 -0600 (Sun, 13 Mar 2011) | 1 line

Fixed Bug #284------------------------------------------------------------------------r24 | Mike | 2011-03-13 06:59:11 -0600 (Sun, 13 Mar 2011) | 1 line

Fixed Bug #2214------------------------------------------------------------------------r23 | Mike | 2011-03-13 06:58:26 -0600 (Sun, 13 Mar 2011) | 2 lines

Finished withEnhancement 227F------------------------------------------------------------------------r22 | Mike | 2011-03-13 06:57:42 -0600 (Sun, 13 Mar 2011) | 2 lines

Working on db structEnhancement 227F------------------------------------------------------------------------r21 | Mike | 2011-03-13 06:57:01 -0600 (Sun, 13 Mar 2011) | 2 lines

Woring on db structEnhancement 227F------------------------------------------------------------------------r20 | Mike | 2011-03-13 06:54:07 -0600 (Sun, 13 Mar 2011) | 1 line

Creating branch to update db structure to take advantage of new indexing------------------------------------------------------------------------r19 | Mike | 2011-03-13 06:52:09 -0600 (Sun, 13 Mar 2011) | 1 line

Fixed Bug #321------------------------------------------------------------------------r18 | Frank | 2011-03-12 09:55:32 -0700 (Sat, 12 Mar 2011) | 1 line

Working on Feature #123------------------------------------------------------------------------

svn log file:///c:/devvc -r {2011-05-11}:{2011-03-13}

Showing additional columns on “Check for modifications”

11

You can add columns of properties usedin any of the objects

Showing additional columns on “Check for modifications”

12

Select columns to show

By default the columns are added to the end of the set of columns – you can drag them anywhere

Unified-diff format

This format is used by many different version control systems.

Unified-diff format

This format is used by many different version control systems.

Repository Information – The log command

The LOG command reports on log messages from a repository

You can filter on - Projects - Revision - Date

C:\sourcetest\payroll>svn log file:///C:/repository/payroll -v------------------------------------------------------------------------r4 | Mike | 2010-11-27 07:16:20 -0700 (Sat, 27 Nov 2010) | 1 lineChanged paths: D /payroll/colors.txt A /payroll/common/emails.txt M /payroll/common/mrtechie.htm

Fixed bug #449------------------------------------------------------------------------r3 | Mike | 2010-11-27 07:15:19 -0700 (Sat, 27 Nov 2010) | 2 linesChanged paths: M /payroll/days.txt A /payroll/scluc_3_wd/test.js

Modified sort routineGerman sort now works------------------------------------------------------------------------r2 | Mike | 2010-11-27 07:14:30 -0700 (Sat, 27 Nov 2010) | 1 lineChanged paths: M /payroll/numbers.txt M /payroll/scluc_3_wd/scl_3_wd.html

Fixed bug #442------------------------------------------------------------------------

Repository Information – Making log info more useful

The LOG command output can be made into XML

C:\sourcetest\payroll>svn log file:///C:/repository/payroll -v --xml<?xml version="1.0"?><log><logentry revision="4"><author>Mike</author><date>2010-11-27T14:16:20.503414Z</date><paths><path kind="file" action="D">/payroll/colors.txt</path><path kind="file" action="M">/payroll/common/mrtechie.htm</path><path kind="file" action="A">/payroll/common/emails.txt</path></paths><msg>Fixed bug #449</msg></logentry>

There are many ways to import

XML into databases

Using log info from SQL

There are only two tables you need to create

Repository ParseSubversion

CREATE TABLE logentry( revisionNumber int, revisionAuthor nvarchar(50) , revisionDate datetime, dateAdded datetime NULL, revisionLogMsg nvarchar(max) )

CREATE TABLE path(revisionNumber int ,pathKind nchar10) ,pathAction nchar(10),pathString nvarchar(max),pathCopyfromPath nvarchar(max),pathCopyfromRev nchar(10) ,dateAdded datetime)

Using log info from SQL

Then you can do any type reporting or analysis you desire

Using log info from SQL

To see how the number of commits change about a project

SELECT MONTH(revisionDate) as 'Month', YEAR(revisionDate) as 'Year', COUNT(*) as 'No. of Revisions'FROM logentry GROUP BY MONTH(revisionDate), YEAR(revisionDate)

Using log info from SQL

To see who is making the most commits

SELECT count(*) as '# of Commits', revisionAuthor as 'Author' FROM logentry group by revisionAuthor

Developers0

50100150200250300350400

Number of Commits CraigA

Frank

EliM

MathewT

SvnIntegra-tion

Bob

Susan

Another Output Example – Code Reviews

The problem

• We have a standard of doing code reviews on development work

• We need to mark which code should be reviewed and who should do reviews

• We need a report showing

• Which code needs to be reviewed

• Who should do the reviews

• Which code has been marked as “to be reviewed” but no one was assigned to review the code

Another Output Example – Code Reviews

Mark which code should be reviewed

Another Output Example – Code Reviews

Mark who should review the code

Another Output Example – Code Reviews

Mark which code has been reviewed

Now we need a report to tell people what to do

What has to be reported

Which files do not have the properties CodeReviewed and CodeReviewer

Which files in the last week (month) changed from CodeReviewed = NO to CodeReviewed = ReadyForReview

Which files have been set to CodeReviewed = YES for over 6 weeks

Hook script workshop

Every hook script should start with a

policy and a procedure

Policy : Any changes to Log messages must be logged.

Procedure : Contact the Subversion administrator and tell them what was changed.

26

Hook script workshop

WHY DO WE CARE ABOUT THE POLICY

“Any changes to Log or Author properties must be logged.”

If we verify changes, and the log messages change how do we know if the Audit is valid.

AuditingSome log information triggers events or is moved to Issue/Bug/Planning systems

ConnectionsSometimes people type things incorrectly

Corrections

27

Hook script workshop

We need to have some way of making sure the changes to Log

Messages are OK

Options

28

Only allow administrators to change logs

Have permissions

to change logs be

turned on/off

Periodically make lists of all logs and

compare

Only allow administrators to change

logs

Have permissions to change logs be turned

on/offOnly allow administrators to change logs

Hook script workshop

Or turn the control over to a hook script

Steps• Which hook script• What language• Write some code• Test it• Implement it

29

Hook script workshop

start-commitpre-unlockpre-revprop-changepre-lockpre-commitpost-unlockpost-revprop-changepost-lockpost-commit

start-commitpre-unlockpre-revprop-changepre-lockpre-commitpost-unlockpost-revprop-changepost-lockpost-commit

pre-revprop-change This runs before any changes Must exist and return 0 to allow changes

post-revprop-change This runs after any changes

pre-revprop-change This runs before any changes Must exist and return 0 to allow changes

post-revprop-change This runs after any changes Can be used to create a file or Email information Will not run unless the pre-revprop-change hook exists

Which hook script ?

30

Hook Scripts - post-revprop-change

post-revprop-change This is run immediately after the revision property

change Will not run unless the pre-revprop-change hook

is run successfully Input parameters :

• Repository path• Name of revision whose property is

about to be modified• Username attempting the property

change• Name of the property • Name of change A (added),

D (deleted), or M (modified)

Common UsesSend an email notification that the property has been changed

31

Hook script workshop

Which language ?

Batch filePythonPerlWindows script (Javascript)Any EXE As long as it can be passed parameters and return an error codeAny code, if initiated by a BAT file

32

Hook script workshop

Plan your program

Subversion

Repository

Log FilePost-revprop-change.jsPost-revprop-change.bat

Get log message

33

Hook script workshop

Where does thelog file go ?

Log FilePost-revprop-change.js

function writefile(textToWrite,repPath) { var fso=new ActiveXObject("Scripting.FileSystemObject"); var txtFile=fso.OpenTextFile(repPath + "\\post-reprop-change.log",8,true,0); var str=textToWrite;txtFile.WriteLine(str); txtFile.Close(); }

34

Hook script workshop

What should the log file output be?

Log File

------Sun Feb 27 15:16:27 MST 2011------ Repository path=C:\demoAdminLock Revision=7 User=MIke Modified Property=svn:log Action=M Old log message=fixed bug 12321 see feature 1881New log message=fixed bug 12321 see feature 1889

------Sun Feb 27 15:17:49 MST 2011------ Repository path=C:\demoAdminLock Revision=6 User=MIke Modified Property=svn:log Action=M Old log message=made a copy of trunk into tags/rel 1.0New log message=made a copy of trunk into tags/rel 1.0Release sent to QA - 2/23/2011

It also could be -tab delimited -comma separated

02272011-15:16:27,”C:\demoAdminLock “,7,”MIke “,”svn:log”,”M“,”fixed bug 12321 see feature”,”fixed bug 12321 see feature 1889”02172011-07:12:33,”C:\demoAdminLock “,4,”MIke “,”svn:log”,”M“,” made a copy of trunk into tags/rel 1.0”,” made a copy of trunk into tags/rel 1.0/0D/0ARelease sent to QA - 2/23/2011“

35

Check out

Edit

Update

Commit

Now you need to do some programming. Going through whatever development steps are your standards.

And use Subversion for version control.

Edit The File

Edit

Test

Edit

Test

36

And we have a new policy and procedure

Policy : All changes to Log properties must be reviewed weekly.

Procedure : The administrator will review the log file for “Changed Log Messages” every week and report anything suspicious to the project leaders.

The log files are in …………Contact the managers by email ………

37

Other hook script output examples

Policy : All branches that are classified as READY-TO-MERGE must be tested by the QA department.

Policy : All changes to projects classified as inactive must be reviewed by a project manager.

38

39

1. Decide which outputs will help you manage your development efforts.

2. Design procedures or programs to create reports.

3. Train staff on what will be reported.

Conclusions

40

Michael Lester

Copyright © 2011 Wandisco All rights reserved.