DVCS in big corporation

November, 14th 2011 DVCS in big Corporation

DVCS in big Corporation

Solutions● Centralization● Visualization

Challenges● Authentication● Authorization

About● Me● DVCS

Quick notes

About : me

About : me on SO

A Lot Rep

Many times during the day

Every single day

ask@me

CVCSServer sideClient side

And then, a miracle:

DVCSServer sideClient side

Git on a client

eclipse

Git on a client

eclipse

Reaction?

Not enthusiastic

Issues? Authentication.

Who is VonC?

X41064

Issues? Communication

Issues? Publication

Centralization

Server

Centralization

itsvcprd git

Server

MUTUALIZED

Server

Server: not root

Sudo apt-get install git

Server: not alone

Services are managed by root

Server: not in control

/usr/local content can change at any time

Recompile Everything

Recompile Everything: root

Recompile Everything: alone● Tailored services (ssh, ldap, https)

Recompile Everything: in control

Your own version of ~/usr/local

Manual recompilation?

Download sources

Configure./configure --prefix=${HULA}/@@NAMEVER@@ --with-lib=${HULL} --with-openssl --with-curl --with-expat --with-iconv=${HUL} --with-gitconfig=${HUL}/var/gitconfig --with-editor=vim --with-perl=${HULA}/perl/bin/perl --with-zlib=${HUL} --with-tcltk=no --with-python=${HULA}/python/bin/python

./configure --prefix=${HULA}/@@NAMEVER@@ --with-lib=${HULL} --with-openssl --with-curl --with-expat --with-iconv=${HUL} --with-gitconfig=${HUL}/var/gitconfig --with-editor=vim --with-perl=${HULA}/perl/bin/perl --with-zlib=${HUL} --with-tcltk=no --with-python=${HULA}/python/bin/python./configure --prefix=${HULA}/@@NAMEVER@@

--enable-ssl=shared --enable-ssl --with-ssl=${HUL}/ssl --enable-proxy --enable-proxy-connect --enable-proxy-ftp --enable-proxy-http --with-ldap --enable-ldap --enable-authnz-ldap --enable-authn-alias --with-apr=${HUL} --with-apr-util=${HUL} --enable-mods-shared=all --with-z=${HUL} @@WITHOUT_GNU_LD@@

./configure --prefix=${HULA}/@@NAMEVER@@ --enable-ssl=shared --enable-ssl --with-ssl=${HUL}/ssl --enable-proxy --enable-proxy-connect --enable-proxy-ftp --enable-proxy-http --with-ldap --enable-ldap --enable-authnz-ldap --enable-authn-alias --with-apr=${HUL} --with-apr-util=${HUL} --enable-mods-shared=all --with-z=${HUL} @@WITHOUT_GNU_LD@@

./configure --prefix=${HULS}/@@NAMEVER@@ --enable-shared --enable-static --with-zlib=${HUL} --with-ssl-engine=${HUL}/ssl --without-privsep-user --with-pid-dir=${HUL}/var/run --with-default-path=@@PATH@@ --with-privsep-path=${HUL}/var/empty @@WITHOUT_GNU_LD@@

Manual recompilation?● Make● Make install

Rinse and repeat

GitGit

Gcc 3.4.6openssl,libssh2,curl,libiconv,expat,libidn,zlibGcc 3.4.6openssl,libssh2,curl,libiconv,expat,libidn,zlib

opensshApache Http, lynxSubversion, Python, perl

32 libraries

14 applications

4 modules (Perl or ruby)

Manual Automated recompilation

You've got git.

Now What?

What is missing?Server sideClient side

Gitolite: authorization script

Repo1: user1, user2

Repo2: user2, user3

gl-auth-command

Server side

Git command

Client side

Cmd output

Gitolite: openssh

Repo1: user1, user2

Repo2: user2, user3

Server side

Git command

Client side

Cmd output

gl-auth-command

Gitolite: forced command

Command= "compileEverything/gitolite/bin/gl-auth-command bjensen",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty ssh-rsaAAAAB3NzaC1yc2EAAA...

~/.ssh/authorized_keys

Gitolite: not for users

Repo1: fisheye

Repo2: sonar

Server sideClient side

gl-auth-command

Repo1: user1

Repo2: user2

SSH is not enoughServer sideClient side

ssh gitolite

Git & “smart http”Server sideClient side

git-http-backend

Gitolite: httpd

gl-auth-command

Server side

Git command

Client side

Http answer

git-http-backend

Gitolite: LDAP alias

<AuthnProviderAlias ldap myldap> AuthLDAPBindDN cn=Manager,dc=example,dc=com AuthLDAPBindPassword secret AuthLDAPURL ldap://localhost:9011/dc=example,dc=com ?uid?sub?(objectClass=*)</AuthnProviderAlias>

Httpd.conf

Gitolite: REMOTE_USER

Httpd.conf

ScriptAlias /hgit/ compileEverything/gitolite/bin/gl-auth-command/ <Location /hgit> AuthName "LDAP authentication for ITSVC Smart HTTP Git repositories" AuthBasicProvider myldap Require valid-user AddHandler cgi-script cgi </Location>

Gitolite: https://itsvcprdgit:8453/hgit

# GitHttp on 8453<VirtualHost itsvcprdgit.world.company:8453> ServerName itsvcprdgit.world.company ServerAlias itsvcprdgit SetEnv GIT_PROJECT_ROOT /path/to/repositories SetEnv GIT_HTTP_EXPORT_ALL SetEnv GITOLITE_HTTP_HOME /home/auser/compileEverything

Httpd.conf

Httpd: multi-domain SSL certificateServer sideClient side

itsvcprdgit.world.company

itsvcprdgit

X509v3 extensions: X509v3 Subject Alternative Name: DNS:itsvcprdgit.world.company, DNS:itsvcprdgit

Are we there yet?Server sideClient side

gitolite

GitWeb

gitweb.cgi ?Server sideClient side

Gitweb.cgigl-auth-command

GitWeb: GL_USER

# finally the user name$ENV{GL_USER} = $cgi->remote_user || "gitweb";# now get gitolite stuff in...unshift @INC, $ENV{GL_BINDIR};require gitolite; gitolite -> import;

~/gitweb/gitweb.conf.pl

GitWeb: repo_rights()

$export_auth_hook = sub { my $repo = shift; return unless $repo =~ s/^\Q$projectroot\E\/?(.+)\.git$/$1/; # check for (at least) "R" permission my ($perm, $creator) = &repo_rights($repo); return ($perm =~ /R/);};

GitWeb: https://itsvcprdgit:8443/git

DocumentRoot compileEverything/gitweb Alias /git compileEverything/gitweb <Directory compileEverything/gitweb> AuthBasicProvider myldap AddHandler cgi-script cgi DirectoryIndex gitweb.cgi </Directory>

Httpd.conf

Are we there now?Server sideClient side

gitolite

gitweb

cgit.cgi ?Server sideClient side

cgit.cgigl-auth-command

CGit: repo_rights()

if ($request_uri ne "/cgit/" && $request_uri ne "/cgit/cgit.pl/") { (my $repo)=($path_info =~ /\/([^\/]+)/); my ($perm, $creator) = &repo_rights($repo); if ($perm =~ /R/) system("compileEverything/cgit/cgit.cgi"); else print " <h1>HTTP Status 403 - Access is denied</h1>\n"; }

~/cgit/cgit.pl

CGit: https://itsvcprdgit:8463/cgit

DocumentRoot compileEverything/cgit Alias /cgit compileEverything/cgit <Directory compileEverything/cgit> AuthBasicProvider myldap SetEnv GIT_PROJECT_ROOT=.../repositories AddHandler cgi-script .cgi .pl DirectoryIndex cgit.pl </Directory>

Httpd.conf

And now?Server sideClient side

gitweb

https://itsvcprdgit:8453/hgit

https://itsvcprdgit:8443/git

https://itsvcprdgit:8463/cgit

What do they want?Server sideClient side

gitweb

https://itsvc/hgit

https://itsvc/git

https://itsvc/cgit

NO PORT NUMBER

SHORT NAMES

Reverse ProxyServer sideClient side

gitweb

NGinx: https://itsvc/xxx

location /hgit/ { proxy_pass https://itsvcprdgit.world.company:8453/hgit/;}location /git/ { proxy_pass https://itsvcprdgit.world.company:8443/git/;}location /cgit/ { proxy_pass https://itsvcprdgit.world.company:8463/cgit/;}

nginx.conf

There, there?Server sideClient side

httpd https://itsvc/hgit

https://itsvc/git

https://itsvc/cgit

What!?Server sideClient side

Issue1: authorname

Issue1: gitolite + hookServer sideClient side

gl-auth-commandPre-receive

Issue1: pre-receive hookglog=`git log --format='%cn~%h~%s' $new --not --all`for cns in $glog ; do atLeastOneCommit=true echo branch $name: $cns cn=ècho $cns | cut -d~ -f1` hash=ècho $cns | cut -d~ -f2` subject=ècho $cns | cut -d~ -f3` if [ "$cn" = "$GL_USER" ]; then echo "one commit found with $GL_USER as committer name" exit 0 fidone

Issue1: pre-receive hook effect

remote: no commit with a committer name equals to 'bjensen', so this push is denied.

Issue2: Actual user on server

Issue2: authorname on serverauser@vonc-VirtualBox:~/gitolite/demo$ ../../bin/git commit -m "default user on server"[master c694ed7] default user on server Committer: auser <auser@vonc-VirtualBox.(none)>Your name and email address were configuredautomatically based on your username and hostname. Please check that they are accurate. git config --global user.name "Your Name" git config --global user.email you@exemp.com

Issue2: putty+ git wrapper

Git wrapper

alias agitBjensenItsvcprdgit='alias git="${H}/sbin/wgit u bjensen,bjensen@example.com,itsvcprdgit.world.company,bjensen"'

auser@vonc-VirtualBox:~$ git st[ bjensen,bjensen@example.com for itsvcprdgit.world.company ]# On branch masternothing to commit (working directory clean)

Issue2: authorname on server

[ bjensen,bjensen@example.com for itsvcprdgit.world.company ]

Finally, are we there?Server sideClient side

gitolite

gitweb cgit

Pre-receivehook

Gitwrapper

Conclusion: Server is hard

Conclusion: Application is hard

Conclusion: Big Corporation

Any questions?

If you need to introduce any tool in a big corporation, this presentation will help you be ware of the question you need to be prepare to answer.

This is a more Git-oriented presentation, but most of it equally applies to Mercurial.

Solutions● Centralization● Visualization

Challenges● Authentication● Authorization

About● Me● DVCS

Quick notes

http://www.slideshare.net/dchaffiol/dvcs-in-big-corporation

About : me

The opinions and elements in this presentations are mine and does not represent my current or former clients.

About : me on SO

A Lot Rep

Many times during the day

Every single day

ask@me

CVCSServer sideClient side

And then, a miracle:

DVCSServer sideClient side

Git on a client

eclipse

Git on a client

eclipse

Reaction?

Not enthusiastic

Issues? Authentication.

Who is VonC?

X41064

Issues? Communication

Issues? Publication

Centralization

Server

Centralization

itsvcprd git

Server

MUTUALIZED

Server

Server: not root

Sudo apt-get install git

Server: not alone

Services are managed by root

Server: not in control

/usr/local content can change at any time

http://serverfault.com/questions/281810/how-to-install-packages-on-linux-or-solaris-on-non-default-paths

Recompile Everything

Recompile Everything: root

Recompile Everything: alone● Tailored services (ssh, ldap, https)

Recompile Everything: in control

Your own version of ~/usr/local

Download sources

Configure./configure --prefix=${HULA}/@@NAMEVER@@ --with-lib=${HULL} --with-openssl --with-curl --with-expat --with-iconv=${HUL} --with-gitconfig=${HUL}/var/gitconfig --with-editor=vim --with-perl=${HULA}/perl/bin/perl --with-zlib=${HUL} --with-tcltk=no --with-python=${HULA}/python/bin/python

./configure --prefix=${HULA}/@@NAMEVER@@ --with-lib=${HULL} --with-openssl --with-curl --with-expat --with-iconv=${HUL} --with-gitconfig=${HUL}/var/gitconfig --with-editor=vim --with-perl=${HULA}/perl/bin/perl --with-zlib=${HUL} --with-tcltk=no --with-python=${HULA}/python/bin/python./configure --prefix=${HULA}/@@NAMEVER@@

--enable-ssl=shared --enable-ssl --with-ssl=${HUL}/ssl --enable-proxy --enable-proxy-connect --enable-proxy-ftp --enable-proxy-http --with-ldap --enable-ldap --enable-authnz-ldap --enable-authn-alias --with-apr=${HUL} --with-apr-util=${HUL} --enable-mods-shared=all --with-z=${HUL} @@WITHOUT_GNU_LD@@

./configure --prefix=${HULA}/@@NAMEVER@@ --enable-ssl=shared --enable-ssl --with-ssl=${HUL}/ssl --enable-proxy --enable-proxy-connect --enable-proxy-ftp --enable-proxy-http --with-ldap --enable-ldap --enable-authnz-ldap --enable-authn-alias --with-apr=${HUL} --with-apr-util=${HUL} --enable-mods-shared=all --with-z=${HUL} @@WITHOUT_GNU_LD@@

./configure --prefix=${HULS}/@@NAMEVER@@ --enable-shared --enable-static --with-zlib=${HUL} --with-ssl-engine=${HUL}/ssl --without-privsep-user --with-pid-dir=${HUL}/var/run --with-default-path=@@PATH@@ --with-privsep-path=${HUL}/var/empty @@WITHOUT_GNU_LD@@

Manual recompilation?● Make● Make install

Rinse and repeat

GitGit

Gcc 3.4.6openssl,libssh2,curl,libiconv,expat,libidn,zlibGcc 3.4.6openssl,libssh2,curl,libiconv,expat,libidn,zlib

opensshApache Http, lynxSubversion, Python, perl

32 libraries

14 applications

4 modules (Perl or ruby)

Manual Automated recompilation

https://github.com/VonC/compileEverything

You've got git.

Now What?

What is missing?Server sideClient side

Gitolite: authorization script

Repo1: user1, user2

Repo2: user2, user3

gl-auth-command

Server side

Git command

Client side

Cmd output

https://github.com/sitaramc/gitolite

Gitolite: openssh

Repo1: user1, user2

Repo2: user2, user3

Server side

Git command

Client side

Cmd output

gl-auth-command

Gitolite: forced command

Command= "compileEverything/gitolite/bin/gl-auth-command bjensen",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty ssh-rsaAAAAB3NzaC1yc2EAAA...

~/.ssh/authorized_keys

Gitolite: not for users

Repo1: fisheye

Repo2: sonar

gl-auth-command

Repo1: user1

Repo2: user2

SSH is not enoughServer sideClient side

ssh gitolite

Git & “smart http”Server sideClient side

git-http-backend

Gitolite: httpd

gl-auth-command

Server side

Git command

Client side

Http answer

git-http-backend

Gitolite: LDAP alias

<AuthnProviderAlias ldap myldap> AuthLDAPBindDN cn=Manager,dc=example,dc=com AuthLDAPBindPassword secret AuthLDAPURL ldap://localhost:9011/dc=example,dc=com ?uid?sub?(objectClass=*)</AuthnProviderAlias>

Httpd.conf

Gitolite: REMOTE_USER

Httpd.conf

ScriptAlias /hgit/ compileEverything/gitolite/bin/gl-auth-command/ <Location /hgit> AuthName "LDAP authentication for ITSVC Smart HTTP Git repositories" AuthBasicProvider myldap Require valid-user AddHandler cgi-script cgi </Location>

Gitolite: https://itsvcprdgit:8453/hgit

# GitHttp on 8453<VirtualHost itsvcprdgit.world.company:8453> ServerName itsvcprdgit.world.company ServerAlias itsvcprdgit SetEnv GIT_PROJECT_ROOT /path/to/repositories SetEnv GIT_HTTP_EXPORT_ALL SetEnv GITOLITE_HTTP_HOME /home/auser/compileEverything

Httpd.conf

Httpd: multi-domain SSL certificateServer sideClient side

itsvcprdgit.world.company

itsvcprdgit

X509v3 extensions: X509v3 Subject Alternative Name: DNS:itsvcprdgit.world.company, DNS:itsvcprdgit

Are we there yet?Server sideClient side

gitolite

GitWeb

gitweb.cgi ?Server sideClient side

Gitweb.cgigl-auth-command

GitWeb: GL_USER

# finally the user name$ENV{GL_USER} = $cgi->remote_user || "gitweb";# now get gitolite stuff in...unshift @INC, $ENV{GL_BINDIR};require gitolite; gitolite -> import;

GitWeb: repo_rights()

$export_auth_hook = sub { my $repo = shift; return unless $repo =~ s/^\Q$projectroot\E\/?(.+)\.git$/$1/; # check for (at least) "R" permission my ($perm, $creator) = &repo_rights($repo); return ($perm =~ /R/);};

GitWeb: https://itsvcprdgit:8443/git

DocumentRoot compileEverything/gitweb Alias /git compileEverything/gitweb <Directory compileEverything/gitweb> AuthBasicProvider myldap AddHandler cgi-script cgi DirectoryIndex gitweb.cgi </Directory>

Httpd.conf

Are we there now?Server sideClient side

gitolite

gitweb

cgit.cgi ?Server sideClient side

cgit.cgigl-auth-command

CGit: repo_rights()

if ($request_uri ne "/cgit/" && $request_uri ne "/cgit/cgit.pl/") { (my $repo)=($path_info =~ /\/([^\/]+)/); my ($perm, $creator) = &repo_rights($repo); if ($perm =~ /R/) system("compileEverything/cgit/cgit.cgi"); else print " <h1>HTTP Status 403 - Access is denied</h1>\n"; }

~/cgit/cgit.pl

CGit: https://itsvcprdgit:8463/cgit

DocumentRoot compileEverything/cgit Alias /cgit compileEverything/cgit <Directory compileEverything/cgit> AuthBasicProvider myldap SetEnv GIT_PROJECT_ROOT=.../repositories AddHandler cgi-script .cgi .pl DirectoryIndex cgit.pl </Directory>

Httpd.conf

And now?Server sideClient side

gitweb

https://itsvcprdgit:8453/hgit

https://itsvcprdgit:8443/git

https://itsvcprdgit:8463/cgit

What do they want?Server sideClient side

gitweb

https://itsvc/hgit

https://itsvc/git

https://itsvc/cgit

NO PORT NUMBER

SHORT NAMES

Reverse ProxyServer sideClient side

gitweb

NGinx: https://itsvc/xxx

location /hgit/ { proxy_pass https://itsvcprdgit.world.company:8453/hgit/;}location /git/ { proxy_pass https://itsvcprdgit.world.company:8443/git/;}location /cgit/ { proxy_pass https://itsvcprdgit.world.company:8463/cgit/;}

nginx.conf

There, there?Server sideClient side

httpd https://itsvc/hgit

https://itsvc/git

https://itsvc/cgit

What!?Server sideClient side

Issue1: authorname

Issue1: gitolite + hookServer sideClient side

gl-auth-commandPre-receive

Issue1: pre-receive hookglog=`git log --format='%cn~%h~%s' $new --not --all`for cns in $glog ; do atLeastOneCommit=true echo branch $name: $cns cn=ècho $cns | cut -d~ -f1` hash=ècho $cns | cut -d~ -f2` subject=ècho $cns | cut -d~ -f3` if [ "$cn" = "$GL_USER" ]; then echo "one commit found with $GL_USER as committer name" exit 0 fidone

Issue1: pre-receive hook effect

remote: no commit with a committer name equals to 'bjensen', so this push is denied.

Issue2: Actual user on server

Issue2: authorname on serverauser@vonc-VirtualBox:~/gitolite/demo$ ../../bin/git commit -m "default user on server"[master c694ed7] default user on server Committer: auser <auser@vonc-VirtualBox.(none)>Your name and email address were configuredautomatically based on your username and hostname. Please check that they are accurate. git config --global user.name "Your Name" git config --global user.email you@exemp.com

Issue2: putty+ git wrapper

Git wrapper

alias agitBjensenItsvcprdgit='alias git="${H}/sbin/wgit u bjensen,bjensen@example.com,itsvcprdgit.world.company,bjensen"'

auser@vonc-VirtualBox:~$ git st[ bjensen,bjensen@example.com for itsvcprdgit.world.company ]# On branch masternothing to commit (working directory clean)

Issue2: authorname on server

[ bjensen,bjensen@example.com for itsvcprdgit.world.company ]

Finally, are we there?Server sideClient side

gitolite

gitweb cgit

Pre-receivehook

Gitwrapper

Conclusion: Server is hard

Conclusion: Application is hard

Conclusion: Big Corporation

Any questions?

DVCS in big corporation

Technology

DVCS, Nuclear DVCS & HERMES Recoil Detector

DVCS-2 Mechanical Design Status

DVCS at JLab

DVCS results with unpolarized and polarized target

Git Going With DVCS v1.5.2

Unbiased determination of DVCS Compton form factors

Making the switch to DVCS

Pillaging DVCS Repos For Fun And Profit

Big Rivers Electric Corporation Table of Contents - KY … Rivers Electric... · · 2016-10-06Big Rivers Electric Corporation Table of Contents ... Big Rivers Electric Corporation

2014 DVCS Run Preparations DVCS and GMP Symbiosis Charles Hyde

Git (FS and DVCS)

DVCS branching (with mercurial)

DVCS-2 Mechanical Design Status Charles Hyde Guy Reinmuth LPC Clermont Hall A DVCS Collaboration Meeting 15-16 December 2008 Monastir, Tunisia

Big Rivers Electric Corporation Rivers Electric Corporation... · P.S.C. KY.NO. 27 CANCELLING P.S.C.KY.NO. 26 Big Rivers Electric Corporation 201 Third Street Henderson, Kentucky

Plastic SCM 4 DVCS Value Proposition

Git - A quick introduction to DVCS

Nondiagonal DVCS

Where is the Spin of the Nucleon hidden?DVCS Bethe-Heitler (BH) γ + HERMES, JLAB: DVCS-BH interference: 10use BH as a vehicle to study DVCS H1, ZEUS: measure DVCS cross section directly

A09 - DVCS 6100 - System Presentation

Refreshing Software Development with DVCS