Using both Git and Subversion
A few years ago, all of a sudden, CVS was EVIL and SVN was the redemption.
So everybody switched.
And now that everybody has switched from CVS to SVN, there is Git and, yes sir: SVN is EVIL.
All of a sudden there are thousands of reasons why not to use Subversion.
Oh well… Here we go again…
But to make it a bit of a challenge, I will not be replacing Subversion with Git in this manual; I want them running alongside each other, and I want them to sync automagically. As long as there is support for SVN and Git, I want my users to have a choice which to use. And in a year or 2, when the next Version Control / Source Code Management buzz hits the 'net, I'll just plug that in with these two.
Off we go.
- Note:
- You are adviced to read this entire page before you replay it, because currently I do not end up with a working setup.
This will change in the near future.
- Disclaimer
- Current setup
- Git plans
- Install Git
- Preparations
- Create Git repositories
- Test
- Implementation in production environment
- Conclusion
Disclaimer
You will understand that I will not accept any responsibility for any of your actions (or lack thereof).
Everything I describe below, was done on a FreeBSD server that doesn't run X.
Current setup
- Subversion is installed, configured and being used.
- There are several post-commit hooks: a few to send mails, one to checkout changes to the website to the production environment, etcetara.
- All users use the same system account, and are identified by the SSH key they use.
- All Nerdstock-members have read access to the entire repo and write access to the projects they work on; the rest of the world cannot access the repo directly.
- WebSVN is installed and configured to display certain parts of the repo.
Git plans
- Git and SVN will sync bi-directionally.
- Git will not have any post-commit hooks (if those exist in Git); we wouldn't want to receive all mails twice.
- Git access will be arranged and restricted in the same way as it is for Subversion.
- A webinterface will be installed and configured to display the same parts as WebSVN does.
- Git will send commits to some parts of the code to one or more publicly available Git repos.
Install Git
A good first step would be to install Git; I installed it with these options:
Mmmhh…
I guess I should have checked out gitweb first, instead of starting the installation and search the web while it was installing.
gitweb really looks like crap; I like cgit a lot more.
For now, I'll just install cgit as well; if I ever find the time, I'll reinstall Git without gitweb (and get rid of the dependencies that were installed for it).
The above is for the installation at the server; on my laptop I exclude GITWEB and include GUI.
Preparations
For the sake of this manual, I'll be pretending my Nerdstock Subversion working copy is in /local/copy on my laptop; the repository is in /server/repo on the server.
I'll be using git-svn(1) for the communication between Git and SVN, so I read the manpage.
Furthermore, since the manpage keeps talking about tags, branches and trunk, I do something I should have done a long time ago: I create those directories in the Subversion repo (some of our projects already have these directories, but some of the smaller projects don't). This is simple:
rob@laptop> cd /local/copy rob@laptop> svn status -u # commit or revert as desired; make sure there are no open changes rob@laptop> cd /local/copy/mastermind4bash rob@laptop> mkdir tags branches trunk rob@laptop> svn add branches tags trunk rob@laptop> svn commit -m "Added branches, tags and trunk directories" rob@laptop> svn mv mastermind trunk rob@laptop> cd /local/copy/rotate rob@laptop> mkdir tags branches trunk rob@laptop> svn add branches tags trunk rob@laptop> svn commit -m "Added branches, tags and trunk directories" rob@laptop> svn mv rotate trunk … rob@laptop> cd /local/copy rob@laptop> svn commit -m "Moving projects to trunks"
The Subversion repo is now layed out like this:
/ |- project01 | |- branches | | |- branch01 | | | `- … | | |- branch02 | | | `- … | | `- … | |- tags | | `- tag01 | | `- … | `- trunk | `- … |- project02 | |- branches | | `- branch01 | | `- … | |- tags | | `- tag01 | | `- … | `- trunk | `- … …
See svn.nerdstock.org if this beautiful drawing is not clear to you.
To be sure I log in at the SVN repo server and check /server/repo/hooks/* to see whether they need to be edited after I've made these changes.
Create Git repositories
From what I read, I understand that I should create a Git repo for each project in the SVN repo; so we'll have separate Git repos for moerAskip, XiWell, Rbot, Smarty, etcetara.
To minimize the chance of screwing up, I first create a new Subversion repo for testing; I do this locally, so I can be sure that it's not the network or SSH getting in the way if things go wrong. Copy this script, make it executable, change the basedir and email variables, and execute it:
#!/usr/local/bin/bash
# This script will create a Subversion repo and working copy that can be used for testing.
# See http://nerdstock.org/git_svn for more info.
# NB:
# This script was only tested on FreeBSD.
# (I think only the `find' command behaves differently on Linux.)
# Change these.
basedir="/local"
email="rob"
mkdir -p ${basedir}/svn
svnadmin create ${basedir}/svn
echo -e "#!/bin/sh\necho \"Repo \$1 is now at revision \$2\" | mail -s \"SVN commit\" ${email}\n" > ${basedir}/svn/hooks/post-commit
chmod 755 ${basedir}/svn/hooks/post-commit
mkdir ${basedir}/tmp
for p in 1 2; do
mkdir -p ${basedir}/tmp/project${p}/branches ${basedir}/tmp/project${p}/tags ${basedir}/tmp/project${p}/trunk
for (( d=1; ${d} < ${RANDOM:(-1)}; d++ )); do
mkdir -p ${basedir}/tmp/project${p}/trunk/dir${d}
for(( f=1; ${f} < ${RANDOM:(-1)}; f++ )); do
echo "file ${f}" > ${basedir}/tmp/project${p}/trunk/dir${d}/file${f}
done
done
for (( f=1; ${f} < ${RANDOM:(-1)}; f++ )); do
echo "file ${f}" > ${basedir}/tmp/project${p}/trunk/file${f}
done
done
for p in 1 2; do svn import -m "Initial import of project ${p}" ${basedir}/tmp/project${p} file://${basedir}/svn/project${p}; done
rm -rf ${basedir}/tmp
mkdir ${basedir}/work
svn checkout file://${basedir}/svn ${basedir}/work
find ${basedir}/work -type f ! -path "*/.svn/*" -print | while read f; do echo -e "\n${RANDOM}" >> ${f}; done
svn commit -m "Committing all" ${basedir}/work
svn copy -m "Tagging version 0.1 of project1" file://${basedir}/svn/project1/trunk file://${basedir}/svn/project1/tags/v0.1
find ${basedir}/work -type f ! -path "*/.svn/*" -print | while read f; do echo -e "\n${RANDOM}" >> ${f}; done
svn commit -m "Committing project2" ${basedir}/work/project2
svn copy -m "Creating branch mybranch of project2" file://${basedir}/svn/project2/trunk file://${basedir}/svn/project2/branches/mybranch
svn update ${basedir}/work/project2
find ${basedir}/work/project2/branches/mybranch -type f ! -path "*/.svn/*" -print | while read f; do echo -e "\n${RANDOM}" >> ${f}; done
svn commit -m "Did some work on mybranch" ${basedir}/work/project2/branches/mybranch
find ${basedir}/work/project2/trunk -type f ! -path "*/.svn/*" -print | while read f; do echo -e "\n${RANDOM}" >> ${f}; done
find ${basedir}/work/project1/trunk -type f ! -path "*/.svn/*" -print | while read f; do echo -e "\n${RANDOM}" >> ${f}; done
svn commit -m "Committing project1" ${basedir}/work/project1/trunk
(To save all output to a file, execute it like ./svntestrepo.sh &> ./output.txt.)
After executing this script, I have:
- a (local) Subversion repository in
/local/svn - a post-commit hook
(If you did not receive any mails, solve you mail troubles first,rm -rf /local/svn /local/workand start over.) - a working copy in
/local/work - some history
- a tag called
v0.1in project1 /local/work/project1/tagshas not yet been updated to include the v0.1 tag- a branch called
mybranchin project 2 - some uncommitted work in
/local/work/project2/trunk
This should suffice for testing.
And now the creation of the Git repositories:
rob@laptop> mkdir /local/git rob@laptop> git svn clone --stdlayout file:///local/svn/project1 /local/git/project1 rob@laptop> git svn clone --stdlayout file:///local/svn/project2 /local/git/project2
After executing that, I have (on top of the above):
- 2 Git repositories; 1 for
project1and 1 forproject2
One more thing to do before I continue: create the file ${HOME}/.gitconfig:
# ~/.gitconfig [user] name = Rob la Lau email = rob[at]nerdstock.org
This is a good time to start poking around in the Git repositories, where the .git directories are the most interesting ones, of course. I keep the Git documentation open while I do this.
I also start gitk, the GUI tool, but this doesn't really make things clearer for me (yet).
git help also has a lot of info.
…
Now I'm ready to play with it.
Test
Let me start by modifying the files in both Git repos:
rob@laptop> find /local/git/project1 -type f ! -path "*/.git/*" -print | while read f; do echo -e "\n${RANDOM}" >> ${f}; done
rob@laptop> find /local/git/project2 -type f ! -path "*/.git/*" -print | while read f; do echo -e "\n${RANDOM}" >> ${f}; done
Scenario 1
Commit the changes in /local/git/project1, and see whether I can make them show up in the /local/work/project1/trunk SVN working copy (and whether the post-commit hook works when I commit via git svn).
rob@laptop> git status /local/git/project1/ fatal: Not a git repository (or any of the parent directories): .git
Mmmhh… So Git only looks at the current directory and it's parents…
This means that if my current working directory is /local/git/project1 and I request git status /local/git/project2, I will probably get the status for project1; this is quite error-prone, especially because the name of the repository is not in the output:
rob@laptop> cd /local/git/project1/ rob@laptop> git status # On branch master # Changed but not updated: # (use "git add..." to update what will be committed) # (use "git checkout -- ..." to discard changes in working directory) # # modified: dir1/file1 # modified: file1 # modified: file2 # modified: file3 # modified: file4 # no changes added to commit (use "git add" and/or "git commit -a")
(Your output will probably differ, because we used RANDOM for generation of the files.)
Nice: output tells me what to do next.
rob@laptop> git commit -a -m "First Git commit (project1)."
[master 2cbc917] First git commit (project1).
5 files changed, 10 insertions(+), 0 deletions(-)
rob@laptop> git svn dcommit
Committing to file:///local/svn/project1/trunk ...
M dir1/file1
M file1
M file2
M file3
M file4
Committed r9
M dir1/file1
M file1
M file2
M file3
M file4
r9 = 6456066c125e00332419003865bb3a4e7f67864b (refs/remotes/trunk)
No changes between current HEAD and refs/remotes/trunk
Resetting to the latest refs/remotes/trunk
rob@laptop>
### "You have mail!" ###
rob@laptop> cd /local/work/project1
rob@laptop> svn status -u
* 8 trunk/dir1/file1
* 8 trunk/file1
* 8 trunk/file2
* 8 trunk/file3
* 8 trunk/file4
* tags/v0.1/dir1/file1
* tags/v0.1/dir1
* tags/v0.1/file1
* tags/v0.1/file2
* tags/v0.1/file3
* tags/v0.1/file4
* tags/v0.1
* 2 tags
Status against revision: 9
rob@laptop> svn update
U trunk/dir1/file1
U trunk/file1
U trunk/file2
U trunk/file3
U trunk/file4
A tags/v0.1
A tags/v0.1/dir1
A tags/v0.1/dir1/file1
A tags/v0.1/file1
A tags/v0.1/file2
A tags/v0.1/file3
A tags/v0.1/file4
Updated to revision 9.
Cool! We passed the first test.
Scenario 2
Commit the uncommitted work in /local/work/project2/trunk, get those changes to /local/git/project2 and merge them with the uncommitted changes in the Git repo.
rob@laptop> cd /local/work/project2 rob@laptop> svn status -u M 6 trunk/dir1/file1 M 6 trunk/dir1/file2 M 6 trunk/dir1/file3 M 6 trunk/dir1/file4 M 6 trunk/file1 M 6 trunk/file2 Status against revision: 9 rob@laptop> svn commit -m "Committing project2 stuff." Sending project2/trunk/dir1/file1 Sending project2/trunk/dir1/file2 Sending project2/trunk/dir1/file3 Sending project2/trunk/dir1/file4 Sending project2/trunk/file1 Sending project2/trunk/file2 Transmitting file data ...... Committed revision 10.
Mmmhh… Looks like there is no git svn equivalent for svn status…
rob@laptop> cd /local/git/project2 rob@laptop> git status # On branch master # Changed but not updated: # (use "git add..." to update what will be committed) # (use "git checkout -- ..." to discard changes in working directory) # # modified: dir1/file1 # modified: dir1/file2 # modified: dir1/file3 # modified: dir1/file4 # modified: file1 # modified: file2 # no changes added to commit (use "git add" and/or "git commit -a") rob@laptop> git svn fetch M dir1/file1 M dir1/file2 M dir1/file3 M dir1/file4 M file1 M file2 r10 = 64725ba8b80a9bca728874c29be961430e3400f6 (refs/remotes/trunk)
Okay, it looks like the changes were fetched, but I can't seem to find them…
…
Oh, wait a minute. Looks like I need to commit my changes to Git first and then rebase.
rob@laptop> git commit -a -m "Pre-rebase commit" [master 95288b4] Pre-rebase commit 6 files changed, 12 insertions(+), 0 deletions(-) rob@laptop> git svn rebase First, rewinding head to replay your work on top of it... Applying: Pre-rebase commit Using index info to reconstruct a base tree... Falling back to patching base and 3-way merge... Auto-merging dir1/file1 CONFLICT (content): Merge conflict in dir1/file1 Auto-merging dir1/file2 CONFLICT (content): Merge conflict in dir1/file2 Auto-merging dir1/file3 CONFLICT (content): Merge conflict in dir1/file3 Auto-merging dir1/file4 CONFLICT (content): Merge conflict in dir1/file4 Auto-merging file1 CONFLICT (content): Merge conflict in file1 Auto-merging file2 CONFLICT (content): Merge conflict in file2 Failed to merge in the changes. Patch failed at 0001 Pre-rebase commit When you have resolved this problem run "git rebase --continue". If you would prefer to skip this patch, instead run "git rebase --skip". To restore the original branch and stop rebasing run "git rebase --abort". rebase refs/remotes/trunk: command returned error: 1
Yep, that's it. Now I need to manually resolve the conflicts (vimvimvim), and continue.
rob@laptop> git rebase --continue
You must edit all merge conflicts and then
mark them as resolved using git add
rob@laptop> git add dir1/file* file*
rob@laptop> git rebase --continue
Applying: Pre-rebase commit
rob@laptop> git status
# On branch master
nothing to commit (working directory clean)
rob@laptop> git svn dcommit
Committing to file:///home/rob/Devel/nsgit/svn/project2/trunk ...
M dir1/file1
M dir1/file2
M dir1/file3
M dir1/file4
M file1
M file2
Committed r11
M dir1/file1
M dir1/file2
M dir1/file3
M dir1/file4
M file1
M file2
r11 = f7a04841d124395c9a322c477498ca1a9f8a67dd (refs/remotes/trunk)
No changes between current HEAD and refs/remotes/trunk
Resetting to the latest refs/remotes/trunk
Well, that was a bit more work, but I guess we passed this test as well.
Another question that popped to mind: how do I check whether I have changes that haven't been committed to Subversion yet?
Scenario 3
Create tags and branches in Subversion and make them show up in Git.
I already have a tag (v0.1) in project1 and a branch (mybranch) in project2. I can make these show with
rob@laptop> cd /local/git/project1 rob@laptop> git branch -a * master remotes/tags/v0.1 remotes/trunk rob@laptop> cd /local/git/project2 rob@laptop> git branch -a * master remotes/mybranch remotes/trunk
If I would want to do some more work on one of these, I would have to do a checkout:
rob@laptop> git checkout remotes/mybranch Note: moving to 'remotes/mybranch' which isn't a local branch If you want to create a new branch from this checkout, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b <new_branch_name> HEAD is now at ac224c4... Did some work on mybranch rob@laptop> git branch -a * (no branch) master remotes/mybranch remotes/trunk
So, it looks like the branch is there, but still isn't…
I will do as the checkout told me, and create a new branch for mybranch.
rob@laptop> git checkout -b mybranch Switched to a new branch 'mybranch' rob@laptop> git branch -a master * mybranch remotes/mybranch remotes/trunk
Mmmhh… To me this looks as if I have 2 branches called mybranch now, with only the names to tell me they are related (git log does show the entire Subversion log).
Passed the test, kind of…
Scenario 4
Create tags and branches in Git and make them show up in Subversion.
This has become irrelevant (see below), so I won't spend my time on this.
Implementation in production environment
My intention was to set up Git like I set up Subversion: users would have a choice whether to use Subversion or Git, and Git and Subversion would sync.
After having played with Git, I changed my mind; see the conclusion at the end of this page if you'd like to know why.
I have now decided to set things up so that
- developers use Subversion
- Subversion has a post-commit hook that updates bare Git repositories
- the Git repositories have post-commit hooks that push changes to Github and Gitorious
- Cgit displays the Git repositories at git.nerdstock.org
This setup enables me to take advantage of the fact that Git can push changes to sites like Gitorious and Github, without having to accept the downsides.
Unfortunately…
Something I should have thought about earlier: I need an SSH key to push my changes to Github and Gitorious. This means this cannot be automated (I refuse to generate an SSH key pair without a passphrase).
So this is where I stop for now.
I have some other work to do, and in the meantime I'll think about a setup that's acceptable to me.
I will get back to this, because I did some Ruby stuff, and the Ruby community seems to have adopted Git as their SCM tool of choice (which is the reason why I started looking at Git in the first place).
For now, I hope the above was of any help, and check back soon to check whether I've finished my setup.
Have fun,
Nerdstock.org
P.s.:
The conclusion below is still valid, as far as I am concerned.
Conclusion
After all this, the only added value I see in Git is that it enables me to push my changes to sites like Github and Gitorious, allowing me to promote my software a bit more.
Comparing Git and Subversion:
- If your team exceeds 2 developers (as most teams do), they will still want to have a central repo through which they communicate their changes; this may be a 'bare' repo, but this is essentially the same as a Subversion repo.
- Git takes more time and work to setup: since each Git 'working copy' is a repo, for a team of 10 developers, 11 repos need to be configured (10 dev repos + 1 central repo), whereas for Subversion there is only 1 repo to configure.
- Git is harder to work with than Subversion:
- More user actions and a larger learning curve are required (
cd branches/branchXis easier thangit checkout branchX, because *nix users typecdtens or hundreds of times a day). - It may seem attractive to 'pull' someone else's changes into your repo, but since all branches live in the same directory, it's easy to pull them into the wrong branch.
- More user actions and a larger learning curve are required (
- The chant I keep reading about "you don't need a network connection, because you have the entire repo" is entire bullsh*t, because
- If you have a team of developers, they still need to sync their repos.
- If all development takes place at the same server, having a central bare Git repo is no different from having a local Subversion repo.
- Pulling changes from someone's Git repo means you need read access on the other person's repo; even if the repo is not located in the user's home directory, most developers like to keep personal stuff in their working copy (TODO lists, rants, notes, etcetara); they may not appreciate the fact that others can read those.
- The
git format-patch,git send-emailandgit amcommands invite developers to mail their patches, effectively circumventing the security and privacy SSH offers. - A decentralized (distributed) system is likely to cause chaos and frustration, because people will lose track of what has been committed, pushed or pulled where, which is a certain way of code getting lost.
- In Subversion, the creation of tags and branches implies a commit, and the deletion of tags and branches must be followed by a commit to effectuate it; a commit usually requires a log message. In Git the deletion of tags and branches is not logged.
All this leads me to believe that the only reason Git is hyped, is that one of the authors is a wellknown and respected name in the open source world.
For me, this is not reason enough to ditch Subversion.
| Responses are welcomed: rob[at]nerdstock.org |
 http://creativecommons.org/licenses/by-nc-sa/3.0/nl/deed.en |
