Use git and subversion together.

Are you still using SVN but have some users clamoring to try out git? Kris.me.uk details all the commands and workflow to make it work.  While I’ve known that git had some svn integration available, I prefered to switch cold-turkey to git.  You can read the reasons I switched to git.

This article looks at not replacing subversion but continuing to use it as the master source repository and use one or more git mirrors to provide git for those that are familiar or wish to try it. Having subversion as the master repository can also alleviate management worries around authoritative sources, learning curves and any lost productive during upskilling, and other barriers to change. Those users using git can interact with both the subversion master and git mirrors, committing any work to be shared with all users back to subversion as and when appropriate.

Subversion Master With Git Mirrors – kris.me.uk

Why switch to git?

Get ready to clone.

Clone … What could possibly go wrong?

If you’re a coder, you’ve already heard about distributed version control systems (DVCS) and git in particular. I was content, almost complacent, in my usage of subversion to manage my source code, both for personal projects and at work. Subversion was intended as a “compelling” upgrade for CVS, and the next version control system (VCS) I used had to be equally as compelling. I think git has cemented itself as the preferred DVCS within the Open Source community. No doubt services like github have helped popularize it over bzr and mercurial. I’ve switched over to git over the course of the last month, these are some of my motivations and a record of what I learned along the way.

Why not stick with Subversion?

Subversion, since it has been around longer, enjoys better support in text editors, IDEs, and GUI front ends . It’s easy to setup a repository that you can share with a collaborator, or sign up for a hosted repository. It’s killer feature is the ability to fetch parts of a remote repository via the svn:externals feature. I’d used that extensively to organize projects and track 3rd party software such as Pressflow or Zend Framework. Git does not have an alternative as straightforward as svn:externals.

Over the last year I noticed that I wasn’t committing code as often as I should be, resulting in a lot of conflicts when I did try to do an inevitably large commit. Ideally, I would want to have a lot of frequent and small commits. When I tried to use branching and merging in Subversion, it was tedious and felt that one misstep would result in lost work or a bad merge. Even though merge tracking was added to subversion in version 1.5, my early exposure to it has caused me to avoid it like the plague. That’s a shame, because branching and merging is very useful for keeping new features from mixing in with mainline development until it is ready. Finally, at work we’d moved to a repo hosted on assembla.com and lost some useful pre-commit hooks for keeping PHP syntax errors and debugging code from being committed. A year later, it seems our group is the only ones still using this repo so I’ve been contemplating moving back to one under our control.

I know one option would be to use git-svn to use git locally and somehow push changes to the subversion repo, but I preferred to ditch the latter wholesale and not worry about having two systems in place. There’s enough to keep track of as it is and version control should simplify things, not complicate it.

Git has quite a learning curve

Understanding the terminology of git is confusing, particularly if you’re used to working with subversion. It’s not just your working copy and the repository anymore – now there’s a stash, the index where your commits are staged, the local repository, and the remote (or origin) repository. A good front-end can really help, if you are on a Mac I’d recommend using one of the active forks of GitX. This interactive cheat sheet also helped me to figure out git and what some of its commands do (Hint: click on the colored boxes). I think you’d be better off if you could somehow forget everything you ever learned about subversion.

You’re work flow has to change too, since you now have to remember to push, that is send, your commits to a shared repository, and also fetch and merge (or pull) the commits that your colleagues have pushed to a shared repository. A good front-end or first-class support in your editor/IDE of choice really helps here.

Git is useful once muscle memory kicks in

Once we initialized our repository and started committing changes, it didn’t take more than a day to get comfortable with using git. There were a couple instances where I was jumping to my co-workers cubicle to figure out how to send my changes to him, or to learn about the difference between git’s fetch and pull commands. Google and StackOverflow were most helpful in finding answers to our questions and explaining the commands we needed to invoke.

I’ve found myself committing again quite frequently, after finishing small and discrete fixes and features. In the long run, having a more fine-grained record of changes will be useful. As promised, git also makes branching and merging extraordinarily easy. Its trivial to be working on a new feature branch, switch to the mainline development branch to commit a minor fix, the switch back to your branch and merge in the fixes you just committed. It takes just a handful of commands and you don’t need to worry about version numbers or direction, it just happens. I’ve found myself doing that 2 or 3 times during the day without worrying that I was one command line switch away from total devastation. Another cool feature we found in git, via the GitX front end, is the ability to stage and commit just some of the changes within a file.

If you’ve recently switched, or are contemplating a switch, to using git, please share your thoughts and experiences below. In the near future, I’ll be sharing the details of the work flow we’ve established and how to take advantage of git’s hooks to do useful things before and after commits.

Code and Release Management notes

I'm attending tekx, and while I'm taking notes in all the sessions with Evernote, I'll try to clean up and summarize my notes here as well.  One of the session's I attended yesterday was Eli White's session on Code and Release management.  It focused more on the techniques, policies, and practices to think about about and implement rather than the nitty-gritty howto details of each version control system.  I found that very helpful because oftentimes its difficult to learn how effectively use version control especially as part of a team's workflow.  Also, its clear now that distributed version control systems have a lot to offer, primarily, and this was my AHA! moment, because they do not impose any workflow on you.  With a central subversion repository, all the developers on a project are forced to work in a certain way.  With multiple repos, you could still setup a central repository for everyone to push changes to.  However, everyone can also have their own repository which I think encourages them to commit and track local changes more frequently.  With SVN, especially as you work on larger teams, its tempting to commit less frequently especially if you don't use branches at all.

  • We're talking about managing daily workflow – editing, testing, and releasing
  • Have to use version control, there's no way to get by without it.
    • It's a core component, with same core concepts across different systems – checkout/commit/merge/concurrency
    • Distributed is great when stuff will eventually be thrown away, ie for individual experimentation
  • Policies – come up with general rules for your team.
    • You need to find rules that fit how you work and how you release code balanced against how you support older code, if at all.
  • Use of Tags/Branches/Trunk
    • Trunk – contains core codebase
    • Branches – used to segment into logical areas of responsibility (release/feature/programmer)
    • Tags – mark a specific state of code, a release
  • 3 main styles of branching
    • Stage Branches
      • new work is done against trunk
      • branches for staging/production/etc
      • when ready for testing, merge into staging
      • when ready for live, merge into production
      • fixes made into each branch eventually has to be made back into trunk, otherwise your changes may get deleted next time your merge from trunk to a branch
      • difficult to do parallel work (multiple people working on multiple features that may not be released together)
    • Feature Branches
      • New work is done in its own branch.  You never commit to trunk.
      • Easier to do concurrent development.
      • CON – branching might be difficult depending on your VCS.  and you have to do a lot of merging.
    • Release Branches
      • all new work done on trunk
      • create a branch for each version  when ready for release
      • easer maintenance and allows some parallel work
      • little merging
      • assumes that everyone is working towards a common goal / release
    • Pushing code live
      • Absolutely want to have a script – especially if you have a lot of servers.  Use that exact same script to push changes to testing and staging.
      • Have a way to roll it back – scripted and automated.  If something fails, better to roll back and debug in another environment.
      • Live SVN Checkout
        • it works and its simple, so its a very common practice
        • encourages conflicts, especially if some has been editing on live/production
        • hard to automate and difficult to rollback
      •  SVN Export & Rsync
        • use dcn export to make a copy of the code, use rsync to transfer changed files to live
        • simple and easy to scale & automate
        • export gets a copy without .svn directories
        •  rsync is not atomic so some hits might get partiallly updated codebase which is likely to break.

Subversion 1.5 released

The feature I’ve been looking forward to the most, merge tracking, made it into the latest Subversion release. I’m not sure why, but branching/merging has been a conceptually scary thing for me for a long time. I’d always be afraid that somehow I’d mess up the merge and lose updates or somehow mangle my code beyond repair. Nothing that a little practice of late hasn’t helped alleviate plus the following tidbits.

See what changes a merge will pull in between revision 150 and HEAD into the current working copy from another branch:

svn diff -r 150:HEAD file://var/svn/my_project/branches/new_feature

Make sure you have the merge command line right:

svn merge <b>--dry-run</b> -r 150:head file://var/svn/my_project/branches/new_feature

Submerged – Subversion Blog – Subversion 1.5.0 Released!…

I’d particularly urge you to checkout the training materials that have been created for this release. While Subversion 1.5 is compatible with older clients and servers, to take full advantage of all the features, particularly merge tracking, there are some steps you need to take. The training courses we have created cover these issues and can help guide you through the upgrade process.

Better merge tracking coming to Subversion

The more I use subversion, the more I find ways to make it do more heavy lifting for me.  I’m not talking about the kind of subversion that gets you a one way ticket to gitmo, but the open source version control system.  I use it to manage my websites here, to keep them updated with the development copies I work on at home.  We’ve also started using it more routinely at work on client sites, since it gives us an audit trail of changes. 

One complain aginst SVN has been the lack of automated merge tracking.  Its actually tedious and manual to have to merge changes from different copies, since unlike CVS, there’s no concept of a "branch" per se in SVN.  Well, version 1.5 will introduce features to automate merge tracking.

Subversion’s upcoming 1.5 release is scheduled to deliver the core merge tracking feature with subsequent releases likely extending this. Fundamental to all merge tracking related functionality is the functionality to automatically record what change sets have been copied, or merged, where.

Fixing Subversion Propfind 403 errors

I’ve been using Subversion heavily lately to keep my live and development sites synchronized and its been a huge productivity booster.  I had to move some code over to a lite site for work this evening and it was such a pain, because I had to go through and figure out the files I needed to copy.  With svn, or even cvs, I could use tags and then an update and let the computer do the grunt work.  Instead it was a 15 minute task and i broke the site twice due to missing dependencies.

Then, I wanted to do a quick update to soccerblogs.net, and needed to pull in code from another project within the repository using the svn:externals property.  I set the property and committed it, then when I tried to update my local development copy, the svn client would fail.  I had the following lines in my apache error log:

[Wed Feb 14 00:20:32 2007] [error] [client 65.23.154.104] client denied by server configuration: <PATH TO SVN>

The svn client would complaing about "PROPFIND: 403 Unauthorized".  I triple cheked my apache+svn configuration, and I could browse my repository just fine in a web browser.  It turns out that if you have mod_evasive installed, the access pattern from svn can look like a denial-of-service attack to it, so it blocks with a 403.  Disabling mod_evasive did the trick – and I didn’t really need it anyway.  None of the English language google results mentioned this, but this one in Spanish had it.  Sometimes, being bilingual pays off.

Debian Desktop Destruction and Recovery

Luckily the install is very painless and mostly automated (easier than the last windows install I did). Despite all the critiques of linux installs (and yes, it did cross my mind that a new Mac would be nice), I find myself hunting for drivers and driver CDs (especially for motherboard subsystems) than I have to compared to Windows.

That said, I have to do the following to get the PC back into a useful state – install nvidia binary drivers, setup twinview for the dual monitors, setup apache2 virtual hosts for my local development sites, install mysqll4.1, install php5, install subversion. The most painful one will be recompiling the kernel to include mppe support so I can vpn into the network at Forum One and be productive on Wednesday.

Not sure what it says about me, or my comfort with linux nowadays compared to 3 years ago, but none of the above sound all that challenging. I think its mostly because apt is such a handy packaging system, debian developers do a good job maintaing packages, so that something like subversion is just an apt-get install subversion away.

Update: at 3am, I started installing at 11:30pm, I had a working system with all of the above. MMPE support is now in them main linux kernel tree so no more patching and kernel recompiles for me!