5. Creating Git Mirror of a Subversion Repository

5.1. Installation Stages
To translate Subversion repository to Git and to enable bi-directional synchronization, one have to use SubGit to create and configure Git repository. Mirror creation consists of three stages: repository creation, configuration adjustment and installation.

Figure 5.1. Mirror Creation Stages
At configuration stage SubGit creates Git repositories and stores default mirror configuration into it. Then user has a chance to adjust generated configuration. At the installation stage SubGit performs actual translation of existing Subversion revisions into Git commits and then turns synchronization on. Figure 5.1 schematically shows SubGit installation stages.

5.2. Run `subgit configure`

Run SubGit configure command to make SubGit create empty bare Git repository and link it with a project in Subversion.

When running configure command user have to specify Subversion project URL and, optionally, minimal revision to start synchronization from:

$ subgit configure --svn-url http://svn.svnkit.com/repos/sqljet sqljet.git
SubGit version 2.0.0 ('Patrick') build #2552

Configuring writable Git mirror of remote Subversion repository:
    Subversion repository URL : http://svn.svnkit.com/repos/sqljet
    Git repository location   : sqljet.git

CONFIGURATION SUCCESSFUL

At configuration stage SubGit does not establish connect with Subversion repository, but generates configuration file with the information necessary to proceed with installation. Files added to the Git repository by the configure command are shown below:

git_repository/
  branches
  hooks
  logs
  objects
  refs
  subgit/
    lib/          # SubGit binaries
    logs/         # SubGit logs
    authors.txt   # default authors mapping (empty)
    config        # SubGit configuration file
    passwd        # Password credentials to access Subversion repository
    format        # Information on SubGit version
config            # Git configuration file
HEAD

5.3. Adjust configuration: overview

SubGit configure command creates bare empty Git repository and writes default configuration and mappings to the GIT_REPOS/subgit/config file. Generated default configuration have to be adjusted before proceeding with install stage, in particular user have to provide credentials (or specify where to find them) to let SubGit access Subversion repository. config file generated by default will look like this (some of the less frequently used options are not shown here):

[core]
    authorsFile = subgit/authors.txt
    pathEncoding = UTF-8
    defaultDomain = tmatesoft.com

[svn]
    url = http://svn.svnkit.com/repos/sqljet

    trunk = trunk:refs/heads/master
    branches = branches/*:refs/heads/*
    tags = tags/*:refs/tags/*
    shelves = shelves/*:refs/shelves/*

    fetchInterval = 60
    connectTimeout = 30
    readTimeout = 60

    auth = default

[auth "default"]
    passwords = subgit/passwd
    useDefaultSubversionConfigurationDirectory = false
    subversionConfigurationDirectory = /home/user/subversion

Paths in subgit/config file:
File system paths in subgit/config file could be specified in relative or absolute form. Relative paths are those that do not start with ‘/’ or drive letter on Windows OS. Relative paths are treated as relative to the Git repository root directory.

5.4. Adjust configuration: provide authors mapping

Write Subversion users to Git authors mapping to conf/authors.txt file (see Section 3.3, “Authors Mapping”). For instance:

conf/authors.txt:
semen = Semen Vadishev <vadishev@tmatesoft.com>
dmitry = Dmitry Pavlenko <pavlenko@tmatesoft.com>

Or specify existing authors mapping file:

[core]
...
authorsFile = /var/git/company-authors-mapping.txt

Specify path to the authors mapping file (see Section 3.3) as core.authorsFile option value. Path is either relative to the Subversion repository root (e.g. conf/authors.txt expands to SVN_REPOS/conf/authors.txt) or absolute.

Provide default domain for missing emails:

[core]
...
defaultDomain = company.com

Email has to be part of the Git author information and some of Git operations may fail when email is not present. SubGit will use core.defaultDomain value to generate author’s email when author is not found in the authors mapping or in case authors mapping doesn’t provide email.

Tip: Add author mappings for as many of you team members as possible, even if they don’t use Git as the log history will be more consistent.

5.5. Adjust configuration: path encoding

[core] 
... 
pathEncoding = UTF-8

Git stores paths as a sequence of bytes, without encoding specified. Usually paths ends up in Git as-is, i.e. in the local file system encoding. Use git.pathEncoding option to specify encoding to use when translating string path found in Subversion repository to Git. Most modern file systems use or could be configured to use UTF-8 encoding and this is default and recommended value.

5.6. Adjust configuration: branches mapping

[svn] 
... 
url = http://host/svn/repos

svn.url option value specifies project location. It is used as the root URL relative to which mappings paths are treated.

[svn]
...
trunk = trunk:refs/heads/master
branches = branches/*:refs/heads/*
tags = tags/*:refs/tags/*
shelves = shelves/*:refs/shelves/*

There are four kinds of branches that have to be mapped, each represented by its own option: trunk, branches, tags and shelves. Each option establishes SVN_PATH to GIT_REFERENCE correspondence for a particular kind of entity. Mapping syntax for trunk is:

trunk = SVN_PATH:GIT_REFERENCE

svn.trunk option allows to specify single SVN_PATH:GI_REFERENCE mapping as its value. SVN_PATH is a path relative to the project svn.url location and GIT_REFERENCE is a git reference e.g. refs/heads/master. Mapping meaning is that changes made on or below SVN_PATH in Subversion project translated to the Git commits on GIT_REFERENCE and conversely, commits on GIT_REFERENCE would be translated to the changes below SVN_PATH. Mapping syntax for branches, tags and shelves is:

branches = SVN_PATH[/*]:GIT_REFERENCE[/*]

Mapping could be either one-to-one or many-to-many, in other words when ‘*’ wildcard is used it should be present in the both sides of mapping. In order to specify multiple mappings each mapping must be defined as a separate option, see example below:

branches = branches/*:refs/heads/*
branches = release_branches/b1:refs/release/b1

Above mapping maps all branches in Subversion branches directory to the Git refs/heads namespace and, additionally, maps single release_branches/b1 branch to refs/releases/b1 Git branch reference.

Be careful when specifying mapping, it would not be possible to adjust it when synchronization is enabled. Only branches and tags that are included in the mapping will be imported and kept in sync by SubGit.

5.7. Adjust configuration: credentials

In order to create new revision in Subversion repository, SubGit needs credentials to authorize itself with Subversion server. SubGit user may provide multiple sets of credentials (username/password pairs) so that SubGit could choose with username to authorize with when creating new revision. There are a few sources SubGit looks for credentials:

GIT_REPOS/subgit/passwd file: SubGit looks into GIT_REPOS/subgit/passwd file to read username/password pairs. Passwords are expected to be plaintext. It is a good idea to limit file read access for this file to the user account that runs SubGit.

username password
username1 secret
...
usernameN anothersecret

Path to the GIT_REPOS/subgit/passwd file could be changed in SubGit configuration file:

[auth "default"]
...
passwords = subgit/passwd

SubGit configuration file: SSH and SSL client credentials could be defined explicitly in subgit/config configuration file:

[auth "default"]
...
sshKeyFile = /home/user/ssh-private-key.openssh
sshKeyFilePassphrase = secret

sslClientCertFile = /home/user/ssl-client-cert.p12
sslClientCertPassphrase = secret

Local Subversion credentials cache: To make SubGit look for credentials in the local Subversion credentials cache, set auth.useDefaultSubversionConfigurationDirectory option to true and ensure that Subversion credentials cache path is correct:

[auth "default"]
...
useDefaultSubversionConfigurationDirectory = true
subversionConfigurationDirectory = /home/user/.subversion

HTTP Proxy Configuration
In case Subversion repository access requires proxy to be configured, then auth.useDefaultSubversionConfigurationDirectory must be set to true and proxy settings should be defined in Subversion servers file.

5.8. Credentials and Subversion pre-revprop-change hook

To create new Subversion revision, SubGit has to authroize with Subversion server. In order to do this, SubGit follows the following algorithm:

Git commit author is matched to Subversion author name (AUTHOR)
All configured credentials sources are iterated to find AUTHOR’s credentials
In case matching credentials found, SubGit uses them for authorization
In case no matching credentials has been found, SubGit uses any of the available
As soon as new revision is committed, SubGit sets its svn:author revision property to AUTHOR

The most secure way to configure SubGit credentials is to provide single set of credentials (either explicitly in subgit/passwd file or implicitly by specifying local Subversion configuration path). However, use of Subversion revision properties feature requires pre-revprop-change hook to be enabled in Subversion repository. Add empty hook script with 0 exit code to enable revision properties:

SVN_REPOSITORY/
    hooks/
        pre-revprop-change     # for Linux and OS X
        pre-revprop-change.bat # for Windows

Hook file contents might be as simple as:

Linux and OS X:

#!/bin/sh
exit 0;

Windows:

@echo off
exit 0

Installing hooks to Subversion repository requires direct shell access to Subversion repository server. This might not always be available and in that case SubGit might fail to set proper author or date for a newly created Subversion revisions.

5.9. Adjust configuration: other options

[svn] 
... 
minimalRevision = 1024

Set svn.minimalRevision to Subversion revision from which translation should be started. Revisions older than specified would not be translated and would not be present in the Git repository. Limiting range of revisions with this option may reduce initial translation time.

One will find more options in the subgit/config file, as well as documentation on what these options control.

5.10. Run subgit install

Run SubGit install command to make SubGit perform initial translation from Subversion to Git and to enable synchronization.install command uses configuration previously generated by configure command. Sample SubGit install command output is shown below:

$ subgit install repository.git
SubGit version 2.0.0 ('Patrick') build #2552

Translating Subversion revisions to Git commits...

    Subversion revisions translated: 20.
    Total time: 10 seconds.

INSTALLATION SUCCESSFUL

Initial translation may take a while, during translation time Git repository remains readable, but pushes are disabled for the time of initial import. User could interrupt initial translation with Ctrl-C and resume it later by running subgit install GIT_REPOS

SubGit install command enables synchronization by installing hooks into Git repository. Existing Git pre-receive and post-receive hooks (if there were any) are preserved and executed normally.

Repository after install command is completed

GIT_REPOS/
  branches/
  hooks/
    pre-receive               # SubGit hook, do not edit or remove
    user-pre-receive.sample   # user's pre-receive hook template
    post-receive              # SubGit hook, do not edit or remove
    user-post-receive.sample  # user's post-receive hook template
  logs/
  objects/
  refs/
  svn/
  subgit/
  config
  HEAD

As soon as install command reports successful termination, Git users may start to push to the Git repository - new commits will be translated to the new Subversion repository revisions and vice versa - new Subversion revisions will be translated to the new Git commits.

5.11. Background translation process

Upon installation completion, SubGit launches background process that periodically polls Subversion repository. Whenever new revisions are found, they are translated to Git commits. This way Git repository remains up-to-date most of the time. This background process is optional and could be killed. When background process is not running, new Subversion revisions will be translated to Git commits on Git user push as part of the pre-receive hook processing. Unless disabled in configuration file, user’s push will restart background translation process.

Options that affect SubGit background process could be set in GIT_REPOS/subgit/config configuration file:

[svn]
...
fetchInterval = 60      # check for new revisions every 60 seconds
...
[daemon]
...
idleTimeout = infinity  # backgroud process never exits

Background translation process might be disabled by setting daemon.idleTimeout option value to 0. In this case translation process will exit immediately after push is completed:

[daemon]
...
idleTimeout = 0  # backgroud process disabled

To force translation of the outstanding Subversion revisions, user may use subgit fetch command:

$ subgit fetch [--async] GIT_REPOS_PATH

Alternatively invoke a no-op push from a remote working tree:

$ git push origin +:refs/heads/bogus_branch

Above command will attempt to delete non-existing remote branch and as a side effect will force SubGit to synchronize Git and Subversion repositories.

To stop background process gracefully use shutdown command:

$ subgit shutdown repository.git
SubGit version 2.0.0 ('Patrick') build #7777

About to shut down background translation process.
Shutdown request sent to background translation process (pid 4404).
Background translation process has received shutdown request and will exit NOW.

SHUTDOWN SUCCESSFUL

5.12. Recommended client-side Git configuration

Git users will clone Git repository on the server side to start working with the project locally. There are number of configuration options that is recommended to add to the clone’s .git/config configuration file. These options are not required, but merely recommended for better user experience.

Fetch SubGit mapping information:

[remote "origin"]
...
fetch = +refs/svn/map:refs/notes/commits

refs/notes/commmits reference points to the so called notes that contains Subversion to Git mapping information for each commit. Note that this fetch line should be added as a new line and not appended to any existing fetch lines. With these notes fetched, git log command will show mapping information alongside with the commit message in the log output:

$ git log
commit bee5c8ae9bd8b2077f1acaedc9982ab4bd84d91b
Author: Alexander Kitaev <kitaev@tmatesoft.com>
Date:   Thu Sep 6 18:58:01 2012 +0200

    Ugly bug fixed

Notes:
    r1 trunk

Fetch all tags:

[remote "origin"]
...
tagopts = '--tags'

–tags option will force Git to fetch all tags, even those unreachable from the current branch. These may be convenient in case user would like to see or check out tag set on the other branch.

Space shortcuts

Page tree