This article, like all of the ones in the System Administration Category, is for system administrators. It is about setting up a kick-butt version control system for web development, using Subversion.

After a brief overview of the Subversion Book, I thought that given a couple of hours, I’d be able to set up a subversion repository on my development server, and access it using WebDAV. Over SSL. Authenticated to my LDAP server. With auto-commits and automatic updates to the working dir (which is a website). I was totally wrong about the couple of hours part.

Four days later, I’ve learned a lot about Subversion and WebDAV, and a bit more about Apache 2. This article summarizes what I learned; I hope that it saves you some time!

I’f you’re only interested in solving the OS X dot file problem, then you should skip directly to the solution.

What follows is a summary setting up version control for web development. I’m going to skip the tedious part about compiling and installing Apache 2 with OpenSSL, WebDAV, and LDAP integration. There is plenty of information about these things available. My focus is on getting Subversion to play nicely with WebDAV, OSX and Windows XP. So I’m going to assume that you have a working Apache install with all of the relevant modules, and that you have Subversion installed.

Let’s get into it. By the way, my development server runs Linux, and so my examples will assume a Linux platform.

Create a Repository, or Two

The first step is to create a place on your filesystem for your subversion repository. I plan to have multiple repositories, so my setup will have a root path under which I’ll create my repositories.

sudo mkdir /home/svn
sudo chown brentk /home/svn

I changed the ownership of the directory to my own name for now, to make creating repositories easier.

Next, I’ll create a couple of repositories:

svnadmin create /home/svn/myproject
svnadmin create /home/svn/myotherproject

Ignore the OS X Files for the Command-line Import

Keeping the OS X dot files out of auto-committed WebDAV shares is trickier, and I discuss that below

For the command-line import of files into your repository, you should update your subversion config file to explicitly ignore the Mac OSX dot files (even though according to documentation, .DS_Store files are ignored by default…). The config file is in ~/.subversion/config, and the line to uncomment and change is this:

global-ignores = *.o *.lo *.la #*# .*.rej *.rej .*~ *~ .#* .DS_Store ._* .Trash*

You don’t want .DS_Store, ._filename or .Trashes files & directories in your subversion repository.

Import Your Data

Now that your config file is setup to ignore useless files, its time to import some files into the repository:

cd /home/html
ls -1
  myotherproject/
  myproject/
  mythirdproject/

svn import myproject file:///home/svn/myproject/trunk -m 'Initial Import'
Adding         myproject/contact_us
Adding         myproject/contact_us/index.php
Adding         myproject/styles
Adding         myproject/styles/main.css
Adding         myproject/images
...

Committed revision 1.

The svn import command took 3 arguments: 1) the name of a directory that contains the files to import (”myproject”), 2) the subversion repository (”file:///home/svn/myproject”) and directory within that repository (”/trunk”) in which to import the files, and 3) a comment (”Initial Import”). You can leave out 3) and it will open your text editor so that you can type a comment.

Setup a Working Directory

In my case, the files in the “myproject” directory make up a website, and I want those files to be automatically updated when new versions are checked into the repository, so that the changes appear automatically on the webserver. How to do this is mentioned in the Subversion FAQ, and we’ll get to the specifics soon enough. For now, we need to change the directory that we just imported into a working directory. If you’re not familiar with what a “working directory” is in the version control context, then you should stop now and read the first chapter of the Subversion book, entitled Fundamental Concepts.

Since the files in the myproject directory are committed to the subversion repository, its OK to delete the originals. So that’s what I’m going to do, and then I’ll checkout that directory from the repository so that it is a working directory:

rm -rf myproject
svn checkout file:///home/svn/myproject/trunk myproject
...
A    myproject/about/index.php
A    myproject/availability
A    myproject/availability/index.php
A    myproject/index.php
Checked out revision 1.

The svn checkout command took two arguments: 1) the repository & directory to check out the files from, and 2) where to put them (into a directory named “myproject”).

Prepare the Repository for WebDAV

If you are going to setup autocommit with WebDAV, then the webserver process will need permission to write files to your repository. If you still plan on using the command-line svn client to commit data to the repository, then you’ll need a setup where both the apache process and your users can write to the repository. There are a number of ways to do this. One of them is to make the repository owned by the apache process and group-owned by a group to which your developers are members. Then make sure the files and directories are group-writable:

sudo chown -R apache:devgroup /home/svn/myproject
cd /home/svn/myproject
find . -type f -exec chmod 664 '{}' ;
find . -type d -exec chmod 775 '{}' ;

Set up an Apache VirtualHost

My Apache 2 configuration will consist of two vhosts: one for serving the website in my /home/html/myproject (working) directory, and one for providing WebDAV services for my Subversion repository.

This vhost definition points to my working directory, which will be automatically updated when a commit is made to my subversion repository, via webdav auto-commit.

NameVirtualHost 123.45.678.90:443
<Virtualhost>
	ServerAdmin nospam@netmojo.ca
	ServerName myproject.netmojo.ca
	DocumentRoot /home/html/myproject
	UseCanonicalName On
	ErrorLog /var/log/apache2/myproject.netmojo.ca-errors
	CustomLog /var/log/apache2/myproject.netmojo.ca-access combined

        SSLEngine on
        SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL

        SSLCertificateFile /path/to/my/certs/myproject-cert.pem
        SSLCertificateKeyFile /path/to/my/certs/myproject-key-unenc.pem
        SSLCACertificateFile /path/to/my/certs/cacert.pem

        <Location />
                DAV svn
                SVNParentPath /home/svn
                SVNListParentPath On
                SVNAutoversioning on

                SSLRequireSSL
                AuthName "NetMojo Development Area"
                AuthType Basic
                AuthBasicProvider ldap
                AuthLDAPURL ldaps://ldaphost.netmojo.ca/dc=netmojo,dc=ca?uid?sub
                AuthLDAPBindDN "userid=ProxyUser,ou=sysusers,dc=netmojo,dc=ca"
                AuthLDAPBindPassword "MyLDAPProxyPassword"
                require ldap-user user brentk sam joe sally marry
        </Location>

         BrowserMatch ".*MSIE.*" \
                 nokeepalive ssl-unclean-shutdown \
                 downgrade-1.0 force-response-1.0

</Virtualhost>

The Problem with WebDAV Autocommits

When I first tried following the guidelines in the Subversion Book, and the WebDAV and Autoversioning Appendix, WebDAV worked, but I was plagued by a problem with OSX: When I wrote files to the WebDAV share, OSX created .DS_Store files, .Trashes directories, and add a ._filename file for each file that I altered. The result was for each file change, it was making 17 commits (versions).

This happened despite the global-ignore rules, because when you autocommit via WebDAV, or actually via the dav_svn module, your ~/.subversion/config file is ignored. I even tried using propset, as mentioned on various mail lists:

cd /home/html/myproject
svn propset svn:ignore -F ~/ignores .

Where ~/ignores is a text file with one ignore pattern per line:

cat ~/ignores
.DS_Store
._*
.Trash*

This gets ignored by dav_svn too. I also found some references to a global subversion config file, in /etc/subversion/config. I tried copying my ~/.subversion/config to there, and to /usr/local/etc/subversion/config, but that also gets ignored when using autocommit with WebDAV.

Windows XP has its own bag of problems. When I tried to add my WebDAV share as a “Network Place”, it would fail with the informative message, ‘The folder you entered does not appear to be valid.’

On the server, I could see that it would connect, negotiate the SSL properly, authenticate properly, but then I would find these in my access logs:

[02/May/2007:11:54:26 -0700] "GET /_vti_inf.html HTTP/1.1" 403 215
[02/May/2007:11:54:26 -0700] TLSv1 RC4-MD5 "GET /_vti_inf.html HTTP/1.1" 215
[02/May/2007:11:54:26 -0700] "POST /_vti_bin/shtml.exe/_vti_rpc HTTP/1.1" 403 229
[02/May/2007:11:54:26 -0700] TLSv1 RC4-MD5 "POST /_vti_bin/shtml.exe/_vti_rpc HTTP/1.1" 229

WTF!?! Leave it to Microsoft to POST to a WebDAV share…

I noticed that the server was throwing back 403 (forbidden) errors, which it shouldn’t have. So I added a line to the apache config change that into 404s. This made no difference to the failure to mount the share, but it is more technically correct, so why not.

The Solution to the Problem

The solution to the OSX dot file and XP problems is to use Apache’s Rewrite feature to disallow PUT requests from OSX for those dot files, and a series of directives to make Apache play nicely with Windows XP.

<Directory /home/svn>
        Options Indexes SymLinksIfOwnerMatch IncludesNoExec
        IndexOptions FoldersFirst FancyIndexing XHTML DescriptionWidth=* NameWidth=* IgnoreClient 

        # Deny OSX dot files
        RewriteEngine On
        RewriteCond %{REQUEST_METHOD} ^PUT$
        RewriteRule .DS_Store - [F]
        RewriteRule .Trashes - [F]
        RewriteRule .TemporaryItems - [F]
        RewriteRule ._.* - [F]

        # Fix broken Windows XP
        RedirectMatch 404 ^/(MSOffice/|_vti_bin/|_vti_inf.html$) 

        BrowserMatch "^WebDAVFS/1.[012]" redirect-carefully
        BrowserMatch "Microsoft Data Access Internet Publishing Provider" redirect-carefully
        BrowserMatch "Microsoft-WebDAV-MiniRedir/5.1.2600" redirect-carefully
        BrowserMatch "^WebDrive" redirect-carefully
        BrowserMatch "^WebDAVFS" redirect-carefully

        <ifmodule>
            Header add MS-Author-Via "DAV"
        </ifmodule>
</Directory>

With this setup, I got 1 commit per file change in OSX. Microsoft Windows XP could mount the WebDAV share as either a “Web Folder” (by File -> Open in IE) or as a “Network Place”. It could delete and replace files, but it refused to open files on the WebDAV share. So files could be copied off, changed, then copied back. From what I’ve read, this due to a generally buggy implementation of WebDAV on Windows XP. Apparently it works on Windows 2000, but I haven’t tested that, nor do I have a need for it.

A work-around is to use a free program that used to be provided by Novell, named NetDrive. It will allow you to connect to the WebDAV share like a normal network drive and work on it. It might be diffcult to find though, since Novell sold it and is no longer distributing it. WebDrive is a commercial product based on NetDrive, which works well.

*** Update: The above works great for editing existing files, but I discovered that the rule:
RewriteRule ._.* - [F]
…causes OSX clients to produce a “permission denied” error when one tries to drag & drop new files into the WebDAV share. I can save a new text file with TextMate, but using Finder to add new files doesn’t work. I’m still looking for a work-around; if you know of one, please comment.

Setting up Automatic Updates

Once subversion is working properly and auto-commiting webdav changes, its time to implement automatic updates to the working directory, which is also in my case the DocumentRoot for a vhost. As per the Subversion FAQ, this is done via a post-commit hook. A hook is a shell script in the hooks sub-directory of your repository (/home/svn/myproject/hooks). In that directory are some templates. Copy “post-commit.tmpl” to “post-commit”, and open it with a text editor. Comment out the last couple of example lines.

I tried the C program that is mentioned in the FAQ, setting it to +s so that it would run as the user that owns the working directory. It worked well from the command line, but it failed as a post-commit hook. The reason was due to the special characters in some of the filenames in my project. When I run it from the command line, my LANG environment variable sets the character set to UTF-8, so it works. When apache executes it, it runs with a different character set, and problems ensue.

Since, in this case, the site is password-protected, I decided to give apache write permission to the working directory, and run the update directly from the post-commit script, which includes the setting of the LANG environment variable. My post-commit script is:

#!/bin/sh

export LANG="en_US.UTF-8"
umask 002
cd /home/html/myproject
/usr/local/bin/svn cleanup
/usr/local/bin/svn -q --non-interactive update

Now when a developer writes a file to the WebDav share, one and only one commit is made to the Subversion repository, and the new version is automatically updated in the working directory, which is my development site. Awesome!

Tags: OS-X, Subversion, Version-Control, WebDAV, Windows-XP

This entry was posted on Thursday, May 3rd, 2007 at 1:27 pm MST and is filed under Articles, System Administration. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.