At first glance, you'd conclude the process would work much the same way: Loop through the individual directories in the large repository, create smaller repositories for each one, then dump and import the contents of each directory into its small repository.

The tricky part is that the Subversion dump command dumps everything in the repository, by revision. In order to pull just a single directory, you must filter a complete dump with the "svndumpfilter" command. This blog post by AllMyBrain.com basically explains how to accomplish this in Linux. I usually have to work on a Windows box on the job, so I wrote up a script to accomplish this in a Windows batch script.

The strategy is the same as the Linux script, though. We're going to use "svnadmin dump" the large repository, then use "svndumpfilter" to filter by just the directory we want, then "svnadmin load" the results into the newly created repository. All of this can be combined into a single statement via piping:

This will make a little more sense when we look at the full script. Let's just put it out there and then go through it.

First, we're setting our paths. "SmallRepoPath" will be the directory holding all of the small repositories we'll be creating. "PathToRepo" and "UNCToRepo" point to the big repository as DOS and UNC paths, respectively. "PathToChkout" points to a Subversion checkout of the large repository.

First, we check out the large repository with the "svn co" command. We do this just so that we can call the "dir /A:D /B" command, which says, "List just the directories in the checkout directory." We use that output to loop through each directory in the large repository.

Then, for each directory in the large repository, we create a corresponding small repository, then do our dump/filter/load combo. Again, we're dumping the contents of the large repository, using "svndumpfilter" to filter by directory, then loading that filtered dump into the new small repository.

Finally, we just do some cleanup by removing our temp files and the checkout directory.

There are a few caveats with this code.

First, it will import all of the large repository's revisions into the smaller repository. There are svndumpfilter arguments to prevent this, such as --drop-empty-revs and --renumber-revs, but I found the Windows Subversion binaries to be problematic with these arguments. The end result is that you have more revision numbers than needed, but only the relevant data is actually imported into the repository, and viewing logs on just the imported directory will still obviously show revision logs related to that directory, so there's really little harm done.

Second, the dump/filter/load action doesn't always work on a directory that has been moved (copied/deleted) from another location within the large repository. What's worse, it won't fail, it just won't load any data into the small repository. To address this, use the --revision argument on the "svnadmin dump" command to do a dump starting at a revision after the move took place. Doing so will give the "svndumpfilter" command something it can work with.

This process is certainly more complicated to explain, but ultimately there's not that much more going on. Hopefully this explanation is helpful to you.

That said, there are advantages to having one single repository. No big deal, that can be done after the fact with code.

Here is some Windows code to combine all the repositories in a directory into a single big repository:

There's really not much happening here; the process is simple. First, we create the new "big" repository with the svnadmin create statement. Second, we loop through the directory, processing each Subversion repository in the directory with a three-step process: (a) Dump the repository with the svnadmin dump statement into a temporary *.dmp file. (b) Explicitly add a new directory in the "big" repository for the current repository we're processing, with the svn mkdir statement. (c) Import the dump into the "big" repository with the svnadmin load statement. Really, the rest of the code is just looping, commenting, or cleanup code.

What have we produced? As you might expect, we now have one big repository that has all of the files and commits that were in all of the smaller repositories. The big repository will maintain its own revision numbering, so the revision numbers in your smaller repositories will not match the big repository's revision numbering, although the original commit dates will be preserved. This can be really handy for searching or similar actions that you might do from a more global perspective.

However, this approach is not without its caveats. During the import process, one entire repository is imported at a time. All of a particular repository's revisions will be "grouped" together in the big repository. As a result, revision numbers in the big repository will change every time you recreate it, if there was any new activity in the repositories it contains. For instance, revision #1050 in the big repository may parallel revision #500 in Repository X, but if a commit was added to a repository that is imported before it and the big repository is recreated, that revision would now be #1051. Additionally, although all history and dates are preserved in the revisions, the big repository has commits that are not in chronological order since the import was processed by repository. This inconsistent date/commit ordering can be confusing to some repository reporting tools and may actually render those tools useless to you when they are reporting by date.

Filtering by revision. Note that my svnadmin dump statement includes the -r argument, which specifies the beginning and ending revisions to dump. By default, I'm using "0:HEAD", which basically means "dump every revision", or "dump from the first revision to the HEAD, or latest, revision". Changing the beginning and ending revisions can be useful, especially when used with dates instead of actual revision numbers. For instance, you could change the value to {2007-01-01}:{2007-12-31} to only dump revisions that were committed in 2007.

Combining all of your smaller repositories into one big repository after the fact isn't a perfect solution, but it can be handy, and it's really easy to do when you have a script like this ready to run.

As it turns out, there is a 301 error section in the Subversion FAQs. It says that this typically means your Apache configuration is invalid (nope, the rest of my repositories worked just fine) or your repository has the same path as a literal directory on your web root. Ahhh!

Sure enough, my subversion path was http://myserver.com/xyz/, and I had a literal directory named "xyz" in the web root. I changed that directory name, and Subversion would then allow me to checkout the repository with no problem.

Using the Windows binary installer, I had no problem installing Subversion v1.4.5 on the server. I tested everything and Subversion still worked, it showed the new version when accessing via web access, and Trac still worked fine.

Alas: Don't forget that an upgraded version of Subversion will not upgrade your repository. It will upgrade a working copy of a checked-out repository, but it will not upgrade the repository itself.

That said, I was unaware of one more step that you must take to upgrade Subversion on a Subversion/Trac setup: You must also upgrade the Python bindings to Subversion.

This became apparent the next time I created a new repository, which was not a v1.4.x repository, and when I tried to build a Trac environment to point to it, Trac got upset because of the classic "Expected version '3' of repository; found version '5'" error. To fix this, you must set up new bindings to the new version of Subversion, as explained in the TracSubversion page.

Now, I obviously love Subversion and I love Trac, but honestly, straight-forward documentation that is easy to understand for someone who doesn't want to get in the thick of it isn't really the strong suit for these communities, at least when it comes to installation and deployment on the server. What exactly it means--and how to do it--when they say, "Update the Subversion bindings" is not easy to understand. However, the solution is simple. All that is needed is to download the appropriate "svn-python" Windows installer that matches your version of Subversion and Python (in my case, 1.4.5 and py2.3) and run it on the server.

In my case, I had to stop Apache for the installation to succeed. Upon restarting Apache, everything worked great.

C:>svn co http://MyServer/Path/To/Proj/ MyProj
svn: REPORT request failed on '/Proj/!svn/vcc/default'
svn: REPORT of '/Proj/!svn/vcc/default': 400 Bad Request (http://MyServer)

Yeah, that's not cryptic. Fortunately, the solution is simple. Sander Striker explained in a thread REPORT request failed ... 400 Bad Request that if you're behind a proxy at work, and the proxy isn't configured to support the necessary Subversion calls REPORT, your request will fail.

Like he says in his message, you could request that the proxy be configured to allow the necessary requests, but you could just as well configure your server to work on a different port. Now, I like the fact that my Subversion calls can work on port 80, and I don't want to change that. So, I configured Apache to have my Subversion sites work on an additional port.

In the following example, let's use port 81 as the additional port. So my example URL http://MyServer/Path/To/Proj/ would become http://MyServer:81/Path/To/Proj/.

At the following spots in your httpd.conf file, make the one-time additions as marked in bold italics:

Listen 80
Listen 81

And...

NameVirtualHost *:80
NameVirtualHost *:81

And for every virtual host entry you have for a Subversion site (I host a couple different Subversion sites), add *:81 in the VirtualHost header:

< VirtualHost *:80 *:81 >

After restarting Apache, you will now be able to continue to use the URLs you normally use, but anytime you need to checkout from the repository while at work behind a proxy, you can use port 81 to do so successfully.

Cannot load /etc/httpd/modules/mod_dav_svn.so into server: /etc/httpd/modules/mod_dav_svn.so: undefined symbol: dav_xml_get_cdata

Please note! Some people have indicated that this is because Apache wasn't configured with DAV support when it was compiled on your distro. The answer might be a lot simpler than that.

Garrett Rooney noted that it might be as simple as just making sure that mod_dav is loaded before loading mod_dav_svn! I was experiencing this error, and a simple rearrangement of my LoadModule commands in httpd.conf fixed it.

Summing it up as a glorified backup app is an obscene oversimplification. As a developer, I immediately recognized in Time Machine the likelihood that it is running on top of a revision control system, and that is what really fascinates me about it, and incidentally that is also what I feel many people misunderstand about it, leading them to discount it. I'm not alone in making this connection; really, any developer who uses and understands source code repositories probably had the thought cross his or her mind. One of the major advantages mentioned in discussions about revision control is the fact that you can always retrieve your file as it existed at any point in the past when it was committed to the repository.

And it's not a stretch to see Apple borrowing revision control software to implement Time Machine. Subversion is a very popular revision control project in the opensource community. Sitting on top of FreeBSD, Apple has incorporated many other opensource projects in Mac OS X. For instance, OpenGL, Apache, SSH, FTP, CUPS, and Samba come to mind, just to name a few. Why not add Subversion to the list of excellent technologies to incorporate into an excellent OS?

If Time Machine is running on top of Subversion or a similar revision control system, that would address the concerns that some people have--people who might not understand how revision control systems work. Some people have expressed a concern that a backup drive would not be able to support Time Machine for very long before filling up. For instance, if you have a 1GB file (perhaps a video clip) and make a very small change to it, will Time Machine record that change by making another copy of the entire file? If Time Machine made its backups as full copies of the changed files, you can imagine how quickly a backup volume would fill up. But that simply isn't how most revision control software works. Most revision control software, including Subversion, uses Delta Compression to calculate only the changes made to a file, and then saves only those changes. Thus, a file can be committed to a repository a dozen times, and yet the amount of drive space taken up would only amount to the changes made, plus perhaps a very slight amount of overhead in the repository for tracking each change.

So that covers feasibility. With some software like Subversion, Time Machine could provide a backup of your entire volume and actually support it for a decent period of time thanks to efficient delta compression. What about the logistics of finding the changed files and saving those changes? This would be simple even with Subversion in its present state. Out of the box, Subversion provides the ability to search for files that have changed since the last "commit", or the last time changes were saved. Time Machine would just have to ask Subversion to report all the files that have changed, then run the commit. This could be scheduled to execute at a given time, say, midnight. In the event that the computer was off at the scheduled time, the steps could be executed immediately upon startup.

In the past, I've thought how nice it would be if all the files on my computer could go under revision control just like my source code when I'm working on an application. It would be just Apple's style to make that possible not just for tech geeks who use Subversion, but to make it possible for anyone, even the guy who doesn't care a lick about understanding revision control.

Put Those Development Files in Version Control!

To give yourself the best organization within your web application project's repository, be sure to take all aspects of your development into consideration, not just the organization of the deployable code itself. For instance, just like in compiled application development, you may have some workfiles that ought to be in the repository: Photoshop images, Flash project files, uncompressed audio or video, and similar development files. Or perhaps unit testing code, which need not be deployed on your production server.

There's nothing worse than realizing that you no longer have these development files when the time comes to make some requested changes. It is definitely wise to include them in your repository. So why not have a subdirectory structure under trunk separating your dev files from your web files? Perhaps something like this:

• trunk
• dev
• www

Keep Them Separated 

Furthermore, in some cases, you may have one web application, but portions of it reside on different websites. For instance, perhaps you have a public-facing web application on your company's public website, but the administrative portion resides on your intranet site. Or perhaps the web application is just one module of a larger project whose technology reach is more broad than just the web. It would be wise to have that clearly defined in your repository structure as well.

Multisite Example.

Multitech Example (VB and web app combination).

Clearly Mark the Live Release 

One aspect of web application development that is really handy with revision control systems like Subversion is that you can also use the revision control system to help with deployment of the application. In fact, you can even use it to help keep track of which version of the app is a "live" release.

In order to accomplish this, you can check out your web application directly onto your webserver. This can be difficult if you only have FTP access to your server, although it can be done with utilities that map FTP connections to local paths on your computer. However, if you have command-line access or network access to your server, checking out files from the repository directly onto your server should be very similar to checking out files on your workstation. Please note that checking out a working copy of your repository onto your webserver is a security risk unless you supply your webserver a directive to disallow read access to the .svn directories that Subversion creates in the working copy. The Website Releases via Subversion article at RedBalloon Labs explains how to do this nicely.

So, what is the benefit of checking out a working copy onto your server? If you have a QA server you will be testing your code on, then perhaps you will check out the web application in the trunk directory. Once you've checked out the code, all you need to do is run an update against the working copy on the server to update it to the latest trunk code!

That's great for testing purposes, but how can we garner the same benefit for our production server? When testing is completed, you will likely tag a version of trunk in the tags directory. At this point, you could check out the version in the tags directory onto your production server. However, this course has a couple caveats: (a) It assumes you will always have the latest tagged release on your production server. (b) It will require a switch command to migrate the server code to the new version. If you're fine with these considerations, then by all means, check out the latest version from the tags directory and consider yourself done.

I, on the other hand, am not comfortable with those conditions. If your company has any sort of delay between the moment technical work has been completed and when it can actually be deployed, then a tagged version may not be the version on your production server, even though it is the latest and greatest tagged code. Certain versions may never even see any time on the production server if they are bug fix releases that are held back until perhaps more bug fixes are made and a later version is deployed as a cumulative release. Furthermore, if someone else needed to redeploy the application, they should be able to know what version to redeploy.

The solution to these problems is to create a copy/branch of the tagged version as a copy called "live" or something similar. You could keep the "live" copy in the tags directory, but I found it more straightforward to keep it at the repository's root level. This will mean your repository's root might look like this:

• trunk
• branches
• tags
• live

Live being, not an actual directory, but simply a copy of a version in the tags directory, which is actually a copy of trunk at a particular revision! By following this approach, it will always be clear to anyone how to deploy your website or web application onto the production server. All they have to do is check out the live directory at the root of your repository.

Furthermore, updating is a breeze on the server end. Once you are set to change live to a new version, simply delete live and make a copy of the new version. Then, on the server, all that needs to be done is to run an update on the working copy, and all of your changes will fall into place. I love this part of the process. It feels so clean and inobtrusive, which is naturally important for a production deployment upgrade.

Repository Structure Overview

If we were to employ all of these tips, our repository might look something like this:

• trunk
• dev
• www
• public
• intranet
• branches 
• tags
• v1.0 (Copy of a revision of trunk)
• v1.0.1 (Copy of a revision of trunk)
• live (Copy of a version in tags)

For many of us, simply taking the step of using a revision control system is a great acomplishment alone. If you're ready to begin leveraging revision control for web development even further, then the tips we just covered might be helpful to you. I invite you to share with me any standards or practices that you are employing in revision control that I haven't covered here. I'd love to learn about them.