New Subclipse Build Posted -- Dialog Improvements

I just posted a new development build of Subclipse - 1.3.3. You can currently only get these builds as part of the CollabNet Merge Tracking Early Adopter program. The reason being that these builds require development build of Subversion 1.5 and I am coordinating these builds so that they use the same Subversion binaries (to make it easier to test).

This new build includes some dialog UI improvements I have been wanting to make for a long time. I have really grown to like the simplicity of the CVS commit dialog and have heard comments many times about the general usability difference between the CVS plug-in and Subclipse. So the intent here is to close the gap some more and incorporate some of the same UI while maintaining the Subclipse features that we can. Here is a fairly complicated example that shows most of the features:

There are a few of major changes.

1. The dialog uses a wizard-style UI which is pretty common in Eclipse. This gives us a chance to include a graphic and just generally make the dialog look better.
   
2. The presentation of files has changed from a table with checkboxes to the more friendly and graphical mode that CVS uses. There are three presentation models to choose from.
   
3. Because we no longer have a table to show data and text we needed a way to show when there are Subversion property changes. We are using a second decorator to do this. Currently we only use this in these dialogs, there are no plans to do this in Eclipse views.
   
4. The biggest change is that you now have to right-click and use Remove from view to not commit something. You used to be able to uncheck a check-box.

Earlier this year I wrote a post that detailed the Features of the Subclipse Commit dialog. You can review that post if you want to compare the differences in the UI. Here is another screenshot that is a little simpler to give another taste of the changes.

I think everyone will agree the dialog looks better. I think where there might be some controversy is in how you decide to not commit a certain file. This new approach is definitely optimizing for the scenario where you typically commit everything in the dialog. Users that work mostly from the Synchronize view, as an example, should really like this better.

Personally, I find this approach more usable. Even though it is a little more difficult to right-click and remove something than it was to uncheck it, the fact that the item no longer shows up in the view makes it more obvious what is going to be committed.

The Revert and Lock dialogs got the same treatment:

Other dialogs like Switch and Create Branch/Tag also received the new wizard look. Please give these builds a try and let me know what you think. The best places to reply would be the Subclipse users@ mailing list or in the issue tracker for this issue 682.

Introducing the CollabNet Merge Client

I have not posted to this blog in quite a while. I have been thinking about using it to blog some personal thoughts (like on the Red Sox) but I know it is syndicated in a few places for its Subversion/Subclipse content so I haven't. Instead, I have been doing all of my Subversion blogging over on the Submerged blog at openCollabNet.

It has been a busy summer. I have been more involved in the Subversion community than at any time in the past. Earlier in the summer I was made a full committer which was a proud moment and I have been doing a lot of work to help drive us to the final 1.5 release. Beyond all of that, what I have really been working on the most, however, is the development of a new GUI merge client for Subversion to coincide with the Subversion 1.5 release. Yesterday, we officially announced the client and released an early version. You can read the blog post here, or skip right to the details here.

The CollabNet Merge Client is built on top of Eclipse and Subclipse and has the simple goal of making merge easier. I think it does that and a lot more. It has really turned out great. There are details on the web site, so I will not reiterate them here.

As part of developing this client, we have also made a lot of really nice enhancements to Subclipse. We have focused on taking advantage of all of the great new features that are coming in Subversion 1.5 and also on general usability. I think the final result is going to be great.

The Subversion community is working hard to get the Subversion 1.5 release process started by mid-October so that it can be GA in December. The GA of Subclipse 1.4 and the CollabNet Merge Client will be at the same time.

If you use Subversion and Eclipse, even if you do not use Eclipse, I would encourage you to check this out and give it a try. It requires using the latest build of Subversion 1.5 on the client, but that is not as risky as it sounds, and we will be refreshing the client with new builds as needed. We are also providing Linux JavaHL binaries to make it easier for Linux users to try. Currently only OSX users are shut out, which is ironic because I do all my development on OSX. We could probably make a 1.5-trunk installer for OSX but it would take some work to make it co-exist with the existing 1.4.4 client for OSX. I know Jeremy has been busy so I have not asked him to do the work required. At worst this problem will go away when 1.5 is GA, but hopefully we will have something available before then.

CollabNet Desktop - Eclipse Edition

There was one other aspect to this, yet another thing I have not blogged about. That is the CollabNet Desktop - Eclipse Edition. This is a unified set of tools to access all CollabNet products from Eclipse. We include Subclipse for accessing CollabNet Subversion repositories. There is also a small UI layer on top of this to integrate into the CollabNet perspective and sites view.

It also includes connectors for Mylyn to integrate the issue trackers available in CollabNet Enterprise Edition (CEE) into Eclipse. There are a lot of popular open source sites that use CEE such as tigris.org, java.net, openoffice.org, netbeans.org, dev2dev.bea.com etc. Now the issue trackers for all of these sites are available in Eclipse and Mylyn.

We are also currently working on a Mylyn connector for the issue tracker in SourceForge Enterprise Edition, which is now a CollabNet product. Note: This is not the same product that runs on sourceforge.net.

Last but not least, we also provide integration and access to CollabNet CUBiT. This is a really cool product and the integration came out great. CUBiT allows you to virtualize your build and test environments. From Eclipse, I can provision a server in our CUBiT environment, this includes picking the operating system to run and possibly a set of other components I want to install such as MySQL, Apache etc. These servers are accessible via SSH, and the CollabNet Desktop client makes it simple to access the servers and setup port forwarding to access services running on the servers etc. We integrated the SSH Terminal from the Eclipse DSDP/TM project to provide terminal access. We also support launching external clients but none are required.

I use CUBiT to dynamically provision servers to test Subversion releases (which I have to test on Windows, Linux and Solaris), but most people are using it to build, run and debug server-based web applications. Anyway, the CollabNet Desktop was just another of the many things I have been working on this summer and it went GA a few weeks ago.

It has been a busy summer.

Unstoppable Subversion

Lots of Subversion news this week as the popularity and adoption continues to soar.

First, Forrester Research released their latest Forrester Wave where they analyzed the Software Configuration Management marketplace. For the first time, they included Subversion in their analysis, and what do you know, but Subversion actually came out on top for standalone SCM! They also analyzed integrated change management systems and included Subversion. It did surprisingly well considering that it really does not specifically address that space. You can download a copy of the report from CollabNet.

Next, the latest SecuritySpace.com numbers came out for the month of May and Subversion adoption continues to soar. See this chart for more information.

Finally, I participated in a webinar this week with Carey Schwaber of Forrester and my CollabNet colleague Auke Jilderda. You can view a replay here. This was my first time doing a webinar. I will probably try to make a post about the experience. It was a lot of work for 10 minutes and of course I screwed up the slides when control was transitioned to me. That really flustered me but I recovered. I just got up the courage to watch the replay this AM and it was a lot better than I thought, I actually came in at exactly 10 minutes. I am not sure how eager I will be to do another one anytime soon though because it just took up too much time to prepare. It is hard to do these well if you are not a pro speaker because you are so constrained for time. Anyway, the first 20 minutes of the webinar is Carey going over her report, the criteria used and an overview of SCM for distributed development teams. I then gave a quick overview of Subversion and why it is popular before covering the upcoming 1.5 release and some of its features. Auke then finished things off with an overview of the merge tracking feature which is the defining feature for 1.5. We then concluded with a fairly lengthy Q&A session. There were over 500 people connected which I understand to be a very large audience for one of these events. Clearly interest in Subversion is skyrocketing.

Subversion binaries for OS X

I am pleased to report that we are now providing OS X binaries (including JavaHL) on openCollabNet. This is an issue I raised before and soon after beginning employment at CollabNet, so I am glad to see it become a reality. Lack of a complete binary for OS X (including the JavaHL library) was a bit of a problem since Subversion 1.4 was released. There were some command line packages available, but nothing with JavaHL. I know this caused pain for a number of Subclipse users.

Thanks goes to Jeremy Whitlock for investing his own time and effort to get a Universal Build created, as well as learning the ins and outs of Mac packaging. I have been running this package on my Mac (which I now work on most of the time) for several weeks.

The other part of this news is that we are going to launch a project on openCollabNet to be the home for "community-driven" binaries. Basically, we can provide the bandwidth to host the downloads as well as community forums to solicit feedback etc. CollabNet does not have the resources to officially support every operating system users might be using, but this gives us a way to assist the community in providing quality binaries and also can allow us to gauge interest in the operating systems we are not officially supporting. I will follow-up to this post when the project launches.

In the mean time, I hope all you Mac users out there will download these binaries and try them out. You can post any questions or issues in the Subversion users forum on openCollabNet.

Subversion blog posts to read

I have been real busy with the new job and have not had time to blog. When I have had some time, it went into blogging over on the Submerged blog. Anyway, here are some links to some good blog posts I have seen recently about Subversion.

On the Submerged blog:

Computing the Differences Between Tags by yours truly :)
In this post I show the new svn diff --summarize command and how you can use it to get a list of files that changed between two repository locations.

Authz and Anon Authn Agony by Mike Pilato
In this post Mike explains a difficult problem you can run into when you want to provide anonymous access but also have private folders within the repository structure.

Multiple Subversion Repositories? by Guido Haarmans
This post touches on the issue of having multiple replicated repositories to support globally distributed development. More specifically, why you may not really need this with Subversion as you do with other tools I will not mention.

Subversion LDAP Authentication with Apache by Jeremy Whitlock
Jeremy gives a primer on how to configure Apache to authenticate users via an LDAP directory, such as Microsoft Active Directory.

How Subversion Conserves Disk Space by Guido Haarmans
This is a high-level overview of how the Subversion repository stores your data.

Finally, I have been meaning to link to another set of posts I have looked at. On his blog TrajicklyHip, Aaron West has written a whole series of detailed posts about setting up a Subversion environment and then working with various clients. He has good links within the posts themselves, so I will just link to my favorite, the one about Subclipse. :) I am fairly experienced with Subversion and these posts are aimed at getting people started, so I cannot say that I have poured over every word in detail. That being said, it all looks quite well done and what I read was accurate. I would recommend you check it out.

First blog post and article on openCollabNet

I have agreed to participate in the Submerged blog on openCollabNet and made my first post recently. I guess they did not read this blog too closely and notice my penchant for really long posts. Anyway, we decided to make the post into an article instead. The article is about installing the svnserve server option to run as a Windows service and you can find it here.

My first blog post then just became an introduction to the article. You can find that post here. If you have any comments on the article, you can leave them on the blog post. (Yes, that is four links to the same post if you are counting).

In the future, I will probably do more of my Subversion-specific posts on the Submerged blog and do the Subclipse posts here. I will figure it out as I have things I want to write about.

Incorrect Article about me at EclipseZone

There was an article posted today at EclipseZone (which I am not going to link to) announcing that Subversive has been approved as an Eclipse project. Congratulations to them, although this actually happened a couple of months ago so I do not know why it is news. Anyway, the article includes the following quote:

Since Mark Phippard has been quoted as joining Polarion on a flyer included in the EclipseCon information pack, perhaps the combination of the existing skills and technology will finally bring kosher subversion support for Eclipse by default.

Even worse is that this same statement has been included in the two-sentence blurb that gets repeated in aggregators like Planet Eclipse.

This is completely untrue.

However, I have recently started a new job at CollabNet, which I am very excited about, and will eventually blog about in more detail. I am assuming that the author of the post somehow got confused about this detail. CollabNet, as well as myself, are very committed to Subclipse and will continue to ensure that it is the best provider of support for Subversion in the Eclipse IDE. In addition, CollabNet provides support for Subclipse in their Subversion support packages available at openCollabNet.

By the way, the Subversive project being provisioned at Eclipse.org, does not mean that the Subclipse-based proposal cannot also be provisioned. Although eventually there does have to just be one project, it is still early in the process. Our proposal got put on hold while I was looking for a new job. I could have advanced it to the next stage back in late November but decided not to. Once the dust settles on new job at CollabNet I will provide more updates on the proposal.

Update: The text of the EclipseZone article has been updated to remove the part about me. Thanks.

What makes an Open Source Company?

One of the core Subversion developers, Garrett Rooney, posted an interesting piece entitled What makes an Open Source Company?

I agree with what he wrote, so I do not have a lot of commentary to add. My definition of an open source company would primarily be based on how the company interacts with and supports open source software development. Do they have developers that significantly contribute to one or more projects? Do they engage the community, or at least allow their developers to engage the community? For me, those are the determining factors.

By the way, Garrett was the author, now co-author with Daniel Berlin, of one of the best Subversion books available: Practical Subversion.

Happy Birthday Subversion

A belated Happy 3rd Birthday to Subversion and those that made it possible. One of the developer's, Mike Pilato, summarizes the history best in this blog entry.

I knew about the birthday from some entries to the Subversion mailing list, but I did not know about Mike's blog entry until I saw a link to it on the OpenCollabNet site.

Subclipse 1.2.0 Released

Subclipse 1.2.0 was released today. This is the official stable release for Eclipse 3.2 and higher. The official release includes two last minute fixes to improve support for 3.3 M5.

This release has been in development and available via the 1.1.x release train since a little before the official Eclipse 3.2 release last June. I imagine that most Eclipse 3.2 users have been using this release since it was available. For those that have been using the 1.0.x releases, this new release contains a lot of new features and UI improvements, such as my previous blog entry about Live Annotations. The full changelog is available here.

One note to those that try to update and do not see the release. By default, the Eclipse update mechanism will not show you an update which increments the minor version number. So you either need to repeat the first time install instructions, or view the upgrade instructions on this page, which show you how to change the Eclipse preference that controls this.

Subclipse 1.1.10 Released

Subclipse 1.1.10 was released today. Details can be found in the changelog.

There are several changes in this release that should address the most common problems reported of late. Most of those are solved by the inclusion of the Subversion JavaHL 1.4.3 libraries and the SVNKit 1.1.1 library. Each of these has several fixes that were affecting Subclipse users.

There was also further refinement to how we handle read only files and the lock dialog when the validateEdit() method is called. This problem really impacted users of the Rational 7.0 IDE's as it seems a lot of their code somehow explicitly calls validateEdit() and bypasses the normal Eclipse ResourceRuleFactory mechanism that should be filtering out any items that are not read-only in the file system. Anyway, with the help of some of our users that were able to test the changes, we seem to finally have all of the scenarios working properly.

Finally, I have a feeling that the next release will finally be the long-awaited 1.2.0 release, which will become our new stable, recommended release. There have been some patches floating around for a couple of new enhancements, and if they get finalized and committed there may be a couple more releases in the 1.1.x line to stabilize those features. But right now, things have very much stabilized and if no new problems show up from this release, I think I will put out the final 1.2.0 release.

openCollabNet

Just a quick post to point you to a site that you might not know about: openCollabNet. This is a site being hosted and supported by CollabNet to provide an online community for Subversion and CollabNet users.

It is a pretty cool site that seems to have a lot of information contained within it such as articles, forums, podcasts etc. I will probably start compiling a list of interesting links and add them to a new post in the future, but for now I would suggest you poke around the various parts of the site and you will likely find something useful.

One of the more immediate things you might find interesting on this site is that they are providing binaries for various Subversion components and tools (such as TortoiseSVN and Subclipse) and I believe they will provide support for those binaries. Currently, I think they are only offering Windows binaries. I would really like to see them provide some for OS X as there currently are not any good sources for up to date binaries for OS X. Metissian used to provide a very nice package, but they seem to have stopped providing them. Not many user's are going to want to download and install XCode so that they can build Subversion from source, and many that do seem to run into problems with various parts of the process -- at least those that are building JavaHL. Anyway, hopefully they have OS X support on their to-do list. Linux binaries would also be nice, but at the same time I think the Linux distributions are doing a decent job providing Subversion packages that work properly, so it is slightly less urgent when compared to OS X.

How to Undo a Commit in Subversion

A frequent question that is asked on the Subversion and Subclipse mailing lists is how to undo a change once it has been committed. A variant of the same request is how to get back a file that has been deleted.

If you need to undo a change completely, as in remove all traces it ever happened, there is really only one way to do it. You need to dump the repository up to the revision you want to remove, and then recreate and load the repository from the dump file. I will not be covering that option in any detail in this article. This article will just be about reversing the effects of a commit, as in restoring a deleted file/folder or some change to a file or set of files. Depending on the circumstances, there might be several ways you can accomplish this. For example, the svn copy command can be used to restore something that is deleted. The technique I am going to focus on in this post is called a "reverse merge". Essentially, you use the svn merge command with the revisions flipped around so that the merge command undoes the changes in the selected revisions. For command-line users, the process to do this is explained well in the Subversion book. See this section on undoing changes.

When using a GUI tool like Subclipse, you could just use the Merge option and fill out the dialog using similar values to what you would do from the command line. However, Subclipse offers a built-in technique that makes it very easy to undo a change.

To give proper credit where it is due, the technique that is described in this article first appeared in the TortoiseSVN product. The Subclipse feature described below is based on the TortoiseSVN feature.

Open the History View
The first step is to show the history of the item containing the change you want to restore. If you just want to restore the changes to one file, then select just that file. Otherwise, you can select your project or a folder so that you can undo changes to multiple files.

If you want to restore something that is deleted, then you need to show the history of the item's parent, so that the history will show you the deletion.

Select the Revision(s)
Now, just select the revision, or contiguous range of revisions, that contain the changes you want to undo. Right-click and choose the option to revert the changes from those revisions:

Review and Commit
When the option runs, it will invoke the merge command, supplying the proper parameters to do a reverse merge. Review the results of the merge and commit the changes to finish the process.

Partial Undo?
It would not be uncommon to only want to reverse part of the changes from a commit or series of commits. There are a couple of ways to deal with this:

1. Select the correct context. If you only want to undo the changes to files in a specific folder, then select that folder when you show the history. Then the merge will only undo the changes in the context of that folder and other changes in the commit will be skipped by the merge process.
2. Revert what you do not want. Suppose you delete five files from the same folder and want to restore one of them. You have to select the parent folder when you take the option, and this will restore all five files. When the command completes, just use the Revert option to undo any changes you do not want, and then commit the rest. The merge process only restores the change in your working copy. Nothing is forcing you to commit the results of the merge without editing it first.
   

Conclusion
Undoing the effects of a change in Subversion is a fairly easy process once you understand what it is that you have to do. The Subclipse and TortoiseSVN UI further simplifies the process and saves you from having to lookup and enter the revisions correctly in the merge dialog.

Enhanced Support for Branches and Tags in Subclipse

This post largely draws on content I previously wrote and posted on the Subclipse web site back in December 2005 when this feature was added to Subclipse. I have attempted to freshen the content a bit in some places and repost it here for a new audience that may not be aware of this feature.

The issue of tags in Subversion, and how they differ from CVS, is a frequent topic on the Subversion users@ mailing list. The same issue came up a lot on the Subclipse mailing lists, especially since the CVS plug-in in Eclipse handles branches and tags very nicely. The design for this feature evolved over time out of requests and suggestions from Subclipse users, along with a few niceties that I added while developing the feature when I realized I could use the information in additional ways. I was initially against adding the feature, as I generally try to stay true to the features of Subversion and do not like inventing concepts that are not really supported and backed by Subversion. However, in the end I really like the way this feature turned out, and I find it very useful in my own projects. There are some areas it could still be improved, but all in all, it is a nice feature.

In no way would I attempt to make the claim that this answers every request/idea/complaint that Subversion users have had about the way tags are handled. That being said, it does address a lot of them :)

Overview
In many ways, the support for branching and tagging in Subversion represents a vast improvement over CVS. It certainly addresses many of the complaints with this feature in CVS. There is one significant drawback in the implementation in Subversion. Namely, the process of creating a branch/tag does not leave behind any "breadcrumbs" in the source of the branch/tag operation to indicate that the branch/tag was created. In other words, there is information in the branch/tag that indicates where it came from, but there is no similar information in the origin of the branch/tag that would let you see or know that the branch tag exists. The net result is that when looking at a file or folder there is no easy way to know which revisions belong to which tags.

Subclipse provides a way for you to work around this problem by allowing you to maintain a Subversion property in your project that indicates the branches/tags that have been created from that project. The name of the property is subclipse:tags. The format of the property value is:

revision_number,tag_name,relative_path,branch/tag

The above screen shot shows what the property looks like if you were to edit it directly. However, Subclipse also provides a Configure Branches/Tags option on the Team menu that allows you to edit the property using a custom editor.

Besides providing basic CRUD capabilities, the dialog also includes a built-in repository browser. This allows you to select one or more folders and add them as a branch/tag in one action. When folders are added in this manner, Subclipse automatically fills in the Revision number based on the Last Changed Revision of the selected folder.

In addition to the Configures Branches and Tags option, there is also support for automatically updating the subclipse:tags property when you create a Branch/Tag using Subclipse. When you take this option, if we detect the subclipse:tags property on the item you selected, then we pop-up a dialog that lets you confirm that you want to add this new information to the subclipse:tags property. You then just have to commit that property change after creating the Branch/Tag. This feature does not exist when creating the Branch/Tag from the repository browser as it is only possible to modify a property value in a working copy.

A nice aspect of using a Subversion property to drive this feature is that only a single user or build process needs to assume the responsibility of managing the property value. They just commit the property change and all users receive the benefits in their UI when they update.

Features
Once the property has been defined, there are a number of ways that Subclipse takes advantage of it to provider a better user experience in the UI.

History Browsing
This was really the number one reason for pursuing this feature in the first place. The History View has been enhanced to show the tags for a revision.

When the History option is invoked against a local resource, we read the subclipse:tags property value from the local working copy and use its contents to populate the tags column in the view. A preference controls whether to show this information when browsing history directly from the repository. In this scenario, in order to show the tag information, we have to search the repository for the presence of the subclipse:tags property. On a slow connection you probably would not want to do this.

Compare Revisions
The Compare with Revision ... option has been enhanced to show tags in its history view. This makes it easy to know which revisions you want to compare.

URL Chooser
The URL Chooser that allows you to pick a URL in many of the Subclipse dialogs has been enhanced to show Branch/Tag information. This allows you to just select a branch/tag by it's name or identifier and the proper URL to that branch/tag will be automatically created. The enhancements we were able to make to this dialog turned out to be the best "side-effect" of adding this feature. Not having to type out, or browse to select, really deep URL's in the repository is a real time saver and usability improvement.

Compare with Branch/Tag
Technically this feature is not dependent on the property, but it was added at the same time and the property makes it easier to use. Subclipse provides a Compare with Branch/Tag option in the Eclipse Compare With menu. The option gives you the choice of producing a unified diff file or doing a graphical compare as shown in the screen shot below. The Eclipse graphical compare option can be fairly slow if you do not have a fast repository connection. Also, the Eclipse compare option will show differences in svn keywords which the unified diff option will not. That being said it is still convenient to have and combined with the enhanced URL Chooser, it is a very nice option.

In the future, I would like to enhance the graphical compare option to either be driven by a unified diff file (which would be much faster and accurate) or perhaps a new routine that used the new svn diff --summarize option to produce the initial list of differences, instead of letting Eclipse calculate them. Either option would make it run a lot faster and eliminate the false positives caused by the use of keywords.

Future Enhancements
Eugene Kuleshov has requested some enhancements to this feature that might come in the future.

The first is issue 471, which is to remove the revision number from the property value and instead just discover and cache it when needed in the UI. The reason he wants to do this is so that we can change the way we update the subclipse:tags property so that we do it before we create the branch/tag. Then the updated property information would be included in the new branch/tag. See this mailing list thread for his reasons.

I do have some concerns that this will unexpectedly slow down the UI when the cache is populated against a slow server. I suppose that could be solved with proper usage of a progress dialog, but I have the feeling this would happen so deep in the code that might be more difficult that it sounds. What I would ideally like to see is an ability to update the property in the branch/tag as part of creating it, but this would require new Subversion API support.

Eugene's next request was issue 472. This request is to enhance the create branch/tag process to allow you to process several projects at once. I would really like to see Subclipse focus on this area once the 1.2.0 release is finalized. Basically, we need to make it easy to perform all of the branch/tag options on multiple projects. This would include, creating the branch/tag, but also switch, merge and probably compare. This is also entered in issue 596.

Conclusion
This is a nice feature that improves the user experience for Subversion users. There is always more to do, such as better multi-project support, but we will get there.

The one negative I have found about this feature is that your branching policy will sometimes make it less effective. For example, the Subversion and Subclipse projects both have a policy where we create a branch, such as 1.2.x, to stabilize a release. All release tags are therefore made from the release branch, which means the subclipse:tags property, containing the tag information, lives on the release branch. However, developers mostly work in trunk and thus cannot really benefit from the History view feature of showing the tags in the view. This is an area where I think native Subversion support for knowing what tags a revision was copied to would be needed. If Subversion ever added this feature, it could likely tell that a file that was copied to a branch and then tagged is the same file that is on trunk. I guess the problem is whether Subversion can ever add a feature like this and still support the concept of "cheap copies" which is the foundation of the whole product.

Features of the Subclipse Commit Dialog

This is a post I have been wanting to write since I started my blog. In this post, I am going to cover some features that exist in the Commit dialog within Subclipse. It is very likely that you will not know at least about one of these features.

Might as well start with a screen shot that shows the dialog and some of these features:

Integration with Issue Tracking Systems
I decided this topic was too big for this post, so I first wrote up a post on how to Integrate Subversion with Your Issue Tracking System. Refer to that post for the details.

Message Width Marker
Look closely at the right hand side of the commit message. See the feint line drawn through the edit control? That lets me see where a specified column of text resides so that I can format messages according to whatever guidelines might exist for the project I am working on. The Subversion commit message is just a blob of text that is stored the way you enter it, including (or not) any line feeds. A lot of people might have routines that parse or reuse the messages and often it is desired that the messages fit nicely within someone's terminal window.

This feature first appeared in TortoiseSVN, and we added the same feature in Subclipse. To turn it on, you need to define an SVN property named "tsvn:logwidthmarker" with a numeric value that indicates the column where you want the line to appear, such as "79". Since most Eclipse users will be checking out Eclipse projects, you should only need to set this property on the project root folder. Otherwise, just set it on all folders. This rule applies to all of the subsequent features I will describe that involve SVN properties.

Handling Unversioned/Missing Files
The commit dialog will show any unversioned or missing files that appear beneath the folder you selected when you took the option. By default, unversioned files are not selected in the dialog, but you can change this in the Eclipse preferences under Team -> SVN. Missing files are never selected automatically. If you select these files, when you click OK, Subclipse will execute the svn add/delete command prior to running the svn commit command so that the items are properly included in the commit process.

Remember Previous Messages
There is a combo box in the middle of the dialog that shows previous commit messages you entered, including the text you entered before you hit the Cancel button. This can be useful if you write the commit message and then suddenly realize you left some debug code in that you want to remove before committing.

Message Templates
Subclipse supports two kinds of message templates. The first one comes from TortoiseSVN, and that is to set an SVN property named "tsvn:logtemplate" where the value of the property contains the template text. If this property is set, the dialog will come up with the text of the template already filled in. The second template system is an Eclipse feature first introduced by the CVS plug-in. This features lets you setup any number of templates in the Eclipse preferences. These templates are all then available in the drop-down combo that shows previous messages that were entered.

Require a Message
This feature also comes from TortoiseSVN. You can set an SVN property named "tsvn:logminsize" where the value is a number. When this property is set, the OK button will not be enabled on the commit dialog until the number of characters entered in the message is equal to or greater than the value of this property. So you can set this to a value like "10" and that would require the user to enter at least 10 characters. If for no other reason, entering a value of at least "1" can save a user from accidentally clicking OK without entering a message.

Internal Resizing
All of the sections of the dialog can be resized. So if you like a large area to enter comments, with just a couple of files showing, you can arrange the dialog that way. Likewise the opposite is true if you would prefer to be able to see more files.

Show Differences
This one is my favorite, and one that a lot of user's do not seem to know exists. You can double-click on any file in the list and see the differences in that file. This can really help in writing proper commit messages.

The only negative to this compare feature is that the dialogs are all modal so you cannot look at the compare while you are typing the commit message. I'd like to enhance the commit dialog someday so that the compare results showed in a new section of the dialog, but I am not sure if that would work well from a screen real estate point of view. There is also the question as to whether the Eclipse compare UI can be embedded in a dialog like that. I suspect it can.

Conclusion
The commit dialog in Subclipse contains a number of features that are designed to make the process of working with Subversion easier and more usable for you. Hopefully there were one or two features described in this post that interest you and that you did not already know about. Most of these features were first developed and included in TortoiseSVN, so I would like to just conclude this post with a thank you to the TortoiseSVN developers for paving the way. Thank you.

Integrating Subversion with your Issue Tracking System

This article will explore a technique for integrating Subversion development with your issue tracking system. This technique was cooperatively developed by the TortoiseSVN and Subclipse development teams, and both products support the features that will be described.

Background
Many popular issue tracking systems have developed procedures for integrating with version control tools like CVS and Subversion. A common approach that is used is for the issue tracker to provide one or more Subversion hook scripts that interrogate the message provided in the commit process, looking for references to issues in the message text. Depending on the tool being used, there will often be some meta data stored within the issue tracker to record the files that were modified for that issue. Some tools support features that change the state of the issue, others might perform some kind of validation and possibly reject the commit transaction. This part is really up to the issue tracker and the hook script(s) they provide.

The problem with this approach is that it only works as well as the ability of the issue tracker's hook script to correctly identify the references to issues in the commit message text. With no help being provided by the version control tool to help the user consistently format their message, this can be a difficult task to perform accurately. Also, even when everything works correctly, the integration is only in one place, the issue tracker. In other words, you can view an issue in the issue tracker and see what files were modified (or some other reference to the version control tool), but when looking at history in the version control tool, you do not have easy access to the related issues that were addressed by the commit.

With all this in mind, these were the goals of the issue tracking specification that we developed:

1. Provide a place in the UI of the commit dialog for the user to supply issue ID's. These issues are then inserted into the commit message in a consistent manner.
   
   



2. Provide for the possibility of some simple validations in the UI. Such as verifying that the issue is a number, or warning if no issue is entered.
   



3. When showing Subversion history in the UI, convert any references to issues in the commit message into clickable hyperlinks to the issue tracker.
   
   

You can view the full specification in the TortoiseSVN Subversion repository here. (username = guest with no password)

These features combine to give the user a better experience in terms of referencing their issue tracker from their Subversion client and also makes it much easier to write server side hook scripts that can accurately parse the commit messages. A nice aspect of this feature is that since the integration between Subversion and your issue tracker is still ultimately based on the text of the commit message, even users that are using a client that does not specifically support this feature can still participate in the integration by entering their commit messages in a format consistent with what this feature would have done for them.

Details
With all this in mind, how do you turn on this feature and/or configure it? The way that we settled on was to use Subversion properties to hold the configuration information. This allows one person to set it all up, and the configuration can then be pushed out to other users via the update and/or checkout process. In addition, the configuration just needs to be set on the top-level folder in the working copy. At runtime, the client will walk the working copy back to the root looking for the configuration. This means if you just set it on the root folder, that is enough. Keep in mind, that the client is walking the "working copy" not the "repository". If you have an environment where someone might conceivably check out at any folder level in the repository, then you might need to store this configuration as properties on every folder. Subclipse users are working in Eclipse, and Eclipse has a fairly rigid project structure. So for Eclipse users, it is pretty easy to configure as you just need to put the configuration on the project folder and you can be fairly confident that is what your users are checking out.

Once that is decided, there are then several properties all prefixed with "bugtraq" which determines the configuration of the feature. We chose bugtraq with the slightly odd spelling just to avoid any potential current or future namespace conflicts. Here are the properties and an explanation of each, this is actually copied from the spec verbatim:

NOTE: The specification has evolved a bit since it was first written and now breaks the feature down by a "basic" and "advanced" configuration where the latter is based on the use of regular expressions. Subclipse currently only supports the basic configuration options, and that is also all that this article covers.

Bugtraq Properties:

name : bugtraq:url
value : (string)
This value is the URL pointing to the bug tracking tool. The URL string should contain the substring "%BUGID%" which the client replaces with the issue number. That way the resulting URL will point directly to the correct issue.

NOTE: The URL must be properly URI encoded by the user. This URL can be used by clients to create a link to the bug tracking tool when showing the log message of a revision.

name : bugtraq:warnifnoissue
value : "true"/"yes" or "false"/"no"
If set to "true", then the clients will warn the user if the issue text box is left empty. But it must not block the commit, only give the user a chance to enter an issue number in case (s)he forgot it.

name : bugtraq:label
value : (string)
This can be used by a client as a GUI label describing the text box where the user has to enter the issue number. If this is not set, then a default value should be used, e.g. "Bug-ID / Issue-Number :". Keep in mind though that most GUI clients don't resize their windows according to the size of GUI elements. So keep the size of the label string below 20-25 chars.

name : bugtraq:message
value : (string)
If this property is set, then the client should show a text box in the commit window where the user can enter an issue number. The value string of this property is used by the client to create an additional line added to the log message. The string must contain the substring "%BUGID%"
which is replaced with the issue number the user entered before applied to the log message. The client will add the resulting string as a new line to the end of the log message the user entered:
logmessage + "\n" + resultstring

In case bugtraq:append is set to "false", then the log message is defined as resultstring + "\n" + logmessage

The client should make sure that the string doesn't have multiple lines. If more than one issue number is entered by the user (separated by commas), the client must make sure that there's no whitespace chars before and after the comma. And also the whole issue number string must be trimmed.

name : bugtraq:number
value : "true" or "false"
If this property is set to "false", then the client allows any character entered in the issue text box. Any other value or if the property is missing then only numbers are allowed as the issue number. One exception is the ',' char, which is used to separate multiple issues. The client must never complain if the text box is left empty, i.e. if no issue number is given. Not all commits are assigned to an issue!

name : bugtraq:append
value : "true" or "false"
If set to "false", then the bugtraq:message part is inserted at the top of the log message, if "yes" or not set, then it's appended to the log message at the bottom.

Since these are all Subversion properties, you need to use the svn propset command to set them on a folder within your working copy (there is no way to do it directly against the repository).

svn propset bugtraq:append false .
property 'bugtraq:append' set on '.'

You then need to commit those changes so that other users can get them. Keep in mind that in order to commit property changes to a folder, the revision of the folder in your working copy has to be the same as the HEAD revision of the folder in the repository.

Here is an example of the properties set on one of the Subclipse projects as an example:

If you refer to the previous screen shots, you can get an idea how these properties manifested in the UI of the commit dialog and the text that was inserted into the commit message.

Conclusion
This is a feature in Subclipse and TortoiseSVN that our users lobbied hard for and seem to love. I personally enjoy the easy access to issues that is created by the hyperlinks in commit messages when viewing history. Often reading the history contained within an issue is the best way to understand the context of why a particular change was made.

I do not know the complete list of Subversion clients that have added support for these properties, but I know that it goes beyond Subclipse and TortoiseSVN and includes several of the web-based repository browsers.

The Subclipse integration with Mylar includes partial support for this feature as well. When browsing history within Subclipse, the Mylar "Open Corresponding Task" option will be present and will determine the issue based on these properties and the content of the commit message. I assume this would mean if you were using Bugzilla or another issue tracker with a rich editor in Mylar that the rich editor would be opened in place of the web browser. Unfortunately the Mylar Change Set integration in Subclipse does not currently have a way to plug your issue number into the right part of the commit dialog. It does stick the URL of the issue in the commit message text, so all is not lost. Perhaps in the future it will be possible to enhance this so that the issue ID itself is populated.

Subversion 1.5: Tolerate obstructions during checkout/update/switch

This is my second article about new features in Subversion's trunk, features that will be included in Subversion 1.5. My previous article discussed enhancements in Subversion's move/copy commands. This article will discuss improvements that have been made to the checkout, update and switch commands -- specifically, the ability to tolerate obstructions.

Prior to these changes, if Subversion needed to add a file or folder to your working copy as part of the checkout/update/switch commands, and a file or folder with the same name already existed, then the command would abort with an error. The message you receive will vary based on the command and the situation, but it will look something like this:

svn: Failed to add file 'foo.c': object of the same name already exists

As of Subversion 1.5, you can add the --force switch to any of these commands to override this behavior and tell Subversion to tolerate the obstruction. The output you normally see from these commands will tell you when this happens. For example, it will look something like this:

E foo.c
A bar.c

A new code of 'E' for 'E'xists has been added to inform you of this situation. What Subversion does in this situation is create the ".svn" meta data, as it normally would. The "pristine" copy of the file is stored based on the repository, as it always would be. The file itself, however, remains unchanged. If the file does not match the file in the repository, then svn status will show the file as containing local modifications. Those modifications can be viewed with svn diff, or committed with svn commit. Likewise, you can use svn revert to restore the local file to the pristine copy.

These changes help with a number of scenarios that can happen in normal development. The original use case that we wanted to solve was a developer that downloaded a tarball of some product and winds up making some fixes. Since they started with a tarball, they do not have a Subversion working copy on their disk. If they want to commit their fixes, or even submit them as a patch, then they need to have a working copy. With this change, they could use the svn checkout command, and the --force switch, to checkout the code from the repository on top of their local copy. When the command completes, they will have a valid working copy. They can now create a patch with their changes, or commit them if they have access.

With the code that is currently in trunk, in this scenario, the svn checkout command will still need to pull down the same number of bytes from the server as a "normal" checkout. It is possible that it could be enhanced in the future to more selectively pull down just the differences, something like an rsync. A change of that nature has been deferred for some future release or whenever someone comes along with enough motivation to tackle it.

Here is another scenario where this change solves a problem. Suppose a developer has a project checked out. They create a patch to implement some feature or fix and that patch includes some new files. They submit the patch to the community and a developer reviews and commits it. When the original developer runs the update command, the command will fail unless they remembered to remove the new files they had submitted in their patch. Now, they can just use the --force option with the svn update command, and it will complete normally and just reconcile any issues. If the patch was committed exactly as it was submitted, the developers working copy should not show any modifications after the command completes. If the patch was tweaked, then a diff of the working copy would show whatever tweaks were made, and the svn revert command could be run to get the working copy back to a pristine state, ready for the next patch to be worked on.

We will be making use of this new feature in several places in Subclipse, including several where we have had to code our own workarounds to get around this problem. There are also a couple of new checkout and project share scenarios where we should be able to utilize this new option to implement features that our users have asked for.

Shelves in Subversion

There is a popular feature called "shelves" that was included in Microsoft's Visual Studio Team System. I am fairly certain that VSTS is not the first or only tool to have this feature and in this article I will show you how to get the same feature from Subversion. I think of shelves as essentially creating and using a branch to save some changes you have been working on, but now need to set aside, or shelve, for a while. Here is how Microsoft explains the feature in their documentation, which I found via a search:

Shelve your pending changes when you are not ready to or cannot check in a set of pending changes. There are primarily five shelve scenarios:

• Interrupt When you have pending changes that are not ready for check in but you need to work on a different task, you can shelve your pending changes to set them aside.

• Integration When you have pending changes that are not ready for check in but you need to share them with another team member, you can shelve your pending changes and ask your team member to unshelve them.

• Review When you have pending changes that are ready for check-in and have to be code-reviewed, you can shelve your changes and inform the code reviewer of the shelveset.

• Backup When you have work in progress that you want to back up, but are not ready to check in, you can shelve your changes to have them preserved on the Team Foundation server.

• Handoff When you have work in progress that is to be completed by another team member, you can shelve your changes to make a handoff easier.

The branching model in Subversion, with its use of "cheap copies" is well suited to provide similar capabilities and handle all five of these scenarios. In the rest of this article, I will detail two different ways to shelve changes using branches in Subversion.

Quick Method: Create Branch from Working Copy

The first and easiest method is to simply create a branch from your working copy. A lot of Subversion users do not realize that you can do this. I actually use this technique a lot when creating complex tags. Just get your working copy exactly as you want the tag, and then create the tag based on your working copy.

svn copy . url://server/repos/project/tags/tagname -m "Create tagname"

This creates the tag in the repository based on what is present in the working copy. Subversion does not even need to send any file contents to the repository to create the tag, so it still runs very fast.

You can use this same concept to create a branch or shelf with the changes you are working on. Here is a screen shot from Subclipse showing how to do this:

Notice that the option to create the copy from the Working Copy has been selected. That is the key to this method. When the command runs, Subversion creates a copy in the repository based on the base revision of your working copy, however any local changes are also committed as updates. Likewise, if you have svn added or deleted any files/folders, then those are also included and it does this all in a single transaction.

After the command completes, your working copy will still appear to be modified (because the changes were committed to a different branch then the one associated with the working copy). To get your working copy back to a pristine state, before beginning your next task, you will need to run the svn revert command to remove all local modifications. The svn revert command does not remove files and folders that were added, so you need to manually delete those after the revert.

Hint: The Subclipse revert process will remove selected unversioned files and folders. So if you run the revert option twice, the first time you run it, added files are made unversioned and the second time you run it, they are deleted.

Creating a shelf using this technique is very easy. The problem with this technique comes when you want to work on your shelf again. To do so, you want to use the merge command to merge your changes from the shelf back into the current working copy which is associated with trunk or some other branch. The problem is that since the shelf was created in a single transaction, merging it is a little bit tricky. Suppose the shelve was created with revision 50. The normal way you might expect to merge it into your working copy would be to run a command like this:

svn merge -r49:50 url://server/repos/project1/branches/shelf

This tells Subversion to merge the changes that happened with revision 50 into the current working copy. However, in this scenario, when you run this command you will get an error something like this:

svn: Unable to find repository location for 'url://server/repos/project1/branches/shelf' in revision 49

The problem is that the shelf did not exist in revision 49, so the command errors out. The answer to this is that you have to run the merge using two URL's. The source URL would be the base location that your working copy was pointing to (which was the base of the copy) and the target URL would be the shelf. You also need to know what revision your working copy was at when you made the copy. You can determine this information by running the svn log command on the URL of the shelf. The output of the command will show both the URL and revision it was copied from. Unfortunately, this is also where it can get really complicated. There is a better than likely chance that your working copy contains mixed revisions (see my previous post on Mixed Revision Working Copies). Assuming that is the case, when you made the copy from your working copy, you will have gotten a fairly complex transaction in the repository. As an example, this is the output I get from svn log using a fairly simple mixed revision example:

------------------------------------------------------------------------
r9 | markphip | 2007-01-08 09:34:06 -0500 (Mon, 08 Jan 2007) | 1 line
Changed paths:
A /branches/fromWC (from /trunk:3)
A /branches/fromWC/.classpath (from /trunk/.classpath:4)
A /branches/fromWC/.project (from /trunk/.project:4)
A /branches/fromWC/src (from /trunk/src:4)
A /branches/fromWC/src/org (from /trunk/src/org:5)
M /branches/fromWC/src/org/tigris/subversion/javahl/ChangePath.java
R /branches/fromWC/src/org/tigris/subversion/javahl/ClientException.java
(from /trunk/src/org/tigris/subversion/javahl/ClientException.java:7)
R /branches/fromWC/src/org/tigris/subversion/javahl/JNIError.java (from /trunk/src/org/tigris/subversion/javahl/JNIError.java:8)

As you can see, I have files and folders copied from revisions 3, 4, 5, 7 and 8 and my shelf was created in revision 9. In this example, I know that revision 8 is the right value to use for the source of my copy, but in a more complex example, there may not be a right answer. Let's go back to our example, and just say that the shelf was created in revision 50 and it was created by copying trunk which was at revision 49. Assuming that is the case, then to merge the changes made in the shelf into your working copy, you would need to run a command like this:

svn merge url://server/repos/project1/trunk@49 url://server/repos/project1/branches/shelf@50

This tells Subversion to construct a diff of the changes between trunk @ revision 49 and the shelf @ revision 50 and then apply that diff to the current working copy. When this command runs, your working copy will now contain all of the changes you shelved. You can then go back to working on your change and eventually commit it to trunk or whatever the appropriate branch is to receive the change. Here is a screen shot of the same merge command from Subclipse:

Note that you just have to un-check the box that says to "Use From: URL". This then allows you to enter different "From" and "To" URL's and revisions.

To summarize this method, creating the shelf is easy, but using it again can be difficult. I will now show you what I think is a better, and much cleaner, method for creating shelves.

Better Method: Multiple Steps

There are really two main goals with this method:

1. Remove the issue of mixed-revision working copies from the equation.
2. Isolate the changes you want to shelve from the process of creating the shelf so that they are easy to merge later when you want to use them again.
   

This is always the technique that I use and it is generally much cleaner than the first approach.

The first thing to do is to create the shelf/branch. If you know your working copy is a little old, compared to the HEAD revision of your trunk or branch, then you can use the last revision you committed or updated to when creating the shelf. Otherwise, just use the HEAD revision:

svn copy -r HEAD url://server/repos/project1/trunk url://server/repos/project1/branches/shelf

This command is creating the shelf directly in the repository, based on whatever URL and revision your working copy is associated with. In this example, I used trunk. The next step is to use the svn switch command to switch your working copy so that it is pointing at the shelf URL:

svn switch url://server/repos/project1/branches/shelf

The switch command is also like an update. So if your working copy was not at the same revision you copied when you created the shelf, then it is possible that you will receive some updates, and perhaps even have some conflicts created in your working copy.

Here is a screen shot from Subclipse of the Create Branch dialog. Note the check box on the bottom that lets you create the branch and switch your working copy to it one step:

Once you have dealt with any conflicts that might been created in your working copy from the switch, you just need to commit your changes to the shelf using the normal commit command. When you are done, just use the switch command to switch your working copy back to trunk or wherever you need to be for your next task. Unlike the previous method, you will not need to cleanup your working copy when you do this.

At this point the shelf has been created and your changes have been committed in a way that will make it easy to merge them back later. The only part of this process that is potentially a bit more difficult than the first method is that you might have to resolve some conflicts before you can commit your changes to the shelf, and depending on why you are creating the shelf, you might not have the time to do this. If you create the shelf branch from the revision you have loaded in your working copy, you should minimize any potential for conflicts.

When you need to work on your shelved code again, it is easy. Just use the merge command to merge the changes from your shelf to your current working copy. Suppose you created the shelf with revision 44, spent some time resolving conflicts that were created when you switched, and then commited the changes to the shelf with revision 50. To merge the changes in the shelf to your current working copy, just run this command:

svn merge -r49:50 url://server/repos/project1/branches/shelf

This tells Subversion to merge the changes made in revision 50 to your current working copy (I could have used any value between 44 and 49 for the from revision). Since we separated the process of creating the shelf, from the changes you wanted to store in the shelf, this is an easy command to run. When the command completes your changes will have been merged into your working copy for you to pick up the work where you left off. When you are done, you just commit everything to trunk, or whatever branch is associated with your working copy, and you are done. The branch you created for the shelf can now be deleted from the repository if desired.

Summary
I went into a lot of detail in this article but hopefully you now know a lot more about how branching and merging work in Subversion. You can apply this information in your daily work process and use Subversion as a tool that helps you do your work better and more efficiently. I wrote this article under the pretense of the Shelving concept, but it is really just basic branching and merging in the end. I think one of the strengths of the Subversion design is that they kept it all very simple and you just use the basic functionality to build the process you want.

Combating Sudden Slowdowns in Subversion

Martin Tomes writes about a 20x performance slow-down in checkouts after installing a new anti-virus program. There have been a lot of reports over the years of various anti-virus programs causing slow-down's or other problems with Subversion on Windows. It is a good thing to check if suddenly you are having performance problems or other errors. Hopefully you are at least using a program that lets you fine-tune what and how it is doing the checking.

Another item that people have reported to cause slow-down's is the XP drive indexing. This can be disabled by right-clicking on the drive and choosing Properties. There is an option at the bottom of the General tab that controls whether Indexing is allowed. I do not know if other tools that index your drive, such as the Google Desktop can cause similar problems, but I suspect they can.