So normal output redirection is handled with just the > or >> operators (depending on whether you want to overwrite or append the target file), like so:

dir >> myfile.txt

The operator outputs the stdout stream by default. But it can be modified to output the stderr stream by just specifying that stream with the operator. Remember, stderr is stream 2. So, you would do like so:

dir 2>> myfile.txt

This would write only the error output to the file.

The nice thing about *.ini files. Not that they're all bad. It's ridiculously easy to extract settings from these files with ColdFusion. For instance:

To retrieve the mysetting configuration setting, I just make one call:

It can't get much easier than that to retrieve a value from a configuration file!

The bad thing about *.ini files. The only problem is that these files are restricted to name=value pairs of simple values. You may go into a project with no need for configuration settings that are any more complicated than simple numbers or strings, but what about 3 years from now when the business logic now could use some structs or arrays in the configuration?

Of course, you could change your whole configuration methodology when--and if--that time comes, but ugh. Using XML (especially in conjunction with IoC such as ColdSpring) is so much more scalable. The future you will thank you.

But it's too late. You and I already have complex app XYZ that is using *.ini config files, and you need to add feature X without rebuilding the configuration logic of the app. What to do?

Embed a string representation of your struct or array in the config file! After all, ColdFusion 8 allows implicit struct and array creation, right? Well, half-right. Doing something like x=Evaluate(myStructString) just isn't allowed by ColdFusion.

However, we can effectively do the same thing with JSON representations of structs and arrays. So, try this:

Now, retrieving mysetting with GetProfileString() will still get you a string, but using JSON functions built into ColdFusion 8 will get you a struct:

Your myStruct variable will be a struct with two keys ("one" and "two") just as you would expect from the JSON.

You could achieve similar results using XmlParse(), but in a case like this, the less verbose, the better. And JSON is less verbose than XML.

For reference, here was the object definition we were using in our sample Transfer configuration.

The process is very simple. Recall that we prefixed our Transfer properties with "Enc" to know which fields are encrypted. Let's create a getDecryptedMemento() method. It will loop through all of the keys in the memento, which is just a struct, and decrypt any of the encrypted fields, saving them as new keys in the struct (i.e. "EncHome" is decrypted as "Home" in the struct).

Something like this will do the trick:

Not much to it. Note that we would use the same decryption function as we are using in onMissingMethod for the dynamic accessors. I'm calling it decryptValue() for this sample.

In the end, our getDecryptedMemento() method returns a memento that looks just like the Transfer-generated getMemento() output, with decrypted versions of the encrypted fields!

First, let's consider the object definition in our sample Transfer configuration file.

Note that I have prefixed the Transfer properties with "Enc". This isn't required, but will simplify our processing later on. So in this sample, we have 4 properties: Home, Work, Mobile, and SMS. The concept is that these are client phone numbers that we want to protect with encryption. So, Transfer will generate getEncHome(), getEncWork(), getEncMobile(), getEncSMS(), and corresponding setters. However, we want to provide getHome(), getWork(), getMobile(), getSMS(), and corresponding setters, that get and set the properties after encryption/decryption.

Use onMissingMethod()

To accomplish this, we might have an onMissingMethod() function in the decorator like this:

As a good security measure, we use an "encList" string to check that only proper methods can be called. Then we examine the method name. If it starts with "get" or "set" and ends with one of the list items, we proceed. Thus, we'll process a method like getHome(), but not a method like getFoo(). We then call getEncHome() and decrypt it for getters, encrypt and call setEncHome() for setters.

Dynamic Method Calling

But alas, how do we do this dynamically without a long if-then-else chain? We call the proper method by referencing it from the component's variables scope. Terrence Ryan describes this in his blog post Cheap and Easy Dynamic Method Calling in CFScript. We construct the method name, in our case, by inserting "Enc" after the "get" or "set" prefix of the method name passed into onMissingMethod (thus "getHome" becomes the Transfer method "getEncHome"). Assign the function to the local "func" variable, then call it as if "func" was the function. Finally, handle any encryption or decryption that you will be doing. For this sample, I have encryptValue() and decryptValue() methods handling the details of that work.

Summary

We've used two solutions to handle multiple encrypted fields in the decorator. First, we used onMissingMethod() to handle processing of all of the properties in a single method. Then, we used dynamic method calling to call the Transfer-generated accessors within onMissingMethod().

The end result is being able to use setHome(), getHome(), etc. with the encryption and decryption being handled automatically. Great!

In addition to accessors, we may sometimes want to use the object's memento with the Transfer method getMemento(). In my next blog post, I'll demonstrate how that process can also be done very efficiently.

At first glance, you'd conclude the process would work much the same way: Loop through the individual directories in the large repository, create smaller repositories for each one, then dump and import the contents of each directory into its small repository.

The tricky part is that the Subversion dump command dumps everything in the repository, by revision. In order to pull just a single directory, you must filter a complete dump with the "svndumpfilter" command. This blog post by AllMyBrain.com basically explains how to accomplish this in Linux. I usually have to work on a Windows box on the job, so I wrote up a script to accomplish this in a Windows batch script.

The strategy is the same as the Linux script, though. We're going to use "svnadmin dump" the large repository, then use "svndumpfilter" to filter by just the directory we want, then "svnadmin load" the results into the newly created repository. All of this can be combined into a single statement via piping:

This will make a little more sense when we look at the full script. Let's just put it out there and then go through it.

First, we're setting our paths. "SmallRepoPath" will be the directory holding all of the small repositories we'll be creating. "PathToRepo" and "UNCToRepo" point to the big repository as DOS and UNC paths, respectively. "PathToChkout" points to a Subversion checkout of the large repository.

First, we check out the large repository with the "svn co" command. We do this just so that we can call the "dir /A:D /B" command, which says, "List just the directories in the checkout directory." We use that output to loop through each directory in the large repository.

Then, for each directory in the large repository, we create a corresponding small repository, then do our dump/filter/load combo. Again, we're dumping the contents of the large repository, using "svndumpfilter" to filter by directory, then loading that filtered dump into the new small repository.

Finally, we just do some cleanup by removing our temp files and the checkout directory.

There are a few caveats with this code.

First, it will import all of the large repository's revisions into the smaller repository. There are svndumpfilter arguments to prevent this, such as --drop-empty-revs and --renumber-revs, but I found the Windows Subversion binaries to be problematic with these arguments. The end result is that you have more revision numbers than needed, but only the relevant data is actually imported into the repository, and viewing logs on just the imported directory will still obviously show revision logs related to that directory, so there's really little harm done.

Second, the dump/filter/load action doesn't always work on a directory that has been moved (copied/deleted) from another location within the large repository. What's worse, it won't fail, it just won't load any data into the small repository. To address this, use the --revision argument on the "svnadmin dump" command to do a dump starting at a revision after the move took place. Doing so will give the "svndumpfilter" command something it can work with.

This process is certainly more complicated to explain, but ultimately there's not that much more going on. Hopefully this explanation is helpful to you.

I recently reinstalled the OS on one of my Macs, and I've got backup scripts on my CentOS Linux box that use rsync to back up some pertinent data, so I had to reestablish the SSH key between the machines and had a hard time remembering how to do it. So this time I'm documenting what I had to re-learn.

First of all, there's a great post over at nixCraft that basically explains how to do it. But allow me to explain more thoroughly, ahem, dumbed down to my level. 

The key is remembering which machine is filling which role when you're reading the instructions. I'll call them the "Acting" machine--the one who is taking action and performing a command, let's say an rsync command--and the "Target" machine--the one who is being acted upon. In my case, the Linux server is the acting machine performing the rsync command, and my Mac is the target.

The process is simple. On the "Target" machine, generate a key, and then give that key to the "Acting" machine, which effectively gives it "permission" to login without the need to supply username/password credentials.

So, from the "Target" machine, in this case, my Mac, type the following command:

ssh-keygen -t rsa

This will generate a couple files that serve as a key for accessing the Mac. The ssh-keygen command may ask you where to store the key and what password to use. Just hit enter to use the default path and a blank password.

Next, still from the "Target" machine (my Mac), type:

ssh MyUsername@ActingServer "mkdir .ssh"
scp .ssh/id_rsa.pub MyUsername@ActingServer:.ssh/authorized_keys2

In the code above, MyUsername@ActingServer would be the username and address (for instance, perhaps the IP address) of the "Acting" machine, in my case, the Linux server. In the first line, you're just creating the .ssh directory if it doesn't exist. In the second line, you're copying the key you generated from the "Target" machine to the "Acting" machine, or from the Mac to the Linux server.  Note that the scp command will ask for the password to the MyUsername account because it is connecting to that server to send it the key.

Voile. As if by magic, the "Acting" machine should now be able to SSH into the "Target" machine without a password prompt. Correspondingly, you should be able to perform rsync and other SSH commands without a password prompt. Please note, however, that this is only a one-way key. We only gave my Linux server permission to access my Mac.

What if I want my Mac to similarly login to the server without a password prompt? In that case, the Mac and the server have effectively switched roles; the Mac is now the "Acting" machine and the server is the "Target" machine, so we just have to repeat the process from the other direction. Generate a key from the server and send it to the Mac. At that point, both machines will be able to access each other without a password prompt. 

What if I have multiple "Targets" that the "Acting" machine will connect to? For instance, perhaps I have multiple Macs, and the Linux server is running scripts on all of them. When you're sending the key to the "Acting" server with the scp command, use a different name for each key file, don't overwrite the same file each time! So in the example code above, we're sending the key as "authorized_keys2". When repeating this process for multiple targets, send the keys as "authorized_keys3", and so forth.

Hopefully this will clear up some confusion regarding this process.

The insanity had to stop! A handful of databases like this would put us over the top on that particular server's hard drive storage.

The SQL Server GUI for shrinking the database rendered no effect, and even using the DBCC SHRINKFILE command was not working.

The key, as explained by Pinal Dave, is to run the SHRINKFILE command twice, with an explicit backup log truncation in between both runs. This code here will get you up and running:

This freed up dozens of gigabytes on our server.

That said, there are advantages to having one single repository. No big deal, that can be done after the fact with code.

Here is some Windows code to combine all the repositories in a directory into a single big repository:

There's really not much happening here; the process is simple. First, we create the new "big" repository with the svnadmin create statement. Second, we loop through the directory, processing each Subversion repository in the directory with a three-step process: (a) Dump the repository with the svnadmin dump statement into a temporary *.dmp file. (b) Explicitly add a new directory in the "big" repository for the current repository we're processing, with the svn mkdir statement. (c) Import the dump into the "big" repository with the svnadmin load statement. Really, the rest of the code is just looping, commenting, or cleanup code.

What have we produced? As you might expect, we now have one big repository that has all of the files and commits that were in all of the smaller repositories. The big repository will maintain its own revision numbering, so the revision numbers in your smaller repositories will not match the big repository's revision numbering, although the original commit dates will be preserved. This can be really handy for searching or similar actions that you might do from a more global perspective.

However, this approach is not without its caveats. During the import process, one entire repository is imported at a time. All of a particular repository's revisions will be "grouped" together in the big repository. As a result, revision numbers in the big repository will change every time you recreate it, if there was any new activity in the repositories it contains. For instance, revision #1050 in the big repository may parallel revision #500 in Repository X, but if a commit was added to a repository that is imported before it and the big repository is recreated, that revision would now be #1051. Additionally, although all history and dates are preserved in the revisions, the big repository has commits that are not in chronological order since the import was processed by repository. This inconsistent date/commit ordering can be confusing to some repository reporting tools and may actually render those tools useless to you when they are reporting by date.

Filtering by revision. Note that my svnadmin dump statement includes the -r argument, which specifies the beginning and ending revisions to dump. By default, I'm using "0:HEAD", which basically means "dump every revision", or "dump from the first revision to the HEAD, or latest, revision". Changing the beginning and ending revisions can be useful, especially when used with dates instead of actual revision numbers. For instance, you could change the value to {2007-01-01}:{2007-12-31} to only dump revisions that were committed in 2007.

Combining all of your smaller repositories into one big repository after the fact isn't a perfect solution, but it can be handy, and it's really easy to do when you have a script like this ready to run.

As it turns out, there is a 301 error section in the Subversion FAQs. It says that this typically means your Apache configuration is invalid (nope, the rest of my repositories worked just fine) or your repository has the same path as a literal directory on your web root. Ahhh!

Sure enough, my subversion path was http://myserver.com/xyz/, and I had a literal directory named "xyz" in the web root. I changed that directory name, and Subversion would then allow me to checkout the repository with no problem.

Well, you can find all of your views by querying the sysobjects table, and you can retrieve the SQL behind the views by querying the syscomments table. Something like this works well:

This will retrieve the SQL code and names of all the views in the current database. This simple query is the heart of the solution. But I would like to retrieve this information for all of the databases.

Well, it's easy enough to get a list of all the databases. The sysdatabases table in the master database has that list. You can query that table, perhaps filtering out some of the system or sample databases included with SQL Server:

Now just combine this information. To accomplish this, we'll build a stored procedure that will create a temporary table, loop through the databases and query each one for its views, insert the view information into the temporary table, and return the temporary table.

Something like this will do the trick:

Now just execute the stored procedure and review its output.

This portion of the solution just retrieves the data. After writing this, I developed a really short and simple ColdFusion application that would output the database name, view name, and SQL to a table, and used some simple JavaScript to make it easier to search and filter the views. The client-side methods used to view and work with the data are obviously up to you.