Notes: Are You “Cashing In” on Caching?

Caching is a way of saving data after it has been originally computed or loaded. Alleviates load from server(s). Makes them faster and “healthier”. Also improves the users’ experience.

On the server level, we can cache templates. ColdFusion has a trusted cache of compiled CFML. We can also cache data objects (queries, structs, arrays, CFCs, etc), and then render HTML content, either entire pages or HTML regions.

What is good to cache? Things that are global to all users, like lists of states, navigation, etc. Even if not for all users, things that are used repeatedly in the application. Things that aren’t subject to much change. The more variants of something (based on the number of inputs), the less it will benefit from caching.

So how do we find these things? Examine ColdFusion logs, look for long-running pages. Ugh. Better: Go through debugging output. Look for poor execution times. But usually we look at it in a dev environment where load isn’t heavy. Can also use getTickCount(), <cflog>, <cftrace>. Best: SeeFusion, FusionReactor, CF8 Admin will now give us more insight into performance of server and, more specifically, the threads or tasks that might be causing the heavy load.

Ask yourself:

1. How often do changes take place? The more the item is in flux, the less it is a caching candidate.

2. Up-to-the-second data is critical? No good.

3. How many cache variants are possible based on the variable input arguments?

4. What is the average count/size/length of each cache variant? Memory footprint might become an issue.

What is a “Cache Variant”? A single set of data derived from the application code and based on a single combination of values for its arguments. So if we have SQL with userid=#userID and active=#active#, these are two distinct inputs. If you have 1,000 users, and 2 active states, you have 1,000 x 2 = 2,000 variants for the query cache. Obviously, the number of inputs drastically increases the variant pool. But ask yourself: How many likely variants exist? There likely are many permutations that you know are very unlikely to happen. Consider this when thinking about the likely footprint of your caching. Give attention to where you cache as well. Perhaps memory, disk, database.

Measuring effects of caching. A 500ms page executed 60 times per minute is effectively taking 30s/1m. 30s x 60m = 1,800s per hour. If caching that page takes it down to 20ms, this will be 12s/1m x 60m = 72s per hour. You can see how drastic the caching is on overall system performance.

Trusted Cache. Only have this turned on in the production environment. Can be problematic for a dev server since files are changing constantly. Will recompile or depend on compiled cache based on date/time stamps of the files. This could cause problems when templates are deployed via FTP or source control. For instance, if you deploy an older version to roll back some code, if may not recompile the new code! It will use the cache! So be cognizant of this and clear the cache if that occurs.

Restarting entire cache: Restart Application Server. Ugh. But in CFMX7, can “Clear Template Cache Now” in ColdFusion Administrator. Brian Szoszorek have an article for clearing just specific caches.

“Save Class Files” option. In the CF Admin “Caching” settings. When enabled, generates and saves a *.class file for each CFML template executed in WEB-INF/cfclasses/ directory. Can save a small bit of load when server is restarted. Server-wide setting. This option often doesn’t actually give any decent speed gain, because it uses File I/O.

Query Caching. Adding cachedWithin or cachedAfter to <cfquery> tag. Note that CF Admin has a “Maximum number of cached queries” setting. Should be set high for a server-wide multi-application environment to be useful. Query caching is driven by: Query name, SQL statement, datasource, username/password, input arguments to SQL statement. All of these must be the same to use the cache. Even the tabbing/spacing of the SQL statement will cause it to not reference the cache if different!

Good caching examples: CachedWithin is good if something isn’t changing for a short period of time. Especially great when no inputs that would cause variants. Show top 10 news articles, cache for 15 minutes. CachedAfter is good when running a query on data that won’t change after a certain time. Show records for month of 6/2007. You could do cachedAfter with a date of 6/30/2007.

Pros of query caching: Built-in. Debugging output will show that it was cached. But many cons: No control of where queries “go” when being cached. In one big shared resource pool. Max number of cached queries is hard to set reasonably. Difficult to clear particular items unless clear whole cache. Cannot cache queries with <cfqueryparam> (until CF8). Cannot cache with <cfstoredproc>, however you can invoke the stored procs inside of <cfquery> with the EXEC command. No way to track or view the whole cache. Service/instance specific, not application specific.

Alternatives to data caching. Take queries and put them in Server, Application, or Session scope. But you will have to programmatically start managing that. This obviously can be done with not just queries but also structs, arrays, objects, simple vars.

Content Caching. You can use the <cfcache> tag. Allows you to cache entire contents of a page. Like <cfquery>, you give it a cachedWithin value. Stores the pushed HTML to disk. Decision to use cache is based on the URL requested.

Alternatives for content caching. You can use <cfsavecontent> then put it in a shared scope (server, application, session, etc).  Various custom solutions (cf_superCache, cf_accelerate, cf_cacheOmatic, cf_turboCache, etc).

Notes: Advanced CFEclipse

Presented by Mark Drew.

CFEclipse was started by Rob Rohan in 2003. Mark joined in 2004. Macromedia endorses it in 2005, and was bought by Adobe he joined the Eclipse Foundation. Flex Builder was built on Eclipse in 2006, and added the RDS plugin for Eclipse. And CFEclipse 1.3 has been released in 2007.

Eclipse automatically gives us cross-platform capability, version control (with CVS), Ant task runner, a generic text editor, a built-in web browser, search, and more. CFEclipse sits on top of this functionality.

CFEclipse gives us tag completion, syntax highlighting, outline view of code, methods view, web help, snippets. It has a file explorer view for those of us who like the HomeSite style of editing files. Scribble pad for quick code testing. Multiple syntax libraries (to support BlueDraon, Railo, etc), toolbars like HomeSite, component explorers, variable insight.

Snippets

Can create snippets to effectively “paste in” code you may use over and over again. Hit Ctrl-J or Cmd-J to pull up a snippet after typing the trigger text. Can have snippet variables to insert custom code when pasting the snippet.

<mycode>$${MyCustomVar}</mycode>, for example, will insert what you put in msgbox when it asks for that variable. You can also define default values. For instance, $${MyCustomVar:The default value} will put “The default value” in there.

Development Patterns and Support

The typical Eclipse project paradigm is for local development. Working in the Eclipse workspace and probably use Subversion.

Otherwise, can use “File Location” or “FTP/SFTP” locations to directly edit some text that is somewhere outside your workspace, either on another machine, a file share, etc.

Ant

Ant scripts are typically called build.xml. When in the build.xml file, hit Ctrl-Space to get a build file template. You can then just work on the template.

Ant builds are “Target” oriented. They are basically collections of tasks that can be accomplished. A target can have a depends=”" attribute that will not run until another target is executed. You can then use commands to copy files, zip files, etc. For instance, <copydir>, <mkdir>, <input>, <echo>, <svn>, <ftp>, etc.

Good articles on Ant with SQL Scripts, Ant with Subversion, and Ant with FTP.
Unit Testing

A CFUnit plugin just had a soft release at CFUnited! Supports both CFUnit and cfcUnit. With it, you can do unit testing from right within Eclipse rather than going to a web page and doing it. Sweeeet!

Framework Explorer

Just like there is a CFC explorer, there is a framework explorer. When you open the view in your project, it will automatically figure out what frameworks you’re using, it will show them, and you can open them up and see your beans and other various components of your frameworks. Sweeeeet!

SnipEx

Can have snippets that are stored on a central server.

Notes: LiveCycle Data Services

LiveCycle Data Services used to be separate from CF. Now they are together, so you don’t have to configure the RMI to have them talk to each other.

With it, using AS3 and Flash Remoting, you can do stuff like invoke CFC methods from within ActionScript!  No Flex server is involved.

Messaging. Publish and subscribe. I say hello and everyone will listen to me. Someone says goodbye and I hear, and know not to talk to them anymore. ColdFusion can be both a Producer and a Consumer. Publish with sendGatewayMessage() and subscribe with an event gateway CFC.

New in CF8? Faster. Now uses Java API instead of that RMI.  Starting with CF8, it is included right in the CF installer. An express version comes right with CF8.

Not many notes on this, I wasn’t following too closely since we won’t be getting LCDS, at least not until I’m more heavy into Flex.

Notes: Working With RSS in CF

Presented by Pete Freitag.

Really Simple Syndication. Just an XML file. Like WDDX: They're both syndication structures. But RSS is a defined structure, so it makes it easier to consume because you know exactly what the standards should be for it.

Studies have shown that 69% of those surveyed don't use RSS, 27% are unaware of it, but use RSS, and 4% are aware RSS users. So about 1/3 of population use RSS, but it has successfully evaded technical knowledge requirements for people to use it. RSS feed is one file (called a channel) with multiple articles (called items).

Why publish RSS? Can increase traffic; people subscribed will return to your site when their reader notifies them that new info is available. Also more convenient for your readers. What are the downsides? Easier to steal your content when it is in a prestructured RSS feed. Can actually increase your bandwidth too, because readers may check your feed every hour or something.

How used? Portals like Google Homepage, RSS readers (desktop apps, Google Reader), Aggregators (like MXNA, Fullasagoog), RSS to email. Not just for blogs; can be used for news articles, forums, newsletters.

RSS vs. ATOM. RSS 2.0 is simple and gets the job done. ATOM is more complex with some add'l features. Typically, the add'l features are not used. For instance, you can post multiple versions for different content types (e.g. plaintext vs. HTML).

Generating an RSS Feed

Start by resetting any content. Then immediately do the XML header because it MUST be on the first line.

CFM:
  1. <cfcontent reset="true"><?xml version="1.0" ?>
  2. <cfheader name="Content-Type" value="text/xml">

Nice thing about setting it to "text/xml" is it will actually display the XML when opening in the browser. Nice for viewing the results. Can also choose "application/RSS+XML".

XML:
  1. <rss version="2.0">
  2.   <channel>
  3.     <title>My News Feed</title>
  4.     <description>This is my news feed!</description>
  5.     <language>en-us</language>
  6.     <lastBuildDate>Wed, 27 Jun 2007 16:46:00 GMT</lastBuildDate>
  7.     <item>
  8.       <title>This Article Title</title>
  9.       <link>http://you.com/news.cfm?id=1</link>
  10.       <description>My article content.</description>
  11.       <pubDate>Wed, 27 Jun 2007 16:46:00 GMT</pubDate>
  12.       <guid>http://you.com/news.cfm?id=1</guid>
  13.     </item>
  14.   </channel>
  15. </rss>

Dates in RSS feed must be RFC 822 compliant. Use the GetHttpTimeString() function to display it in the proper format.

So, when doing this with <cfquery>/<cfoutput>, you might do something like:

CFM:
  1. <rss version="2.0">
  2.   <channel>
  3.     <title>My Feed</title>
  4.     <description>My feed description.</description>
  5.     <language>en-us</language>
  6.     <lastBuildDate>#GetHttpTimeString(Now())#</lastBuildDate>
  7.     <cfoutput query="myArticles">
  8.       <item>
  9.         <title>#myArticles.Title#</title>
  10.         <link>http://you.com/news.cfm?id=#myArticles.ID#</link>
  11.         <description><![CDATA[#xmlFormat(MyArticle.Story)#]]></description>
  12.         <pubDate>#GetHttpTimeString(myArticles.DT)#</pubDate>
  13.         <guid>http://you.com/news.cfm?id=#myArticles.ID#</guid>
  14.       </item>
  15.     </cfoutput>
  16.   </channel>
  17. </rss>

Send your feed to feedvalidator.org to see if it is valid. Helps you get your feed up and running.

You can also use enclosures, which is most used for podcasting. It's like an "attachment" to your feed. Technically, you can only have one enclosure. It may allow multiple enclosures, but because you're not supposed to, this can be problematic.

XML:
  1. <enclosure url="http://you.com/audio.mp3" length="12345" type="audio/mpeg" />

Length is in bytes. Can obviously be more than just audio files. Could be a Word or PDF file.

Can also add a <category> tag within the <item> tag. Can add multiple tags. Technorati bases its "tags" on these categories in RSS feeds. Can also add <author> (must be an email). Can also add <source>, where it refers to a source article you may be writing about or rebutting.

More <channel> tags:

  • copyright: Copyright notice.
  • ttl: Time to live. Number of minutes to cache the feed.
  • image: Include an image for the RSS feed.
  • skipDays: Listing of days that the feed will not change.
  • skipHours: Listing of hours where the feed is not changed.

Can see the RSS 2.0 spec for more info.

Feeds in ColdFusion 8

The new <cffeed> tag can create AND parse RSS feeds.

CFM:
  1. <cffeed action="create"
  2.         name="#feed#"
  3.         outputfile="rss.xml"
  4.         overwrite="true"
  5.         xmlVar="xmlout">

Name will be a struct with the content. xmlVar allows you to put the generated RSS XML into a variable instead of the file.

So create a struct that contains title, description, link, and item which would be an array of structs with the various item elements (title, description, pubDate, link). This way, you can work with familiar ColdFusion structs and arrays when generating this stuff, then just feed it all to <cffeed>.

Parsing feeds is pretty easy.

CFM:
  1. <cffeed action="read"
  2.         source="http://you.com/rrs/"
  3.         query="feed"
  4.         properties="feedMetaData">

feedMetaData will be the general channel metadata, and the feed variable from the query="" attribute will be a query form of the items.

Problem with the returned query is that it returns a million columns, and the columns you need to use are different for ATOM vs. RSS. It would be nice if there was one common set of fields that are set depending on whether the feed is ATOM or RSS.

So if using RSS, you may do something like:

CFM:
  1. <h1><cfoutput>#feedMetaData.title#</cfoutput></h1>
  2. <cfoutput query="feed">
  3.   <h2><a href="#feed.rsslink#">#feed.title#</a></h2>
  4.   <p>#feed.description#</p>
  5. </cfoutput>

So <cffeed> is really nice, although problematic due to the different ways it handles ATOM vs. RSS feeds. This isn't really their fault, but it would be nice if there was a black box that took care of the translation between the two feeds.

  Theme Brought to you by Directory Journal and Elegant Directory.