Final Thoughts On: A Symbolic Puzzler

This is the final in a series of posts about a puzzler [ post containing the question ] [ post containing the answer ]. In those two posts I highlighted some bizarre Java weirdness as it pertained to the java.io.File.getCanonicalPath method, that canonicalized paths are cached, which means if a symbolic link changes somewhere the cached value becomes invalid.

Blatherberg

This problem really only ever crops up when your code relies on calls to getCanonicalPath and the links change while the application is running. If you expect your application to run against a filesystem with shifting symbolic links, you have to do one of these three things:

1. Disable the cache by setting the system property sun.io.useCanonCaches to false. (I would also like to briefly nod to the pedantic point that the string useCanonCaches is icky.)
   
   This seems like an easy out (particularly if your application is moderately complex, and deployed, and assuming your application doesn't rely on the default behavior,) but there's a reason Java comes with a canonicalization cache: performance. Reading symbolic links from disk can take time, especially if you do it a lot.
   
   Also, you might be running your application in a Java EE container along with other applications, in which case, you can't isolate the cache behavior to a single application.
   
   
2. Stop using getCanonicalPath (and its sugary sibling, getCanonicalFile) in your application, and rely solely on the non-canonicalized path.
   
   Changing your infrastructure to rely on the symlink paths themselves and not their canonicalized values sounds good, but you might not have control over that code: your application may rely on an application infrastructure that already relies on getCanonicalPath, and then you're kind of screwed. It's easy to say that the cache should be disabled in the name of correctness, but if you're repeatedly resolving symbolic links files by the thousand, the time cost may be significant.
   
   This leads to the other way to look at this problem, which is the lack of accessibility and flexible control over the cache. You might want to cache calls in some circumstances yet not others. The use cases for cache control can be complex, and by hiding the complexity you get, well, surprises like this.
   
   
3. Disallow an application's filesystem to redefine symbolic links.
   
   If you've got that power, go for it.
   

Help From JSR 203

There's actually some hope for the future, and that's JSR203: More New I/O APIs for the JavaTM Platform ("NIO.2") which is scheduled to be part of Java 7. Look back to the puzzler, which points out the use of Filesystem and UnixFilesystem classes. In JSR203, those ideas are explicit. The equivalent of java.io.File is java.nio.file.Path which exposes a method getFileSystem. That's right, the file system is no longer hidden from the user, and you can read all about java.nio.file.FileSystem here. You can have a file system that represents a thin layer on top of your disk, or one that caches all sorts of metadata from your disk, or, heck, create an in-memory implementation for high-speed storage! But the real benefit is that these filesystem implementations can be injected into your classes: no more need for a single static filesystem. Whereas java.io.File objects are created through a constructor, java.nio.file.Path objects are constructed through the FileSystem's getPath method.

This isn't disk i/o nirvana, unfortunately, because like the continuing transition from java.util.Date to java.util.Calendar to something more reasonable like org.joda.time.DateTime, there's still plenty of legacy code using the old and busted APIs. But it's a good start.

If you want some more information about JSR 203, here's a write-up by Alex Miller and a link to a JavaOne talk from 2008. The video is a bit out of date (for instance it highlights the notion of Path.get, which seems to be gone, thank goodness.) But it's got lots of great information about the JSR.

The Last Word

In the end, I want to highlight something underlying this entire journey: the choice to cache the values by default in the first place is just wrong. It reminds me of the saying (that seems to be attributed to Bill Harlan): "It's easier to optimize correct code than it is to correct optimized code.

Answer to: A Symbolic Puzzler

This blog post contains the results and answer to the previous post, A Symbolic Puzzler.

The answer will be covered here, and I'll follow this up with a fourth post that covers my thoughts on this issue.

What were your guesses?

Clearly, the most popular answer was that the test would fail.

What would have been my guess?

Look, nobody codes in puzzler fashion, so without details I'll explain what I expected to occur from my own production code, but in terms of this test. I wouldn't be confident that TESTDIR_SYMLINK.getCanonicalPath() returned the non-canonicalized location, but excepting that, I certainly would assume that once the symbolic link was created at the end, the second call to TESTDIR_SYMLINK.getCanonicalPath()would return the symlinked directory. So my guess would have been a. It passes.

The Answer

The correct answer is: d. It depends. More specifically, it depends on the VM's arguments.

The Explanation

If you ran this test straight up without any special VM arguments, the test would fail (which might lead you to think the answer is b. It fails.)

junit.framework.ComparisonFailure: expected:</testdir/[file]> but was:</testdir/[symlink]>
    at junit.framework.Assert.assertEquals(Assert.java:81)
    at junit.framework.Assert.assertEquals(Assert.java:87)
    at ATest.testSymlink(ATest.java:26)
    ...

Why wouldn't the canonicalization return the updated value? It's because the return value from getCanonicalPath was cached from the previous call. Yes, calls to getCanonicalPath are cached.

Let's look at the code underneath getCanonicalPath. The magic lies in some package-private classes in the java.io package, specfically Filesystem and UnixFilesystem. The key operation occurs in UnixFilesystem.canonicalize:

class UnixFileSystem extends FileSystem {
  public String canonicalize(String path) throws IOException {
     if (!useCanonCaches) {
       return canonicalize0(path);
     } else {
       String res = cache.get(path);
       ... 
     }
  }
  private native String canonicalize0(String path) throws IOException;
}

In other words, the path canonicalization computations are cached when useCanonCaches is true. So just when is useCanonCaches true? For that let's look at the static initialization block for Filesystem, the subclass of UnixFilesystem:

// Flags for enabling/disabling performance optimizations for file
// name canonicalization
static boolean useCanonCaches      = true;
static boolean useCanonPrefixCache = true;
... 

static {
    useCanonCaches      = getBooleanProperty("sun.io.useCanonCaches",
                                             useCanonCaches);
    useCanonPrefixCache = getBooleanProperty("sun.io.useCanonPrefixCache",
                                             useCanonPrefixCache);
} 

So by default, the cache canonicalization is on, but when you specify the VM arg -Dsun.io.useCanonCaches=false, the cache is never used.

Getting back to the puzzler, the first call to TESTDIR_SYMLINK.getCanonicalPath() always returns the path to the symlink, while the second call returns either the cached value (the path to the symlink) or the up-to-date resolved symlink, but only when -Dsun.io.useCanonCaches=false is specified.

If you don't beleive me now, go try running the test twice, once without specifying VM arguments, and once while disabling the canonicalization cache, and you'll see that it fails once, and passes another time. Hence, d. It depends.

Replies to some of the comments

Here are some of the comments that accompanied the survey:

Guess: It fails.
Comment:The target of the symlink doesn't exist so I suspect exists() will return false

In fact, exists() will return true since the symlink exists. And in fact, the next call to getCanonicalPath returns itself, just as the comment suggested.

Guess: It depends.
Comment: Depending on where the root filesystem is mounted, the canonical path might be something else. For example, /etc on Mac is a symlink to /private/etc. In addition, the mountpoint for / might be a networked drive (e.g. netboot) which might have different semantics.

The bottom line is that you can't necessarily assume that the file you use to access a file is the canonical path for that file, without knowing the filesystem.

This is an interesting comment. I had hoped this was cleared up by this statement in the original post: "The path /testdir is not a symbolic link to another directory, and the user running the test also owns /testdir." If my explanation was insufficient, then so be it.

There were other comments to that effect: "Does /bin/rm do what /bin/rm should do?" "Does the user have write permissions?" These are good points, and would be better suited for a UNIX puzzler. I hope people who worried about those cases looked past them to focus on Java's behavior.

Guess: It throws an exception
Comment: I want be the first one to check "it throws an exception"!

Sorry, not the first.

Meta: Thoughts on writing a puzzler

There are at least two places where the puzzler's code could have been simplified without sacrificing its quality:

1. Replace explicit calls that delete the files /testdir/file and /testdir/symlink, with a communicated precondition that /testdir had no files.
2. This test has an interim call that computes the canonical path of a symlink that points to nothing, leading people to spend too much time worrying about that case. A better snippet would:
1. point the symlink to a file that exists.
2. compute the symlink's canonical location thereby (optionally) populating the internal cache.
3. point the symlink to a second file that also exists.

Interim: A Symbolic Puzzler

Last night I published a Java puzzler for your enjoyment. I will publish the answer tomorrow, but for now I thought you would enjoy an interim count of the guesses:



There's still plenty of time to participate in the puzzler. Don't forget to use the text box if you want to back up your reasoning.

A Symbolic Puzzler

Update: If you're reading this post in Google Reader, view the original page to participate in the survey.

Here's a little Java puzzler I encountered back in August. Essentially, the test below creates a symlink to /testdir/file, and validates that the symlink is in fact pointing to it.

This test was run on a Mac Book Pro running OSX 10.5.8 using a 1.5.0 JVM, though I originally discovered this problem using Linux and a Java 1.6 VM. The path /testdir is not a symbolic link to another directory, and the user running the test also owns /testdir.

import java.io.*;
import junit.framework.TestCase;

public class ATest extends TestCase {
  public void testSymlink() throws Exception {
    // Setup
    run("/bin/rm", "/testdir/file", "/testdir/symlink");
    run("/usr/bin/touch", "/testdir/file");

    File TESTDIR_SYMLINK = new File("/testdir/symlink");

    // Symlink doesn't exist yet
    assertFalse(TESTDIR_SYMLINK.exists());

    // And so its canonical path points to itself.
    assertEquals("/testdir/symlink", TESTDIR_SYMLINK.getCanonicalPath());

    // Now point the symlink to the file.
    run("/bin/ln", "-s", "/testdir/file", "/testdir/symlink");

    // The symlink exists
    assertTrue(TESTDIR_SYMLINK.exists());

    // The canonical path should be up to date.
    assertEquals("/testdir/file", TESTDIR_SYMLINK.getCanonicalPath());
  }

  private static void run(String... args)
      throws IOException, InterruptedException {
    new ProcessBuilder(args).start().waitFor();
  }
}

These are your choices. Try to determine the answer by solely looking at the sample code. Vote for your preference, and I'll publish the results along with the correct answer.

Voting is open until some time early Wednesday morning, Oct 21.

Pretend you are your own customer.

Our talk in August at Eclipse Day at the Googleplex focused on what it's like to support a large number of users in an enterprise. Dwelling in this question got me thinking about what kind of Eclipse user I might have been five years ago, and so I looked up the first message I ever posted to our internal mailing list for Eclipse users.

I have been a Google employee for just over five years, and have only spent the latter three supporting Eclipse for my colleagues. At the time of this dialogue, I'd been at Google for less than two months, and my Eclipse experience was limited to about 18 months of mostly untrained use.

Before you bother reading it, please do not waste time troubleshooting the problem -- that's not the point if this post. This is a five year old issue, after all.

Robert Konigsberg
to <eclipse mailing list>
Subject: Problem with Eclipse 3.0 installation
10/10/2004

Hi all,

This afternoon I was working pretty happily in my Eclipse environment. All of I sudden I got a message telling me that I ran out of heap space, and advised me how to change my heap space. It offered to shut down Eclipse. I said "no". saved some work and quit myself.

When I tried to restart Eclipse I got all sorts of odd stuff. I can open Eclipse, but if I try to open the Java perspective, I get the error, "Problems opening perspective 'org.eclipse.jdt.ui.JavaPerspective'. Restarting my machine didn't help. When I try to edit Java preferences, I get similar errors.

Thoughts? Steve, is it Upgrade-To-3.0.1-Day for me?

Cedric Beust
to Robert Konigsberg, <eclipse mailing list>
Re: Problem with Eclipse 3.0 installation
10/11/2004

What does your .log say?  Is it running out of heap space at each launch? In which case, I suggest to launch Eclipse with more heap:

eclipse -vmargs -Xmx600M

Robert Konigsberg
to Cedric Beust
cc <eclipse mailing list>
Re: Problem with Eclipse 3.0 installation
10/11/2004

Don't know which .log I'm looking for. Note below (edited for readability)

~/.eclipse$ find . -name "*.log" -ls
1556 Sep 29 18:20 ./org.eclipse.platform_3.0.0/configuration/org.eclipse.update/install.log
1008 Sep 29 18:20 ./org.eclipse.platform_3.0.0/configuration/org.eclipse.update/error_recovery.log
~/.eclipse$ cd ../workspace/
~/workspace$ find . -name "*.log" -ls
1319 Sep 10 18:17 ./.metadata/.plugins/org.eclipse.tomcat/catalina.2004-09-10.log
1319 Oct 10 10:50 ./.metadata/.plugins/org.eclipse.tomcat/catalina.2004-10-10.log
1017 Oct 10 16:48 ./.metadata/.plugins/org.eclipse.help.base/browser.log
~/workspace$ find . -name "*.log" -exec vi {} ";"

Cedric Beust
to Robert Konigsberg
cc <eclipse mailing list>
Re: Problem with Eclipse 3.0 installation
10/11/2004

Sorry, should have been more specific...

The .log file is in your ${eclipse-workspace}/.metadata directory.

I've removed the rest of the dialogue. Suffice it to say that it was probably a JDT index corruption issue that was resolved by removing the workspace-root/.metadata/.plugins/org.eclipse.core.resources/.root directory.

Here's the point: it's a little embarrassing to see that while I've attained some expertise in this domain, even I had no idea where the workspace log was. Also, I called it the "Eclipse environment" which is kind of a little wrong.

I appreciated both the memory and the mild feeling of humiliation that came with pulling up this old email.

Story Time with Google Collections

Note: this post in no way suggests that Google Collections is wasteful. Quite the opposite, it's spectacularly awesome. If you haven't used it yet, go get it, play with it, discover appropriate uses of its power, and go rock your project.

Several months ago there was a discussion on one of the mailing lists at work about preference of part of the Google Collections API over an imperative equivalent. Specifically, using Iterables.transform to translate a List<X> into List<Y> by creating a Function that translates X into Y. In other words, which was better?

This:

Function<X, Y> function = new Function<X, Y>() {

  public Y apply(X from) {

    return X.asY();

  }

};

List<Y> listOfY = Lists.newArrayList(
    Iterables.transform(listOfX, function));

Or this:

List<Y> listOfY = Lists.newArrayList();
for(X x : listOfX) {
  listOfY.add(x.asY());
}

There was this argument about the value of the functional style over the imperative style, and frankly, I found it all rather confusing. All things being equal, the former was just too damn much. And that's the key here: all things being equal. If we were using a language with proper closures, or if there was a host of Function instances available to substitute for function, I might have (and have had) a different opinion.

To illustrate my point, I submitted this story to the mailing list.

Me: Kevin, tell me a story.

Kevin: Seriously. Go away.

Me: TELL!!

Kevin: OK. Once upon a time there was a programmer. He wanted a List. of Strings. But unfortunately, he had a List of Another Type.

Me: What type?

Kevin: Doesn't matter.

Me: WHAT TYYYYYPE!

Kevin: DOESN'T. MATTER.

Me: Ice Cream?

Kevin: Ice Cream. Fine Ice Cream. So the programmer...

Me: How much ice cream?

Kevin: What?

Me: How MUCH?

Kevin: Well, let me put it this way: if it was a List of ice cream, it would be A LOT of ice cream.

Me: Eeeeeeee!

Kevin: (pause) So... this programmer decided to take the list and return a New List. He decided to transform his list by creating a magical function that converts Another Type ... I mean ... _ice cream_ into Strings. And that Function has a method called 'apply'. Apply takes things of type ... sigh ... _ice cream_ and converts it to strings. It does this by calling its callStringMethod. The End.

Me: Wha?

--- REWIND

Kevin ... it would be A LOT of ice cream.

Me: Eeeeeeee!

Kevin: (pause) So... this programmer created a new, empty list. A place to store the Strings. And then he went through every _ice cream_ element in the old list, called its callString Method, and added it to the new list. And then he returned that list. The end.

Me: *sob* that is so awesome. What happened to the ice cream?

Kevin: I think Josh has it. Hurry up before he eats it.

Nobody here is saying that functional programming is bad or wrong. But sometimes it's not really all that awesome to be cutting edge.

Update: What I mean to say, that my colleague said in much fewer words, is that the primary concern in coding should be making your intention as plain as possible.

A Blatherberg Poem about C++

C++
I don't understand all the fuss
If you want a scoped pointer
Find a lady and anoint her
with watching your mem-o-ree!
Oh I want -- to -- go -- back -- to -- C.
With its malloc and its free
I just get so confused with delete and new
And what deletes who
And comparing an int to a boo-lee-an
I won't ever write a for loop
Thanks to Bjorne Stroutroup
And Alexander Stepanov
And I'll bet you've had enouv
of this poem.