Friday, December 22, 2006

JavaScript Support in Java 6 is WORSE than old Rhino 1.6

Java 6 is out, and with it, it includes built-in support for the Rhino JavaScript interpreter, warts and all. I've had a lot of problems with the Rhino JS interpreter, but astoundingly, the Rhino built-in to Java 6 is actually worse than the Rhino you can download from mozilla.org.

How, you ask?

Exceptions don't work

Well, first, it's got all of the problems you'll find in standard Rhino. The most annoying problem you'll encounter is that exceptions don't work. Here, eval this (in either Rhino 1.6R5 or Java 6):

try { throw new Error("urk"); } catch (e) { throw e; }

To eval this in Rhino, put this text in a file called "bug.js", run Rhino with "java -jar js.jar" and type "load('bug.js')".

"bug.js", line 4: exception from uncaught JavaScript throw: [object Error]

There's two things very wrong with this, one more serious than the other. The first thing wrong with it is that it's reporting the error on the wrong line number. The error was thrown from line 2, but rethrown in line 4. The stacktrace should show the source of the original throw (line 2), not the source of the rethrow (line 4). You'll find this as bug 363543 in Rhino's bugzilla. (I'd suggest that you vote for it, but you can't vote for bugs in Rhino!)

But the second problem is actually way more serious: it's missing the error message of the exception ("urk")! That's bug 351664 (recently patched, but not yet released in any official Rhino release). Since the JS interpreter didn't get the line numbers right either, you now know nothing about the cause of the exception!

Unfortunately, bug 351664 doesn't just apply to rethrown exceptions... it applies to regular old thrown exceptions, too. Here, eval this:

throw new Error("oh no!");

All you get is "[object Error]"... no hint of the error message.

Java 6 won't compile JS into .class files

The coolest feature of Rhino, IMO, was its tight integration with Java classes; you could compile JS directly to .class files that you could run directly in the JVM. (The IKVM guys used this trick to run JS on Mono.)

While Java 6 provides new support for a Compiler API, it doesn't even provide built-in support for compiling a string (you have to roll your own classes for that little convenience), and absolutely no integration with any other compiler, including the Rhino compiler. That means no JS files as classes.

Java 6 has no JavaScript debugger

But aside from inheriting all of the problems in the Rhino implementation, you'll soon find that Java 6 is lacking a critical feature present in Rhino. Rhino includes a JavaScript debugger, allowing you to set breakpoints, step in, step over and step return throughout your code, as well as set up watch expressions, observe the state of local variables, and so on.

But the new JSR 223 scripting API includes no debugger support. There's no way to stop on line 2330 of a long JavaScript file, set breakpoints, observe script-level variables, etc. The Java 6 scripting engine is just a black box with an eval function.

Implementing a JavaScript Language Interpreter for Selenium Testing

In another article, I explained why HTML Selenese has no support for "if" statements and conditional "for" loops.  There, I argued that implementing flow control "properly" in Selenium Core would require writing an entire language parser/interpreter in JavaScript.  Here I'll catalog some of our failed attempts to do precisely that.

Use the Browser's Native JS Parser

The first question everybody asks us is: "why don't you let people write Selenium tests in JavaScript directly, rather than writing them in HTML Selenese tables?"  JavaScript already includes (as part of its language) an "eval" function that can execute arbitrary JavaScript encoded as a string; JavaScript will parse the string, interpret the code, and even return the final result.

You can actually write tests in JavaScript today using Selenium Remote Control and Rhino, the JavaScript interpreter for Java.  The disadvantage of this is that your JS test runs in its own separate process (a Java JVM), and it requires you to set up a Selenium Server.  It doesn't run the JavaScript directly in the browser.

Using Selenium RC, you can write a JS test like this:

    selenium.open("/mypage");
    selenium.type("login", "alice");
    selenium.type("password", "aaaaa");
    selenium.click("login");
    selenium.waitForPageToLoad(5000);
    selenium.click("link=Colors");
    selenium.waitForPageToLoad(5000);
    selenium.click("blue");
    selenium.waitForPageToLoad(5000);
    if (!selenium.isTextPresent("blue moon")) throw new Error("blue not present!");

However, this test requires Selenium RC and a running Selenium Server.  You can't run that test directly in the browser for a very important reason: JavaScript has no "sleep" function; the JavaScript interpreter in all browsers is, by design, single-threaded.  That means that there's no way to implement "waitForPageToLoad" as it's written here.  In another language you might implement a function like "waitForPageToLoad" by doing something like this:

    function waitForPageToLoad() {
        while (!isPageLoaded() && !timeIsUp()) {
            sleep(500);
        }
    }

How can we do this without a sleep function?  Let's try a standard "busy wait" function (i.e. constantly perform mathematical calculations until our time is up).

    function sleep(interval) {
        var i = 0;
        for (var start = now(); (start + interval) > now();) {
            i++;
        }
    }
    
    function runTest() {
        frameLoaded = false;
        var myFrame = document.getElementById('myFrame');
        var myWindow = myFrame.contentWindow;
        myWindow.location = "hello.html";
        var start = now();
        var finish = start + 5000;
        while (!frameLoaded && finish > now()) {
            sleep(500);
        }
        alert("frameLoaded="+frameLoaded);
    }

You can try this test here: Busy Wait.  I tested it in Firefox and IE.  What you'll find is that the window frame refuses to load as long as the busy wait loop is running.  As soon as the test is finished (failing), the frame loads.  (In FF the frame loads while the alert pop-up appears; in IE the frame doesn't load until after you click OK.)

Under the hood, here's what's really happening: when you set the window.location in JavaScript, you haven't actually done anything yet; you've just scheduled a task to be done.  Once your task is completely finished, the browser goes on to perform the next task on the queue (which, in this case, is loading up a new web page).

To put that another way, there's a reason why JavaScript has no "sleep" function: architecturally, it would be pointless.  Normally, you sleep for a few seconds while something else happens in the background.  But as long as your JavaScript is running in the foreground, nothing can happen in the background!  (Try clicking around in the menus, or even clicking the close button, while the busy wait test is running.  The entire browser is locked!)

For a while, I thought I had found a clever workaround to this problem: using a Java applet to do the sleeping.  You can see this in action here: Applet Wait.  The page loads!  That's great.  But then we try to wait for the additional salutations onLoad... and they never come.  (If you run "top" or the Windows Task Manager you can see that this mechanism doesn't pin the CPU; the CPU is idle.)

Actions you perform in JavaScript can have multi-threaded effects, even adding items to the queue of work to be done, but as long as any JS function is working, no other JavaScript work can happen in the background.

Waiting With setTimeout

So how does Selenium do it?  JavaScript exposes a simple mechanism that allows you to schedule events for the future: setTimeout.  It places an item on the queue of events to be done, after a short delay.  It's like saying to the JS interpreter "run this snippet of code (as soon as you can) 500 milliseconds from now."  That parenthetical "as soon as you can" is critical here, because of course, if the JS interpreter is busy at that time (e.g. if somebody is working or is in the middle of a busy wait,) the timeout event won't happen until that earlier job is done.

That means that you still can't write the JS test I highlighted at the beginning of this article:

    selenium.click("blue");
    selenium.waitForPageToLoad(5000);
    if (!selenium.isTextPresent("blue moon")) throw new Error("blue not present!");

Because you cannot simply sleep and then assert something.  Instead, you have to let the interpreter sleep on your behalf, and then call you back when it thinks you might be done.

The Scriptaculous Test.Unit.Runner exposes a common mechanism for writing asynchronous tests using setTimeout: a "wait" function, appearing as the very last line of your test, that includes the "rest" of your test (everything to be done after you're finished waiting).  It must be the very last line of your test because (like the setTimeout function), "wait" simply schedules an event to happen in the future; it doesn't synchronously wait for the job to get done.  A Test.Unit.Runner test looks a little like this:

    selenium.click("blue");
    waitForPageToLoad(5000, function() {
        if (!selenium.isTextPresent("blue moon")) throw new Error("blue not present!");
    }

That might not look so bad from here... but remember that you have to nest these "wait" statements every time you want to wait; a standard Selenium test needs to wait after almost every line!  So the simple looking JS test I quoted at the beginning of this article would look like this:

    selenium.open("/mypage");
    selenium.type("login", "alice");
    selenium.type("password", "aaaaa");
    selenium.click("login");
    waitForPageToLoad(5000, function() {
        selenium.click("link=Colors");
        waitForPageToLoad(5000, function() {
            selenium.click("blue");
            waitForPageToLoad(5000, function() {
                if (!selenium.isTextPresent("blue moon")) throw new Error("blue not present!");
            }
        }
    }

Every time you "wait", you have to create a new nested block.  This code starts to look pretty gnarly pretty quickly...  But that's really the least of it.  The real kicker is that you can't really use for loops together with "wait" statements!

Don't forget, the "wait" statement has to appear at the end of your function.  That means that you can't click/wait/assert on 3 things in a for loop, like this:

    for (var i = 1; i <= 3; i++) {
        open(page[i]);
        wait(50, {
            assertTrue(document.foo == i);
        }
    }

Naively, we might assume that this code would do the following (which is what we'd want):

open(page[1])
sleep(50)
assert(foo==1)
open(page[2])
sleep(50)
assert(foo==2)
open(page[3])
sleep(50)
assert(foo==3)

But, instead, that code will try to open pages 1, 2 and 3 without any delay between them,[*] then wait 50ms, then do all three of your assertions.  In other words, you'd get this:

open(page[1])
open(page[2])
open(page[3])
sleep(50)
assert(foo==1)
assert(foo==2)
assert(foo==3)

[*] In fact, it's even worse than that, because page 1 wouldn't really open until your JS function was finished running.  So really it would only ever open page3 and then do all three assertions on page 3.

I don't want anyone to get the mistaken impression that I think the Scriptaculous Test.Unit.Runner "wait" function is altogether bad.  It's a useful function for doing unit testing, where you'll test one or maybe two asynchronous events at a time.  (Good unit tests test only one thing at a time, after all.)  It's a clever solution to a difficult problem.

But in functional integration tests like ours, where you have to wait after almost every line, using the "wait" function doesn't really help that much; it doesn't give you the ability to write complicated integration tests in a fully-powered language with logic, try/catch, and recursion.

"wait" gives you some of the power you need/expect in a modern programming language, but not enough: although you can use if statements and for loops within each nested "wait" block, you can't use them across code blocks.  (You still can't put an "open" command in a while loop, or in a try/catch block.)

Using Javascript in HTML Selenese

In fact, HTML Selenese makes it pretty easy for you to write asynchronous tests in JavaScript, simply by scheduling a timeout between every line of your Selenese test.  Since Selenese also supports running arbitrary JS in a table cell, it's not too hard to write your test like this:

storeEvalselenium.open("/mypage");
storeEvalselenium.type("login", "alice");
storeEvalselenium.type("password", "aaaaa");
storeEvalselenium.click("login");
storeEvalselenium.waitForPageToLoad(5000);
storeEvalselenium.click("link=Colors");
storeEvalselenium.waitForPageToLoad(5000);
storeEvalselenium.click("blue");
storeEvalselenium.waitForPageToLoad(5000);
storeEvalif (!selenium.isTextPresent("blue moon")) throw new Error("blue not present!");

That's not quite as nice as the test you can write with Selenium RC and Rhino, though, for a couple of reasons.  First, it suffers from all of the same problems as the Scritaculous "wait" function: although you can use "if" statements and "for" loops within each storeEval block, you can't use them across functional blocks.  (Again, you still can't put an "open" command in a "while" loop, or in a "try/catch" block.)

But in fact it's even a little worse than Test.Unit.Runner, because it means that you can't write a test like this:

storeEvalvar foo = 'bar';
storeEvalselenium.click(foo);

That's because "foo" was defined as a local variable in the first block; it goes out of scope as soon as the block finishes, and is undefined by the time we start the second block.  (Test.Unit.Runner doesn't suffer from this problem, because it uses closures to encapsulate the variables from the first block inside the second block.)

Generating Turing-Complete Selenese

Some people, when they see the storeEval tables I show above, start to thinking: "perhaps I could generate that table, from code written in a high-level functional language."

And, of course, you certainly could.  But, as we know, you certainly can't convert JS like this into Selenese:

    for (var i = 0; i < 3; i++) {
        selenium.open("page" + i);
    }

... or could you?  If Selenium had an "gotoIf" command, you could write the code like this:

storei0
gotoIf!(i < 3)5
storeEvalselenium.open("page"+i)
storeEvalii+1
gotoIfTRUE1
echodone

Every "for" loop is just a convenient way of saying "gotoIf"; with the addition of a simple command to the language, we would make it possible to translate simple "if" and "for" statements into Selenese.  "gotoIf" makes Selenese "Turing-complete". (In fact, there's a Selenium Core extension that provides this; I discuss the flowControl extension in more detail in an earlier article.)

"if" and "for" statements are easy, but what about try/catch statements?  What about JavaScript functions?  What about strings run in "eval" blocks?  How would we handle scoped variables (and guarantee that they go out of scope correctly)?  

If you understand this problem completely, you quickly see that the "translator" of JavaScript into Turing-complete Selenese is really a full compiler; it would require a complete language parser and interpreter to know how to translate try blocks, functions, nested scopes, etc. into their "gotoIf" equivalents.  It wouldn't be inappropriate to call it "Selenese Assembly", since it has a lot in common with assembly language: it's powerful enough to handle anything, but so complicated to write that you probably wouldn't want to write a lot of it by hand.

As I argued in the previous article, writing a full compiler for "Selenese Assembly" would be a lot of work, for basically no benefit, because we already have JavaScript support in Selenium RC.

Writing a Full Language Parser in Javascript

Writing a compiler for "Selenese Assembly" would be a lot of work, but that didn't stop a few people from trying.

Somebody has, in fact, written a language interpreter in JS: Brendan Eich, the "father" of JavaScript, has written a meta-circular JavaScript interpreter in JavaScript called "Narcissus" (named after the Greek myth of the boy who fell in love with his own reflection).  Jason Huggins, the original author of Selenium, began working on trying to integrate Narcissus with Selenium, but never finished.

The reason why he never finished is because we don't merely need a JS interpreter written in JS.  To implement a "sleep" function, we would also need the meta-circular interpreter to be able to interrupt its flow of execution using setTimeout.  That means that the meta-circular interpreter would need to be written under all of the constraints that I described in the previous section: it can't use any local variables (except as temporary storage until they get written out to permanent global variables) and it has (at best) limited use of for loops, try/catch blocks, and other language features of JavaScript.

Another way of putting that is that the meta-circular interpreter would need to be written in "continuations-passing style", meaning that all of the information about the current state of the running JS program (what line you're on, where you are in the stack, all of the local variables, scopes, etc.) would need to be stored in a variable that would be "passed around" to all of the other functions; this would allow you to "setTimeout" in the future, simply passing in the continuation object to the next chunk of your code.

Take a look at the Narcissus code for yourself... it's pretty complicated.  Rewriting it in continuations-passing style would require rewriting the whole thing from scratch... with one hand tied behind your back!

Having said that, there is another language in the world that is considerably easier to parse and interpret, and which actually makes it very easy to write your code in that style: LISP.  And, indeed, someone has written a LISP interpreter in JS.  Rewriting that code to support calls to "setTimeout" probably wouldn't be too hard, and at that point, you'd be able to write in-browser Selenium tests using the full power of LISP.

The only disadvantage?  You'd have to write your tests in LISP! ;-)

Oh, and did I mention that Bill Atkins has written a Selenium LISP client?  Or that you can use the SISC Java-based Scheme interpreter with our existing Java Client Driver?

Thursday, December 21, 2006

Why Is HTML Selenese So Simplistic?

Recently, some people have asked on the Selenium mailing list why HTML Selenese (the language of Selenium Core) doesn't include conditional "if" statements or "for" loops, or, more generally, why there isn't some way to reuse code (with functions or subroutines).  Adding them in would be pretty straightforward.  (Andrey Yegorov has written a "flow control" user extension that provides support for "if/goto" statements in HTML Selenese, but you really shouldn't use it.)  In this article, I'm going to explain why we don't just add support for conditionals and loops directly into Selenium Core (which is also why, in my opinion, humans shouldn't write tests that use the flow control extension).  In some cases, nothing is better than something, when the something is considered harmful.

First, I want to highlight that it is absolutely possible to write Selenium tests with "if" statements and "for" loops, just not in HTML Selenese.  Selenium Remote Control (S-RC) provides support for writing Selenium tests in any language, including Java, .NET, JavaScript, Perl, PHP, Python, and Ruby.  Officially, if you need to write a test that has an "if" statement in it, we recommend that you write your test in S-RC.

Goto Considered Harmful

The main reason why we don't want to add support for a "goto" statement in HTML Selenese is that the Go To Statement Is Considered Harmful.  In his classic 1968 article for the Association for Computing Machinery, Edsger W. Dijkstra successfully argued that use of "go to" statements (or "jump" instructions) makes one's code less intelligible, and that "the go to statement should be abolished from all 'higher level' programming languages (i.e. everything except, perhaps, plain machine code)".

The "goto" statement undermines the intelligibility of your code by eliminating the possibility of a clean representational structure.  Code based on "goto" statements quickly turns into "spaghetti", as the code jumps all over your program, seemingly without reason.  (If you've ever had to maintain a lot of code written in Basic, you know what I mean.)  That's why people invented higher level languages like Java, C#, JavaScript, Perl, PHP, Python, and Ruby.

Adding a "goto" statement would completely undermine the entire point of HTML Selenese: simplicity.  Straightforward non-branching tables are easy for non-programmers to read and understand.  Many Selenium users report that their customers are able to grok Selenium tests (and even fix bugs in them), which gets more people involved in the testing/requirements gathering process.  If we added "goto" to Selenese, and people actually used it, that kind of end-user/developer interaction would come grinding to a halt.

Nobody Needs to Write a Goto Statement

Ultimately, though, all of our programs are ultimately compiled down and assembled into plain machine code, which makes ample use of "jump" instructions in order to get its job done.  Under the hood, all of the fancy object-oriented features available in higher level languages are automatically translated into "goto" statements by compilers (who guarantee the correctness of the resulting assembly code).

[In fact, as I understand it, that's exactly how Yegorov uses his flow control extension: he never (or rarely) writes HTML Selenese by hand, but rather he generates his HTML Selenese from another higher-level programming language that makes no direct reference to "goto" statements.  In a very real sense, he "compiles" a higher-level language into HTML Selenese.  (Please correct me if I'm wrong about this; I'm basing this on an old post of his from March 2006.)]

That's certainly one way to do it, but I argue that it's totally unnecessary.  He could simply run his tests in a high-level language directly with Selenium RC.

Implementing Proper Flow Control

Could we implement "proper" flow control in HTML Selenese, instead of simple "goto" statements?  Think about what that means: if, for, while, do/until, try/catch, scoped variables, iterators, closures...  these are programming language features.  To implement proper flow control in Selenium Core, we'd have to implement (and maintain) an entire language parser and interpreter written in JavaScript.

For one thing, that's too difficult.  While JS interpreters have been getting more and more compatible as time goes on, it's still very difficult to write a complicated program that behaves the same way in all browsers, say nothing of writing a full language parser/interpreter in JS.

For another, this problem has already been solved for us!  Java, .NET, JavaScript, Perl, PHP, Python and Ruby all have excellent parsers, compilers and interpreters.  Why would we reinvent that wheel in JavaScript?

(In another article, I'll write up exactly how far people have gone in this direction... it's not pretty.)

Easier to Read/Translate Without Flow Control

Finally, there are two important advantages to not supporting flow control in HTML Selenese, even if we could do it "right".

1) By limiting the expressive power of the language, we make Selenese substantially easier to translate into other languages, as we do in Selenium IDE. You can use Selenium IDE to translate HTML Selenese directly into any language that we support in Selenium RC.  If we supported writing tests in a "full language", we wouldn't be able to provide translators into all of those other languages.  (Language translation is hard; the problem is substantially simplified when you don't have any sophisticated language constructs to translate.)

2) HTML Selenese is about simplicity; that's the whole reason the language exists.  Turning HTML Selenese into a full-blown scripting language, with all the advantages that would bring, would still undermine its simplicity (though not as badly as implementing "goto").

If you want a scripting language, use a scripting language!  We support that 100%! :-)


In conclusion: implementing flow control mechanisms in Selenium Core means supporting "goto" statements (which sucks) or supporting a full language parser/interpreter (which is brutally hard).  Since Selenium RC already supports the native flow control mechanisms available in Java, C#, JavaScript, Perl, PHP, Python and Ruby, and since the whole point of HTML Selenese is its simplicity, there's almost no payoff in implementing yet another parser/interpreter in JavaScript.

In another article I'll discuss a few of our failed attempts to support a full language parser/interpreter in JS.

Compass as compared with Maven's SNAPSHOT system

In this article, I'll describe some of the differences between Maven 2.x and the "Compass" interal home-grown system we use at work.  I'll first describe our repository layout, then describe our component descriptor file, and finally I'll summarize some of the advantages and disadvantages of using the different systems and suggest future work.

The Compass system was designed with Maven 1.x in mind.  The original developers had said, roughly: "You know, Maven's got the right idea, but this really hasn't been implemented the way we'd want it.  We should rewrite it ourselves from scratch."

Repository Layout

Like Maven, Compass has one or more remote repositories containing official built artifacts, (or "components", as we call them,) as well as a local repo on each developer's machine which caches artifacts from the remote repo and contains locally built artifacts.  Where Maven and Compass substantially diverge is in how artifacts are stored in the repository.

While Compass doesn't have a notion of "groupId", our remote repository is divided up into sections, like this:

    thirdparty/
        log4j/
        junit/
    firstparty/
        RECENT/
            foo/
            bar/
        INTERNAL/
            foo/
            bar/
        RELEASE/
            foo/
            bar/

[NOTE: This isn't exactly how it looks, but it's close enough.]

Within a given section, you find a flat list of components.  In this example, "foo" and "bar" are buildable components that we've created; log4j and junit are, naturally, components built by other people.  "RECENT" contains only freshly built components.  "INTERNAL" contains components that have been blessed by some human being, and are intended for internal consumption.  "RELEASE" contains released components and products.

In practice, there are 914 components in RECENT and 671 components in INTERNAL.

Within a given component directory, you'll find a number of subdirectories, which define the "version" of the component.  Thirdparty versions may have any arbitrary strings for their names (e.g. "3.8.1" "1.0beta3" "deerpark").  However, firstparty versions are strictly defined: they are simply the P4 Changelist number of the product at the time it was built.

(A quick note about changelist numbers as opposed to revision numbers.  Most people are familiar with the distinction between CVS revision numbers and SVN revision numbers: CVS revision numbers are "per file" whereas SVN revision numbers are global to the repository.  P4 changelist numbers are like SVN revision numbers.  [Also note that you can calculate something like an SVN revision number in CVS, simply by noting the timestamp of the most recent check-in.])

So, within the "foo" directory in RECENT, you'll see this:

    foo/
        123456/
            foo.jar
        123457/
            foo.jar
        123458/
            foo.jar
        @LATEST -> 123458

That's three numbered directories with a "LATEST" symlink, pointing to the most recent build in that directory.

The first thing to note about this system is that if you build 123458 and then rebuild 123458, it will replace the old "123458" directory.  The second thing to note is that if you change foo at all, it will get a new changelist/revision number, and so it will get a new subdirectory under "foo" once automatically built.

The three sections within the "firstparty" directory (RECENT, INTERNAL, RELEASE) are called "release levels", and we have a process about how components move into each release level.  "foo" and "bar" are automatically built every night and deployed into RECENT; if there are more than three builds in RECENT, we automatically delete the oldest build.

If somebody thinks that a build of "foo" is good enough to keep around, they "promote" that build into INTERNAL by simply copying the numbered changelist directory into INTERNAL.  Once we think it's good enough to release, we can promote that INTERNAL build into RELEASE by copying it there.  There is no tool, nor any need for a tool, to rebuild for release or make even the slightest changes to the released binaries.

Especially note that we don't put any of this information in the filename of the jar.  It's called "foo.jar" whether it's in RECENT, INTERNAL, or RELEASE.  We do burn the changelist number of foo.jar into its META-INF/manifest.mf at build-time...  that information remains constant whether "foo.jar" is copied to INTERNAL or RELEASE.

Component Descriptor File

Compass has a file that looks a lot like the Maven POM XML file... our file is called "component.xml".  component.xml defines a list of <dependency> elements.  Here's an example component.xml file:

<component>
    <name>foo</name>
    <release>6.1.0</release>
    <branch>main</branch>
    <depends>
        <dependency type='internal'>
            <name>bar</name>
            <branch>2.1.x</branch>
            <version>242483</version>
            <release-level>RECENT</release-level>
        </dependency>
        <!-- ... -->
    </depends>
</component>

Note that the component does not declare its own version number.  (Since in Compass-lingo, version numbers are SCM revision numbers, it would be impossible to declare this in the descriptor file; as soon as you checked in, it would be wrong!)  Instead, it allows you to declare a "branch" name, usually something like "main" or "feature" or "2.1.x" as you see above, as well as a "release", which looks more like a Maven version number, but is purely descriptive... it's not used for resolving artifacts at all.

Also note the presence of the <release-level> tag in the <dependency> element, which specifies the release level (RECENT, INTERNAL, RELEASE) of the dependency in question.

We do have a simple tool that automatically verifies whether a component is suitable for promotion to INTERNAL or RELEASE, which we call "DepWalker" (Dependency Walker).  You can use it to check to see if "foo" depends on any components in RECENT, or whether anything "foo" depends on (or anything they depend on, etc.) depends on components in RECENT.  Components in RECENT are temporary, and therefore unsuitable for long-term reproducibility.

Of course, if you like, you can also wire up your <dependency> tag to depend on RECENT/bar/LATEST.  In that case, you can continuously integrate with the latest version of bar.  You do that like this:

<dependency type='internal'>
    <name>bar</name>
    <branch>main</branch>
    <version label='LATEST'>242483</version>
    <release-level>RECENT</release-level>
</dependency>

The presence of the attribute "label='LATEST'" informs Compass that we want to automatically upgrade to the current LATEST version of "bar" that's available.

In an optional step we call "pre-build", Compass automatically modifies the number in <version> to match the LATEST version number.  If you don't "pre-build", the existing version number will be used.  The official nightly build system always runs "pre-build", and then automatically checks in the updated version into source control.

With that said, you don't have to use label=LATEST if you don't want to.  If you don't care about reproducibility, you can just say this:

<dependency type='internal'>
    <name>bar</name>
    <branch>2.1.x</branch>
    <version cl='242483'>LATEST</version>
    <release-level>RECENT</release-level>
</dependency>

Since the version number is LATEST, we can't reproduce this build later.  In that case, the automated build system still performs automated check-ins to modify the "cl='242483'" attribute, but that information is only there so humans can know what "LATEST" was at a given time, and so we can automatically bump the version number (by checking in, we increase the revision number). 

Finally, if you really don't care about reprodicibility, you say this:

<dependency type='internal'>
    <name>bar</name>
    <branch>2.1.x</branch>
    <version>LATEST</version>
    <release-level>RECENT</release-level>
</dependency>

In that case, your build is totally unreproducible, but it has the advantage that we won't bother with automated check-ins.

Advantages/Disadvantages Relative to the Current "Snapshot" System

Advantages:

  • "SNAPSHOT" is a marker that indicates that a given component/project is under development.  But that means that you necessarily have to modify the binaries in order to release them; using today's release plug-in, you actually have to check in modified source code before you can release, which is really troubling.

    The Compass system doesn't use a "SNAPSHOT" marker, and so promoting/releasing is simply a copy step.

  • Under today's SNAPSHOT system, there's no notion of a "build number" for a non-SNAPSHOT release.  If you deploy foo-1.0 today, then make some changes and redeploy foo-1.0, today's deploy system will simply replace the old foo-1.0 with the newer version.

    In Compass, everything always has a build number (and it's the same as the SCM revision number).

  • label=LATEST guarantees reproducibility while allowing for "soft" version numbering.  Maven only allows for unreproducible "soft" version numbers.

    Reproducibility is generally a virtue, but specifically it pays off when "foo" depends on "bar" and "bar" introduces a change that breaks "foo".  When "foo" is a reproducible build, you can say "this (automated) check-in 123456 broke the build of foo", see what changed in that check-in, and immediately identify the source of the problem (a bad new "bar").  It also allows you to roll back to an older check-in of "foo" to fix the problem.

    With unreproducible "soft" version numbers, you find yourself automatically upgraded to the new "bar", and no reliable way to determine this.  The same code in "foo" may build successfully on Tuesday but fail on Wednesday, with no apparent explanation as to why.

Disadvantages:

  • If you don't use a SNAPSHOT marker, it's not as easy to tell whether the file in question is an official release or not.  (This may matter a lot more to open source developers than closed source developers.  Open source developers make pre-release builds available for public download, which creates a risk that someone may download a pre-release binary and then ask for support.  Closed source developers typically keep pre-release binaries a secret and only make release binaries available through official channels.)

    Since typically under the Compass system even the filename doesn't include version information, the only way to figure out the "version" (changelist/revision#) of a given jar is to crack it open and look at the manifest.mf file.  Even that won't tell you whether the jar has been officially released, but it should be enough information for you to check to see if it is an official release at all.

  • label=LATEST does automated check-ins...  in some cases, it does a lot of automated check-ins.  These can clutter your revision logs.

Conclusion

I wouldn't want to suggest from this writing that anyone should cast aside the existing "SNAPSHOT" system in favor of Compass.  However, it is a major goal of mine to make Maven's release mechanism powerful enough that we could follow/enforce the Compass system using Maven.

Here's what I'd like to do, in no particular order:

  • Allow users to optionally specify a named repo within which you require that a given artifact can be found.  (Internally, we'd probably use this as an equivalent to our <release-level> tag, but I think this would be generally useful just to make it simpler to diagnose problems when a given artifact can't be found.)
  • Modify Maven's deploy mechanism and repository layout to ensure that a build number is always available in the remote repository, even for official releases.
  • Allow POMs to declare a dependency on a particular jar/version/build number, even for official releases.
  • Enhance the build-numbering mechanism to allow Continuum and other continuous build engines to deploy using an SCM revision number as a build number.
  • Create a mechanism that will allow you, if you wish, to automatically upgrade dependencies in POM files, by declaring both a literal version number + build number as well as a "soft" version number which serves only as a guideline for the automatic upgrade tool.

Sunday, January 29, 2006

Palm OS Development Suite (PODS) in Anger

First post, I guess. I've decided to create a separate blog for my random technical musings.

Today, I wrassled with the Palm OS Developers Suite (PODS). Getting it installed was a real challenge, so here's the short version of what I learned today:

If you already have Cygwin installed:

  1. Make sure you already have all of Cygwin's relevant "devel" packages.
  2. Download PODS from PalmSource. (Create a free login.)
  3. Run the PODS installer, but don't let it install its own Cygwin.
  4. Download the Cygwin bz2 files from the PRC-Tools download page
  5. Create a dummy setup.ini file and put it in the same directory as the bz2 files
  6. Point Cygwin at your local directory to make it install the bz2s.
  7. Make sure Cygwin and the other Palm resources are added to your path
  8. Run "palmdev-prep" to inform it where you installed Garnet
  9. Launch Eclipse, create a 68k project with the sample application, and try to build it. It should build successfully.
  10. Create a new Debug configuration. (You can't just "Run As" or "Debug As" the way you normally would debug in Eclipse.) "Run -> Debug..." Click on "Palm OS Application" in the "Configurations" panel, and click the "New" button at the bottom there. Click on "Target" and select the "Palm OS Garnet Simulator 5.4" (not the "Palm OS Emulator"; they're different). Click Debug.
  11. You'll get a pop-up saying "The selected Target Environment is being launched. Press OK when it is done launching, or Cancel to stop the debug session." The Palm Simulator should load up, and ask you for a ROM file. For the Simulator (not the Emulator), the ROM files are in sdk-5r4\tools\Palm_OS_54_Simulator\debug. You can use the enUS or EFIGS (English/French/Italian/German/Spanish) ROM; I used "Simulator_Full_EFIGS_Debug.rom". The Simulator should start.
  12. Back in Eclipse, click "OK". Your application should be paused on its first line of code in the "PilotMain" function, ready for you to begin debugging it.

Installing Pre-Requisite Software for PODS

It used to be that compiling Palm software wasn't possible without shelling out big bucks for Metrowerks CodeWarrior. Times have changed, and it is now possible to compile Palm software using GCC and PRC-Tools. But configuring these tools on Windows is challenging to say the least. At a minimum, it requires Cygwin, the Linux-like environment for Windows, whose setup tool is notoriously finicky. Moreover, PRC-Tools doesn't come with the "base" Cygwin packages, and PRC-Tools' automated Cygwin-ified installer is busted.

So the good folks at PalmSource decided that it would be a good idea to package up a known-good version of Cygwin + GCC + PRC-Tools, along with the Eclipse C Development Toolkit (CDT), and make an installer out of it.

If you like, you can go grab PODS for free from PalmSource. You have to create a free login, but then you can download the installer and get to work.

All fine and dandy, but the version of Cygwin they packaged up was rather out of date. Worse, if you read the README instructions carefully, you'll find that you must not install their Cygwin on top of your existing Cygwin, lest they undergo Total Protonic Reversal.

NOTE: If you choose to install the Palm OS Developer Suite provided Cygwin, you should remove all previous Cygwin installations since Cygwin does not support multiple installations of that product to coexist on one machine. Palm OS Developer Suite installer modifies the Cygwin registry settings as part of the installation process and therefore may negatively affect any previous installations.

If you're like me and you already have Cygwin installed, you'll need to follow their readme and, using the Cygwin setup tool, carefully ensure that you have every package on the list that they demand. In my case, I already had all of them just by installing the basic "Devel" package for Cygwin, except for, of course, PRC-Tools.

I did mention that PRC-Tools' Cygwin installer is busted, didn't I? Well, it is. The PRC-Tools Installation Instructions for Windows/Cygwin suggest that you just point Cygwin at "http://prc-tools.sourceforge.net/install" and let 'er rip. Bad idea. Cygwin will actually use that path to look at this file: http://prc-tools.sourceforge.net/install/setup.ini. That, in turn, is supposed to tell Cygwin where to download all of the packages referenced in the setup.ini file.

But, of course, it can't. Go on, try browsing to http://prc-tools.sourceforge.net/install/prc-tools/prc-tools-2.3-cygwin.tar.bz2. It won't work, because that's not really a bz2 file. Instead, it's a redirect to a SourceForge.net HTML page, telling you about the site from which it will *really* do the download. Your browser can handle these circuitous redirects, but Cygwin's setup tool definitely can't.

Instead, I had to download the binaries myself from the PRC-Tools Download page, and then hacking up my own setup.ini, that looks more like this:

# Cygwin-setup.exe config file for prc-tools et al.
# DO NOT EDIT! This file was generated by gensetup.
#
setup-timestamp: 1090157492
setup-version: 2.427

@ prc-tools
sdesc: "GCC for Palm OS and related tools"
category: Base Devel
requires: cygwin
version: 2.3
install: prc-tools-2.3-cygwin.tar.bz2 4285677 fb7bef9542d3ce6957535fd8315fcf0f

@ prc-tools-arm
sdesc: "GCC for Palm OS on ARM (armlets)"
category: Devel
requires: cygwin prc-tools
version: 2.3
install: prc-tools-arm-2.3-cygwin.tar.bz2 4459228 3bc19ac45a9aaee9f790b67f63f8cf97

@ prc-tools-htmldocs
sdesc: "Documentation for prc-tools, GCC, etc as HTML"
category: Devel Doc
version: 2.3
install: prc-tools-htmldocs-2.3-cygwin.tar.bz2 854795 bfe8f60a69b5ccfa3d247567150d463a
I put that file, and all of the bz2 files, into one directory, and then used Cygwin to install everything from a "Local Directory". That finally got prc-tools installed, apparently successfully. (But wait, there's more.)


Selecting the Correct Project Type: 68k, PNO, Protein

Once I was done with that, I tried running the PODS installer to get PalmSource's old version of Eclipse (3.0, missing the cool CDT enhancements available in 3.1, sadly), and told it not to install their version of Cygwin. That seemed to work OK, until it finally came time to try building a sample Project.

I was attempting to follow the instructions from Professional Palm OS Programming, a book which seems like it has most of the information you want, even if it isn't laid out particularly well. (And, sadly, the almost total lack of information available on Google make this book an unfortunately good value.)

As you attempt to create a new Project in PODS, you are immediately given the choice between three types of Projects: "68k", "68k PNO (PACE Native Objects)", or "Protein". The difference here is that Palms used to run on Motorola 68k processors, but most of the modern ones now run on ARM. In order to ensure backwards compatibility, the ARM-based Palms emulate 68k processors, in what they call "Palm Application Compatibility Enviroment" or PACE . (68k used to be big-endian, ARM is little-endian, and PACE handles that sort of thing for you.) If you want to write a teensy bit of native code for the ARM, you can write PNOs, which are ARM-native objects that can be used by 68k applications. And if you want to go whole-hog and write your entire application ARM-native, that's called "Protein."

But there's a catch. (Isn't there always?) Even if you DID want to write your entire application in Protein (the latest and greatest), you wouldn't be able to run it on anything except a Palm device that runs Palm OS 6.0 "Cobalt". But, as of January 2006, there are no Cobalt devices available for sale, even though Cobalt was released in early 2004! PNO will still work for Palm OS 5.4 Garnet applications, if you insist on going ARM-native, but you'll still have to write your application in 68k and then link to your "real" code in PNO.

So, as of January 2006, it's still best to just write your application in 68k... forget about crazy native ARM stuff for now.

Compling 68k applications: Configuring palmdev-prep

OK, so, back to PODS. If you try to create a Protein Project, (let's say you decide to create the Sample Application, which is a MineHunt game,) it will build and debug just fine. But if, realizing that no one can run anything written in Protein, you decide to create a 68k Project and then try to build that, you'll find that it won't compile. Instead, you'll get an error from prc-tools, suggesting that you try running "palmdev-prep".

Early versions of prc-tools contained a hard-coded list of directories to be searched. Unfortunately, more recently there has been a tendency for each new Palm OS SDK to introduce new directories unpredictably and sometimes even to rearrange the old ones. Hence the hard-coded list approach is no longer practical; instead, you can use palmdev-prep to generate a list of directories tailored to the SDKs and other Palm OS development material you actually have installed.

The palmdev-prep utility scans the standard PalmDev directory and any extra directories listed on its command line, and generates the options required to make GCC search as appropriate each of the subdirectories found under each of the root directories given. The "standard PalmDev location" is determined when prc-tools is configured; typically on Unix it is `/opt/palmdev' and on Cygwin it is typically `/PalmDev', which typically corresponds to the Windows directory `C:\PalmDev'.

Pay attention here, kids, because unless you had 20/20 foresight, you probably didn't install PODS into C:\PalmDev. Instead, you probably installed it into the default location, "C:\Program Files\PalmSource\Palm OS Developer Suite".

That means, to get your 68k project to compile, you'll need to make a symlink, in Cygwin. Launch your Cygwin bash shell and then do this:

> cd /
> ln -s "C:/Program Files/PalmSource/Palm OS Developer Suite" PalmDev
> palmdev-prep
"C:/Program Files/PalmSource/Palm OS Developer Suite"

Even once I did this, I still got a warning saying: "palmdev-prep: warning: can't open '/PalmDev/sdk-5r4/include/Core/CoreTraps.h': No such file or directory", but it didn't appear to do any damage. At that point, my Palm 68k application compiled just fine.

Debugging your Application

Debugging Palm Applications is a bit awkward relative to debugging standard C applications or Java applications in Eclipse, because even though techinically you are creating what might be called an "application", the output of the compiler is a .prc file, not something that's actually executable on your operating system. The applications you actually use for debugging are the Simulators/Emulators that get installed with PODS.

Since you won't be using Cobalt until somebody finally gets around to creating a device for it, you'll probably mostly want to use the Garnet Simulator (Palm OS 5.4) or the Palm OS Emulator (POSE), which emulates devices up to Palm OS 4.1.2. To start up a Simulator/Emulator and use it, you'll need to select a ROM file... note that Simulator ROMs don't work with the Emulator, and vice versa. All of the ROMs you need should be included with PODS. POSE ROMs live in "PalmOSTools\Palm OS Emulator\ROMS"; Garnet ROMs live in "sdk-5r4\tools\Palm_OS_54_Simulator\debug". There are several Garnet ROMs, meant to support numerous languages. I tend to prefer the EFIGS ROM (English/French/Italian/German/Spanish), but you may prefer just the plain-vanilla enUS ROM. Regardless, always make sure you use the "Debug" ROMs instead of the "Release" ROMs, because they're the ones with the handy debugging facilities you'll use to debug your Palm application.

Conveniently, Eclipse/PODS can be configured to automatically launch the Simulator of your choice, though it is a little bit screwy if you're familar with Eclipse. Instead of clicking on "Run As..." you need to click on Run -> "Debug..." and click on "Palm OS Application", then finally click on the "New" button to create a new configuration. In the "Target" tab, you're allowed to pick your own Simulator/Emulator. (If you're clever, you can even use this to debug a real live running Palm plugged into your computer over a serial port (COM1/COM2) or over USB, but I haven't tried it yet.) I recommend the Garnet Simulator for now.

At that point, you should be able to click the "Debug" button on your configuration to launch the Garnet Simulator. Once it starts, the Garnet Simulator will open up a port (port 2000 by default) and wait for Eclipse to connect to it. Eclipse will give you a pop-up saying "The selected Target Environment is being launched. Press OK when it is done launching, or Cancel to stop the debug session." Once Garnet looks like it's good and started, click OK, and Eclipse should pause on your first line of code.

I had a very hard time getting this to work on my first try, though. Instead, it at first told me: "Launching (Error: Target request failed: Target is not responding (timed out).)" Apparently I'm not the only one to have this problem, but Google provided very little in the way of helpful workaround information. A big part of the problem is that all the magic is happening by opening ports. Anything could go wrong in this picture... a common thing that can happen is that another application may be holding the port, so PODS can no longer use it. Another common problem is that a software firewall like Symantec, McAfee or ZoneAlarm may be blocking port-to-port connections, or even the built-in firewall in Windows XP SP2. So let's consider some common tricks you can use to test this.

First, if you're having trouble, I highly recommend that you download a copy of Process Explorer from Sysinternals. (The download links are at the bottom of the page.) Process Explorer is a really awesome piece of software; I can't recommend it highly enough. In this case, you'll use it by looking for any PalmSim* processes that may be running. Make sure you start with no PalmSim* processes before you begin debugging, and that the right one starts up when you push Debug. Get the Properties of the PalmSim process by right-clicking on it. There, you should be able to see the command line which was used to start it, which may tell you something useful. You can also click on the TCP/IP tab on the Properties dialog to see what ports the process is listening on and/or using.

One problem I had was that Eclipse accidentally "orphaned" my Simulator, so I had one invisible copy running in the background, hogging the ports. Once I killed that, Eclipse worked fine. Another problem I had was a software firewall blocking the port. An easy way to test this is to try to telnet to the port in question. So if on the TCP/IP tab I saw that PalmSim was "LISTENING" on Local Address "mymachine:2000", I could try "telnet mymachine 2000". If it worked, I'd expect to see a blank screen. (Press Ctrl-] to pause telnet, then type "quit" to exit.) If it didn't work, I'd expect to see something like: "Could not open connection to the host, on port 59939: Connect failed". Note that you should expect this if you telnet to the wrong port.

If you're still uncertain as to what's going on, try rebooting, and make sure all software firewalls (including the Windows XP Firewall) are disabled. Good luck!