Bundling Local JARs for your Groovy scripts the Grape Way

A FAQ which often does not receive a satisfactory answer is how to add local JAR files to a Groovy script without having to fiddle with the dreaded CLASSPATH environment variable. Folks see that things are quite nice when using Grape with a Maven repository and wish they could do the same with local JAR files. When creating IFCX Wings I implemented Groovy For OpenOffice (pre-Grape) which also used Apache Ivy for dynamically extending the classpath. One difference though is that I directly supported including JAR files which are not in a repository but are simply referenced by URL, whether local (file) or remote (http/https). It turns out though that Grape can be used with local JAR files in fairly nice fashion by keeping them in a Maven-format repository in the local filesystem. Here's how to do that in pretty friendly manner.

Use Maven to put the JAR files in a Local Repository

Maven will install files in a local repository for you without fussing around with a POM file. We'll use the mvn deploy-file command and a local file URL that is relative to where our script is located which will be file:repo in this example. Notice that the file URL has no leading slash and so is relative to the working directory. Naturally you'll need to use an appropriate groupId and artifactId (please be a good netizen and use a groupId that the reverse of a domain name that is assigned to you or your organization). For this example I'm demonstrating a little script that uses Apache Lucene and I'll use my groupId so that Grape will only use my copy. One flaw in this scheme at the moment is that transitive dependencies are not managed although it should be possible to work that out and I'll look into it later. The upshot of that is we have to list each JAR file individually, which shouldn't really be an issue since we're already just wrangling JAR files by hand anyhow.

Layout of my Working Directory

$ find *
lib
lib/lucene-analyzers-common-4.5.1.jar
lib/lucene-core-4.5.1.jar
whats_new.groovy

Top of my Groovy Script

#!/usr/bin/env groovy

@GrabResolver(name='script', root='file:repo')
@Grab(group='org.ifcx.lucene', module='lucene-core', version='4.5.1')
@Grab(group='org.ifcx.lucene', module='lucene-analyzers-common', version='4.5.1')

import org.apache.lucene.analysis.standard.StandardAnalyzer
import org.apache.lucene.document.Document
import org.apache.lucene.document.Field
import org.apache.lucene.index.DirectoryReader
...
The full script (using the default Maven Central repo) can be see in this gist https://gist.github.com/jimwhite/7609525.

Create the Local Maven Repository

We use the command mvn deploy:deploy-file -Durl=file:repo -Dfile=/path/to/jar -DgroupId=name -DartifactId=name -Dversion=ver to copy the JAR file to a new Maven repository in the local directory repo.
$ mvn deploy:deploy-file -Durl=file:repo -Dfile=lib/lucene-core-4.5.1.jar -DgroupId=org.ifcx.lucene -DartifactId=lucene-core -Dpackaging=jar -Dversion=4.5.1
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-deploy-plugin:2.7:deploy-file (default-cli) @ standalone-pom ---
Uploading: file:repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.jar
Uploaded: file:repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.jar (2244 KB at 16025.9 KB/sec)
Uploading: file:repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.pom
Uploaded: file:repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.pom (399 B at 389.6 KB/sec)
Downloading: file:repo/org/ifcx/lucene/lucene-core/maven-metadata.xml
Uploading: file:repo/org/ifcx/lucene/lucene-core/maven-metadata.xml
Uploaded: file:repo/org/ifcx/lucene/lucene-core/maven-metadata.xml (306 B at 298.8 KB/sec)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.820s
[INFO] Finished at: Sat Nov 23 00:42:08 PST 2013
[INFO] Final Memory: 3M/81M
[INFO] ------------------------------------------------------------------------

$ mvn deploy:deploy-file -Durl=file:repo -Dfile=lib/lucene-analyzers-common-4.5.1.jar -DgroupId=org.ifcx.lucene -DartifactId=lucene-analyzers-common -Dpackaging=jar -Dversion=4.5.1
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-deploy-plugin:2.7:deploy-file (default-cli) @ standalone-pom ---
Uploading: file:repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.jar
Uploaded: file:repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.jar (1549 KB at 12196.3 KB/sec)
Uploading: file:repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.pom
Uploaded: file:repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.pom (411 B at 401.4 KB/sec)
Downloading: file:repo/org/ifcx/lucene/lucene-analyzers-common/maven-metadata.xml
Uploading: file:repo/org/ifcx/lucene/lucene-analyzers-common/maven-metadata.xml
Uploaded: file:repo/org/ifcx/lucene/lucene-analyzers-common/maven-metadata.xml (318 B at 155.3 KB/sec)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.745s
[INFO] Finished at: Sat Nov 23 00:42:28 PST 2013
[INFO] Final Memory: 3M/81M
[INFO] ------------------------------------------------------------------------

A New Repository is Born

$ find *
lib
lib/lucene-analyzers-common-4.5.1.jar
lib/lucene-core-4.5.1.jar
repo
repo/org
repo/org/ifcx
repo/org/ifcx/lucene
repo/org/ifcx/lucene/lucene-analyzers-common
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.jar
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.jar.md5
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.jar.sha1
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.pom
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.pom.md5
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.pom.sha1
repo/org/ifcx/lucene/lucene-analyzers-common/maven-metadata.xml
repo/org/ifcx/lucene/lucene-analyzers-common/maven-metadata.xml.md5
repo/org/ifcx/lucene/lucene-analyzers-common/maven-metadata.xml.sha1
repo/org/ifcx/lucene/lucene-core
repo/org/ifcx/lucene/lucene-core/4.5.1
repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.jar
repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.jar.md5
repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.jar.sha1
repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.pom
repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.pom.md5
repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.pom.sha1
repo/org/ifcx/lucene/lucene-core/maven-metadata.xml
repo/org/ifcx/lucene/lucene-core/maven-metadata.xml.md5
repo/org/ifcx/lucene/lucene-core/maven-metadata.xml.sha1
whats_new.groovy

We're Ready to Rock

$ ./whats_new.groovy index_dir ~/Downloads/ucsd_2.0_content/editor_source/command.text 
SEGMENT PROCEDURE EDITCORE;

(* Core procedures.  Execute these commands until either a set environment
   comes along or a quit command. *)
...
There are 263 new lines of 263 total.

Note that the script was run with the working directory being the one containing the Groovy script. That is essential because the root URL we used in the @GrabResolver annotation is relative to the working directory. That isn't quite as inconvenient as it might seem since the URL will only be used if the artifacts are not in the Groovy Grapes cache (~/.groovy/grapes), which typically will only be the first time the script is run by a user. The caching though does mean that if the JAR changes then the version number needs to change and the script must again be run from the working directory.

Update

As a result of addressing this issue I've created an improvement for Grape that makes it work really nicely with these Maven repositories collocated with the Groovy scripts that use them. Follow GROOVY-6451 to see when this feature is ready to use. I'll also post here as well of course.

Go Forth in Groovy Style

That's all there is to it. The working directory can be delivered using your favorite method such as tar and ssh or what-have-you. The lib directory doesn't need to be included of course and is only shown here for completeness - the path given to mvn deploy:deploy-file could easily have been elsewhere (including your ~/.m2 or ~/.groovy/grapes cache).