Grid computing with Groovy

Alex Tkachman posted a question about Groovy style for checkpointed calculations.

Here's my first try:

// Proof of concept for Groovy checkpointed calculation with closures.
// @author Jim White <jim@pagemsiths.com>
// http://www.ifcx.org/

Session session = new Session()

// def initState = new Expando(data:'datalocation')
def initState = [data:'datalocation']

def endState = eachWithCheckpoints(session, 'state1', initState, [
      { stepCount = longInitialConditionsCalculation(data) }
    , { accumulator = 0
        iterateWithCheckpoints(session, 'state2', it.state
           , 0..stepCount
           , { accumulator += 2 * it.step })
      }
    , { result = "We did $stepCount iterations and the answer is $accumulator!" }
])

def longInitialConditionsCalculation(d) { 10 }

println endState.result

class Session {
    Object loadCheckpoint(String k) { null }
    void saveCheckpoint(String k, Serializable s) { }
}

def eachWithCheckpoints(Session session, String key, def initialState, List<Closure> closures) {
    def control = session.loadCheckpoint(key)

    if (control.is(null)) {
        control = [step:0, state:initialState]
    }

    while (control.step < closures.size()) {
        Closure clos = closures[control.step]

        clos.delegate = control.state
        clos.resolveStrategy = Closure.DELEGATE_FIRST

        clos.call(control)

        control.step += 1

        session.saveCheckpoint(key, control)
    }

    control.state
}

def iterateWithCheckpoints(Session session, String key, def initialState, Range range, Closure clos) {
    def control = session.loadCheckpoint(key)

    if (control.is(null)) {
        // Leave reversed range case as an exercise for the reader...
        assert !range.isReverse()

        control = [step:range.from, state:initialState]
    }

    clos.delegate = control.state
    clos.resolveStrategy = Closure.DELEGATE_FIRST

    while (control.step <= range.to) {
        clos.call(control)

        control.step += 1

        session.saveCheckpoint(key, control)
    }

    control.state
}
==> 
We did 10 iterations and the answer is 110!

The idea is either you're doing a sequence of different steps or iterating the same step some number of times. Notice that the state can be any serializable thing. Map or Expando is handy, but some bean or other class would be fine.

Obviously there are further refinements possible such as reversed and stepped ranges, and also a more functional style is possible. Either a builder or controller class would streamline things a bit by hiding some of the details of the eachWithCheckpoints/iterateWithCheckpoints calls.

If Groovy had serializable iterators for the control step, that would make iterateWithCheckpoints nicer and let it change from taking just a simple integer range to a list.

Naturally where there this is headed is cloud-powered click-and-it-goes Wings via your web browser using OOHTML.