Write your first Gearpump Application

We'll use the classical wordcount example to illustrate how to write Gearpump applications.

/** WordCount with High level DSL */
object WordCount extends AkkaApp with ArgumentsParser {

  override val options: Array[(String, CLIOption[Any])] = Array.empty

  override def main(akkaConf: Config, args: Array[String]): Unit = {
    val context = ClientContext(akkaConf)
    val app = StreamApp("dsl", context)
    val data = "This is a good start, bingo!! bingo!!"

    //count for each word and output to log
    app.source(data.lines.toList, 1, "source").
      // word => (word, count)
      flatMap(line => line.split("[\\s]+")).map((_, 1)).
      // (word, count1), (word, count2) => (word, count1 + count2)
      groupByKey().sum.log

    context.submit(app).waitUntilFinish()
    context.close()
  }
}

The example is written in our Stream DSL, which provides you with convenient combinators (e.g. flatMap, groupByKey) to easily write up transformations.

IDE Setup (Optional)

You can get your preferred IDE ready for Gearpump by following this guide.

Submit application

Finally, you need to package everything into a uber jar with proper dependencies and submit it to a Gearpump cluster. Please check out the application submission tool.