Gearpump is a real-time big data streaming engine. It is inspired by recent advances in the Akka framework and a desire to improve on existing streaming frameworks. Gearpump is event/message based and featured as low latency handling, high performance, exactly once semantics, dynamic topology update, Apache Storm compatibility, etc.

The name Gearpump is a reference to the engineering term "gear pump," which is a super simple pump that consists of only two gears, but is very powerful at streaming water.

Gearpump Technical Highlights

Gearpump's feature set includes:

• Extremely high performance
• Low latency
• Configurable message delivery guarantee (at least once, exactly once).
• Highly extensible
• Dynamic DAG
• Storm compatibility
• Samoa compatibility
• Both high level and low level API

Gearpump Performance

Per initial benchmarks we are able to process 18 million messages/second (100 bytes per message) with a 8ms latency on a 4-node cluster.