LogBus

A "Swiss Army" knife for handling events with a focus on processing system & application logs. See the docs for more information.

Hacking

See just help for list of helpful tasks.

Unit Testing

just unit-test will test plugins. Plugins should have their spec files colocated beside them. Plugins should strive to be simple functions, but here are some ways they are not:

• If a plugin needs to manage state, then that can persist in its closure and managed however the plugin sees fit. For example, the write-opensearch output plugin batches events into an array until it is ready to perform a bulk insert.

• Some plugins may require special start & stop handling.

End-to-End Testing

just e2e-test will test more real-world like scenarios and exercise the main engine bits, but limited to simpler inputs & outputs (files).

Known Issues, Limitations, Warts

• There should be an example demonstrating how to "commit" events. For example, when consuming from a kafka topic, should only commit offsets once the final pipeline stage has confirmed that the event was received successfully. Not yet sure if there needs to be 1st class support for such a mechanism or if can achieve by wiring up the latter stage's output to the earlier stage's input.

• Would be nice if plugins could communicate the type (and maybe shape as well) of the events they process. Then, a user could be warned if a pipeline configuration contains stages wired up erroneously.

• Some plugins require pipeline-defined functions. For simple functions, inlined in yaml is not cumbersome. For more complicated functions, would be nice to define those in a proper package for a better dev & test story.