More dynamic file destinations for file sink #17

Open
opened 2026-08-18 00:33:53 +00:00 by erik · 0 comments
Owner

I have a pipeline where it would be nice if the destination file could switch on event data. Supporting truly dynamic files is probably too gnarly, but supporting fewer than 10 explicitly enumerated file paths should be simple enough while handling a vast majority of use cases. The next challenge will be how to do the actual switching:

  1. A pipeline config that says which field's value to use? It must match one of the configured paths?
  2. A JsFunc that takes the event as input and returns one of the configured paths?
I have a pipeline where it would be nice if the destination file could switch on event data. Supporting truly dynamic files is probably too gnarly, but supporting fewer than 10 explicitly enumerated file paths should be simple enough while handling a vast majority of use cases. The next challenge will be how to do the actual switching: 1. A pipeline config that says which field's value to use? It must match one of the configured paths? 2. A JsFunc that takes the event as input and returns one of the configured paths?
erik self-assigned this 2026-08-18 00:33:53 +00:00
Sign in to join this conversation.
No description provided.