Presentation: Patterns of Streaming Applications

Stream processing engines are becoming pivotal in analyzing data. They have evolved beyond a data transport and simple processing machinery, to one that's capable of complex processing. The necessary features and building blocks of these engines are well known. And most capable engines have a familiar Dataflow based programming model.

As with any new paradigm, building streaming applications requires a different mindset and approach. Hence there is a need for identifying and describing patterns and anti-patterns for building these applications. Currently this mindshare is scarce.

Drawn from my experience working with several engineers within and outside of Netflix, this talk will present the following:

• A blueprint for streaming data architectures and a review of desirable features of a streaming engine
• Streaming Application patterns and anti-patterns
• Use cases and concrete examples using Flink

Attendees will come away with patterns that can be applied to any capable stream processing framework such as Apache Flink.