Streams Example: Simple Data Aggregation

This is a quick example of how you can use UE Streams to aggregate data from multiple sources into a single datastore (e.g., database, data lake, etc.).

The general architecture approach looked like this:

You can see the full video here!

Looking for the code?

No problem! Here are the three services I wrote to make it happen. The UI dashboard is not included but we may publish that in the future:

• Stock Tickers - Query stocks on a cadence and publish to a stream
• Twitter Trends - Query trending tweets on a cadence and publish to a stream
• Data Lake Listener - Subscribe to a stream and write all data to mongo as it comes in

Also, here's a generalized version of the stock ticker and twitter trend publish services you can use as a template.

In the future, we will create more streamlined examples by abstracting some of the logic you'll see into new packages. But for now, feel free to explore with UE Streams!