A set of reference use cases for Joule is here! If you haven’t already, check out the Joule Platform overview page to learn more about the platform, features and capabilities. This post provides a quick guide on what to expect from the packaged use case examples.
To get started, download the following resources to prepare your environment and work through the provided documentation.
Get hold of the latest Joule docker image from docker hub
Download the examples code from GitLab
Work through the README file
Inspiration
Inspiration for the use cases came from experience across various industries; telecommunications, investment banking, fintech, healthcare etc,. The examples were developed to reflect the general nature of enterprise solution requirements when applying event driven processing and therefore aims to bootstrap applications.
The examples aim to demonstrate key features such as out-of-the-box external integrations, pre-build processors, custom component development using Joule SDK, and the agnostic deployment model. Joule’s ambition is to provide an extensible use case delivery platform using a combination of mature, modern technologies and self describing use cases.
Use cases
Six use cases have been developed starting with a basic filtering process and ending with a simulated real-time marketing campaign geospatial triggers. To reflect an agile development and delivery process each use case delivers a distinct feature while building upon previous capabilities. We believe building use cases incrementally supports business impact validation, iterative pilots, solution quality and reusability.
The final use case, in the form of geospatial marketing, demonstrates a representative set of features often required within an enterprise solution; Data privacy and encryption, enrichment using live data, basic and advanced analytics, and real-time alerting (next best actions).
1. Opt-out Filtering
Within sensitive processing environments users may opt-out of personal data processing, for example real-time marketing or cross site tracking. To address this requirement event filtering is applied at the first stage of processing, known as opt-out.
Joule has an OOTB filtering processor which applies a custom Javascript expression to perform the filtering. This example simply filters a single IMSI from the processing pipeline.
2. Reference data enrichment
Often real-time streaming processing events require additional data to support further processing functions. Joule provides a solution where events are enriched in place using locally cached reference data.
Reference data is co-located, in-memory, within the same process as Joule. This improves processing throughput by removing the need to retrieve data from traditional data stores which are typically out of process.
A key feature of the enricher processor is the ability to load reference data on startup from connected distributed Geode data clusters thus reducing the I/O overhead on cache misses, this feature is generally known as Get Initial Image.
3. PII Anonymisation
This example performs field level anonymisation on the customer IMSI using RSA encryption. Encryption keys are dynamically generated from the provided salt and stored in a user defined directory.
4. Basic analytics
This example performs a simple arithmetic function within a select statement before the event is passed on to be published.
5. Geospatial Triggers
This use case demonstrates geospatial location alerting using mobile location data and user defined geofences. When a mobile enters a geofence zone a tracking trigger is generated until the user exits the geofence. Three tracking states are generated per entity; Entered, dwelling and exited. Geofences are manually defined within the configuration file.
6. Geospatial marketing
This example simulates geospatial real-time marketing through the use of geofence triggers and custom messages. Geofences are automatically added to the processor through the use of auto reference data binding.
Deployment
Joule is deployed using local resources via Docker images and a single platform install, a docker-compose file is provided.
The architecture uses three external components to simulate a real system; InfluxDB for time series data, distributed cache for in-memory reference data and Postgres for the actual relational reference data.
Reference Data
Providing live reference data without having to restart consuming processes is often a key requirement in enterprise applications. Read heavy applications can cause relational database issues which in turn can have an adverse effect on application responsiveness.
To mitigate this issue the Joule platform has developed a custom in-memory reference data transport. Processors are able to leverage external data sources through the use of the supported enterprise data caching platform, Apache Geode. The transport builds upon the in-memory data caching feature by providing integration to relational databases to enable initial data set loading, ad-hoc data loading, generic reference data structure and data write-back.
Used reference data
Points of interest (e.g. London transport stations)
Device information (e.g. manufacturer, type, etc,.)
Data bundle configurations
We’re Here to Help
Feedback is most welcome including thoughts on how to improve and extend Joule and new practical examples. For the next instalment an IoT solution with Joule deployed to a cloud platform and Raspberry PI, using MQTT, local telemetry and analytics.
You’re in this with the entire FractalWorks community, who’s openly sharing ideas, best-practices and helping each other on our Community Forum. Feel free to join us there! And if you have any further questions on how to become a partner or customer of FractalWorks, do not hesitate to engage with us, we will be happy to take your call.