top of page

Joule Release 1.0.4

Updated: Jul 31, 2023

Joule is a Low Code Platform for use case development. The platform brings simplicity to developing use cases by providing an expressive language to define processing pipelines using prebuilt and custom processors and data connectors.


Out-of-the-box Joule provides standard data connector implementations, and useful processors that enables you to start building and running use cases quickly.

This early release brings a number of new features, bug fixes, optimisations and general usability enhancements.




Key Features

Feature Engineering

Joule provides a feature engineering processor that enables users to define how features are to be created ready for predictive analytics use cases.

The processor generates for each declared feature field an engineered value. Two methods are supported; raw and compute values using custom expression and plugins. On completion a feature map is generated will all the required features and placed in the StreamEvent ready for the next processor in the pipeline.


To get you started OOTB plugins are provided for the following functional categories:

  • Scripting

  • Scaling

  • Transform

Example


feature engineering:
    name:retailProfilingFeatures
    versioned:true
    features:
        as values:
            -location_code
            -store_id

        compute:
            spend_ratio:
                scripting:
                  macro:
                    expression:1-spend/avg_spend
                    language:js
                    variables:
                      avg_spend:133.78
            age:
              function:
                age binning:
                   source field:date_of_birth
            day:
              function:
                day-of-week transform:
                   source field:date
                

See documentation for further details.


Machine Learning & Auditing

Joule provides a PMML predictor processor to perform streaming predictions / scoring. The implementation leverages the JPMML open source library developed by Villu Ruusmann.


Example

pmml predictor:
    name:irisScorer
    model filename:/hom/joule/models/pmml/iris_rf.pmml
    response field:flowerPrediction
    audit configuration:
        target schema:ml_audit
        queue capacity:5000
        flush frequency:5

Auditing

An optional configuration provide the ability to audit predications to enable model retraining, feature and prediction drift management, model observability, and any local business governance requirements.


The configuration will dynamically create an in-memory database table, using the process name as the target table, and rest endpoints to enable direct access and export functions.


With these new features Joule can now provide stream based advanced analytics using PMML models within Docker containers. An example is illustrated below.




See documentation for further details.


SQL Query API

Joule embeds DuckDB, an in-memory database, in to the runtime process. The solution is ideal for supporting custom processor logic using various methods such as:

  • Hosting and accessing custom reference data

  • Scratchpad for stateful processing

  • Ad-hoc custom complex queries

  • Capture and exporting streaming events

See documentation for further details.


Websocket Support

Joule now supports publishing events on to a Websocket publisher. Events are serialised as Json.


Example


websocketPublisher:
    pathOverride:/joule/websocket/stream

See documentation for further details.


Project Templates

To kick start transport and processor custom development a projects template project is provided. The project can be found on this link.


Documentation

As an ongoing effort to provide good quality documentation for the platform this release has reorganised, added new sections, more details and much more. The online documentation can be found here.


Getting started

To get started, download the following resources to prepare your environment and work through the provided documentation. Feel free to reach out with questions.

  • Download the examples code from GitLab

  • Pull the latest docker image from DockerHub

  • Work through the README file


We’re Here to Help

Feedback is most welcome including thoughts on how to improve and extend Joule and ideas on exciting use cases.


You’re in this with the entire FractalWorks community, who’s openly sharing ideas, and best practices and helping each other on our Community Forum. Feel free to join us there! And if you have any further questions on how to become a partner or customer of FractalWorks, do not hesitate to engage with us, we will be happy to talk about your needs.

27 views0 comments

Recent Posts

See All

コメント


bottom of page