DataDistributionModule

Overview

The DataDistributionModule is an edge module that is used by the Nexus platform when an Asset Hierarchy is deployed. The module functions as edge-side measurement hub, with the following responsibilities:

• Distribution of measurements from other data-collector modules according to the tag configuration and routing settings.

• Hot-path streaming of module logs or measurements, directly to cloud, on demand.

• Caching and exposing historical measurements for all tags, within a limited time window.

• Generation of calculated measurements from formula equations defined in the Asset Hierarchy.

• Historize measurements in supported local historian databases, either running locally as a module on edge, or on-premises.

The figure below shows an overview of the functionality in the DataDistributionModule.

The figure shows the data flow and internal services in the DataDistributionModule.

An input queue receives measurements from other modules on the edge device. The data flow is:

1. Every measurement is always stored locally in the circular measurement buffer.

2. If the measurement is used as an input in a formula equation, the Formula Evaluator service is invoked with the measurement. This potentially generates a new calculated measurement, that is handled separately by the data distribution. Circular references are detected and avoided.

3. The tag configuration is retrieved from the cache, and the measurement is routed according to its configuration.

Dependencies

The DataDistributionModule is dependent on the LocalBlobStorage module that provides storage for both tag configuration and routing information, as well as temporary storage for cold path measurement batches.

Measurement buffer

The DataDistributionModule contains an in-memory cyclic buffer that keeps most recent measurements available at the edge, for all tags. The buffer uses a retention policy to delete the oldest measurements. It accepts late and out-of-order timestamps, preserving event time so downstream systems can reconstruct the correct sequence.

The buffer is used to ensure, that the correct measurements are routed, even if measurements arrive out-of-order. It also ensures that limited tag history is available at the edge, for visualization, analytics or other purposes, without calling cloud services.

Other modules can use the DataDistributionModule to retrieve most recent or historical measurements by calling the direct command "GetSnapshot".

Tag Configuration Cache

Due to the potentially very large amount of lookups required to get the tag configuration belonging to each measurement that is distributed, a tag configuration cache is maintained in the DataDistributionModule. This cache is updated on startup, and every time the tag configuration changes due to a deployment.