Creating and Editing Workflows

Here is a starting point to develop your own style of workflow development. There is no single best way to develop Flux workflows, but we have had success with this approach,

Synch Flux Engines to a Network Time Server

Do not assume that multiple servers in a Flux cluster will have synchronized system clocks. There are many reasons why system clocks can deviate and the only safe means to assure synchronization is to have all the servers update their times from a network time server.

Error Handlers

When an error occurs in a workflow that triggers an error handler, the error handler's actions are inserted into the workflow while the error handler runs.

To help operators understand which actions belong to the error handler, and which actions belong to the main workflow, we recommend adding some descriptive text to the name of each action in the error handler to help it stand out — for example, prepending all of the error handler's action names with "ERROR HANDLER".

Using this technique when you design your error handlers will help operators immediately and visually identify which components are part of the workflow proper and which components have been inserted by the error handler to remediate an error.

Namespace Best Practice

Namespaces are the organization structure in Flux that allow you to name your workflows and group them together in logical units.

Namespaces are used in several places through Flux:

Because of the wide range of applications for namespaces, we've developed a few simple best practices to keep in mind when building your namespace structures.

Workflows should share a common namespace folder in any of the following scenarios:

File Transfer Workflows and Error Handling

A common pattern encountered by workflow designers involves a workflow that fires at a given time every weekday, then polls a remote server for files which are fetched and processed. Often such workflows need to support selecting files for processing based on a naming convention – such as a file description followed by a valid date in the filename.

A best practice in Flux workflow design is accommodating for exceptional situations during processing. Often customers utilize wild card processing to select and process sets of files, not realizing that if a file in the set fails to correctly process, all following files in the set will fail to process.

The above workflow starts with a timer trigger to fire at specified intervals. A File Exists Trigger fires collecting the set of filenames available to fetch from the remote server. Then, using a For Each Collection Element Action that submits each filename to a Regular Expression Action to compare a substring of the file to match a valid date (in yyyy-MM-dd format). If a filename matches the regular expression, the filename is submitted to a File Move Action where the file itself is moved to a local folder to be decrypted. If a filename does not match the regex expression, then the flow goes back to the For Each Collection Action to pick up the next filename in line. 

After fetching the remote files, the workflow uses another file trigger to pick up the filenames of local files to be decrypted. This pattern of using a For Each Collection Element Action provides per-file processing, meaning (in this case) if the decryption action fails, the Flux Operations Console has access to the filename of the file that failed (rather than a list of files and no knowledge of which file failed to decrypt). This same pattern is used again to reencrypt the files. 

If anything should fail in the workflow, mail actions send an email to a set of specified email addresses to alert the customer's team that something failed. After the email has been sent (containing the filenames which failed) the normal processing of the remaining files continues.

Directories, server names, and numerous other elements of this workflow can be externalized to runtime configuration files allowing the same workflow to be used as a template for many purposes, providing consistency of deployment.

Documenting a Flux Environment

The following information should be kept in an editable document, reviewed quarterly, and made available to Flux Support Personnel when requested for assisting in Flux-related issues. While this list is not exhaustive, it should serve as a starting point for keeping your Flux environment managed.

Version Control of Workflows

Flux does not provide a built-in VCS mechanism. Workflows can be downloaded to XML files, from the Repository and the Designer, so if version control is required, Flux recommends the following:

  1. During workflow design, save workflows to the Repository as normal.
  2. Once the workflow is in a state where it should be committed to the VCS, use the "Download" option from the Repository to download a local copy of the file.
  3. If you need to test two different versions of the same workflow, or would like to have a way to tell the different stages of a workflow apart, append a suffix of your choice to the workflow name (e.g., -A, -1, etc.). 
  4. Place the file into the VCS and commit / update as normal through the VCS tool's interface.
  5. Once you have your final workflow design, feel free to clean up old versions of the workflow from the workflow repository. This can easily be done from the Flux Monitor, which allows for bulk operations: just select all the workflows you'd like to remove, and click on the 'Remove' button found at the top. 

Many VCS applications will offer XML comparison / change tracking tools out of the box, while others, like Git, can be extended through custom merge drivers, allowing you to track specific changes within a workflow using standard VCS tools.

Version Control of Flux Environment

In addition to Flux workflows, ensure the following files are under version control so that prior releases can be restored, and differences can be detected and reviewed.

Environment Control

Keep your Flux environment clean to make problem determination simpler and more straightforward.

Deployment

Coordinating activities on a Flux deployment is essential to provide for effective rollout through different environments such as Dev to QA or QA to production.

Maintenance

Troubleshooting

Where to find the Flux logs

When using the default Flux installation:

The engine logs can be found in the root of your Flux installation folder (usually located in 'C:\flux-8-0-x').

The operations console logs are also located in the root of your Flux installation folder

Where to find system log files

If you're running Linux, UNIX, Solaris, etc. here's a list of system log files and how to find them and view them:

http://www.cyberciti.biz/faq/linux-log-files-location-and-how-do-i-view-logs-files/

If you're running on Windows, you can refer to the page below to know where to find the system logs and view them using the Event Viewer:

https://support.microsoft.com/en-us/kb/308427

For further troubleshooting, head on to our Troubleshooting wiki.

Java Coding

Performance Considerations