Examples of assyst Message Bus Request Mappers

The underlying technology is ActiveMQ (Classic), so the documentation for that may answer some of your questions. 

The fields which are required are either Queue or Topic (depending on how you want delivery to work), and Body. 

The main limitation is that only text/string messages can be consumed by ETM channels. So, you could have Channel A submit a Message with a CSV format data which was then consumed by Channel B (using the CSV message format). However you could not have Channel A submit messages using XLSX/PNG etc. data for consumption by another channel. 

One scenario where using the message bus would be useful would be to support asynchronous processing on a Remote Upload channel whilst using a Response Mapper. The Remote Upload channel could submit the received Payload to the message bus, then immediately send a custom response to the caller (via a response mapper). Another channel could process the queued message in the background. 

Do you have more specific questions or uses cases?

Thank you very much, Paul!

I had not come across that article, and the specifics of Queues implemening a sort of “loadbalancer” were new to me.

However, the main limitation I’m experiencing is the lack of a direct example to work from, such as a example datamapper to add a message to the message bus.

I’ve spent a lot of time trying to get the syntax correct when adding data to the Body, Message Property, Priority, and Time To Live, and this is where the main issue arises as I keep getting different errors and not figuring out the syntax.

If there was an example of a Message Bus Request Mapper on the wiki, it would be much easier to know what format ETM is expecting as input when data mapping, or even better, a set of example channels that display some “best practices” for both adding and reading data from the message bus.

The specific use case I’m trying to get to work is the one you proposed in this thread;

but I’ve not had the opertunity to set aside time to try getting that to work. However upon trying to do so a few weeks ago led me to realize i did not know enough about the use of destinations and the message bus.

How to use the message bus seemed out of topic to bring up in that thread, and that’s why this is a sepererate question😅

The simplest mapper would have a Body with expression “Hello, World!” and a Queue expression “MyQueue”. When this mapper processes an Import Record it will post a message to the specified queue, which will remain in the queue until something reads it, or the ActiveMQ process is restarted.

A Channel to consume these messages would specify:

• Source: Assyst Destination
• Message format:  Free Text
• Queue: MyQueue

Such a channel would create a single record import for each invocation of the mappper.

From that simple example it should be possible to incrementally add complexity. 

Thank you very much for the response.
Due to a change in team priorities, I’ve been unable to work on the problem requiring this until now.
However, there might be some miscommunication or something I’m not quite grasping here.

The issue I have is that for Message Bus Response mappers, there is only a limited set of fields to choose from, and I’m not sure how to apply what you described to this.

To help clarify what I’m trying to do:

I’m trying to get ETM to post some details after a channel has run, such that these details can be picked up by a different channel for further processing. To illustrate the general idea:

The plan is to consolidate and reuse the Get latest status logic as this has to be called asynchronous from the reservation, and is similar for both start inputs. 

However, I’ve not been able to get this working correctly, given that the options in the Message Bus Response data mappers do not include anything about queue or topic, only response headers and body are available, as far as I can tell.

So what I could really use is an example of a Message Bus Response data mapper that display how to set the different properties in a Message Bus Response using the available fields, as this is not provided on the wiki. Given such an example I could then easily modify with the correct topic or queue name as needed.

Message Bus Response mappers are used for Message Bus/Destination Source channels. They send a reply back to a special queue that is created as part of the initial message submission. 

For example if an assyst Action Destination  with target “ETM_NewActions” is configured to wait for a reply then when an action is logged a new message is sent to the ETM_NewActions queue. That message includes a JMSReplyTo property with a value something like ETM_NewActionsReplies. asyst then waits for a new message to be sent to the ETM_NewActionsReplies queue (with a correlation id linking the original & reply messages).

See the wiki  for details of if, when & what message is sent to the reply queue. 

Messages created by Message Bus Request mappers don’t support replies - they are asynchronous. So they can trigger another channel - but don’t wait for that channel to respond. 

Waiting for a response would be an enhancement, as would explicit support for channel chaining.

Note that in principle you could use an HTTP Request Mapper & a Remote Upload channel to support synchronous inter-channel communication. However I would caution against this - there are risks of oddities like worker thread starvation and startup race conditions that could lead to ‘interesting’ behavior.

Thank you very much for the clarification ​@Paul McCulloch , and I also want to apologize for the confusion in my previous post.

When returning to this problem after a 10 month “break”, I had forgotten parts of the original context. Combined with some English↔Norwegian “false friend” terminology quirks on my end, I ended up mixing up the mapper types when revisiting the problem. This naturally led to more confusion in how I explained it.

After correcting that misunderstanding and switching to an assyst Message Bus Request Mapper, what I’m trying to achieve might make more sense. I’ve also added some context to help clarify what I’m working on in general terms: 
 

Returning to the original issue

With the mapper‑type confusion out of the way, some of my underlying challenge remains:
there are currently no examples in the documentation that show how to populate or structure the fields available in an assyst Message Bus Request Mapper (ref. ref. Message Bus Request mappers on the wiki).

To help anyone else who stumbles across this thread, here’s what I’ve managed to resolve since the original post:

1. The “ETM_” prefix issue

One mistake on my end was accidentally adding an extra ETM_ prefix when specifying the queue or topic in the assyst Message Bus Request Mapper, causing any test to not show up in the test channels.

This only became clear after enabling the ActiveMQ admin console and seeing queues/topics like: ETM_ETM_MyQueue.Removing the redundant prefix fixed this part.
This was an issue caused by me as I had not noticed the difference in text between the Message Bus Request Mappers and assyst Message Bus Request Mappers articles on the wiki. 

2. Generating the Body field during data mapping

I initially also struggled with generating the Body content.
I eventually got it working by constructing the body as XML, for example:

var xml =
    '<?xml version="1.0" encoding="UTF-8"?>' +
    '<testMessage>' +
        '<test1>Test1 - Success</test1>' +
        '<test2>Test2 - Success</test2>' +
    '</testMessage>';

xml;

While attempting the same using JSON.stringify did not work:

var MessageBody = {};

MessageBody.test1 = "Test1 - Success1";
MessageBody.test2 = "Test2: Success1";

JSON.stringify(MessageBody);

This JSON approach simply caused an error during import com.thoughtworks.xstream.io.StreamException:

I haven’t yet been able to determine whether XML is required or whether this is just a quirk of the test mapper or my setup.

3. Persistence field

I still don’t understand how the Persistence field is intended to be populated in an assyst Message Bus Request Mapper.
The mapper expects an object here, but without an example it is difficult to know what structure is valid.

Summary of my remaining questions

1. Is XML the required format for the message body?
   Or should JSON also work when constructing the Body in a Message Bus Request Mapper?

2. How exactly is the Persistence field meant to be used?
   What object structure is ETM expecting here?

3. Could the wiki be updated with one or two concrete examples?
   Even a minimal working example with the all the fields in use would make these mappers far more intuitive to reason from and prevent others from potentially running into the same issues.

Again thank you very much for taking the time to help. 

Hi Richard.

I don’t really understand the flow chart. Are you saying that the same (2nd) channel is being triggered both by an assyst Destination (when an action is logged) AND by the 1st channel?

If that’s the case then you’ll need to construct an XML messages that looks like an assyst Destination message. The ‘assyst Destination’ message format is XML (all “assyst mapper” driven interaction with assyst uses XML behind the scenes). The simplest way to see such a messages is to just stop your channel 2 and log an action - you can then view the queued message in the AMQ console.

Persistence is a boolean. It’s optional and just determines if the message is in-memory only or not. The same option is presented as a checkbox in the AMQ web consoles’ send message. 

Yes, that is exactly correct.

The intention is for the second channel to support two different entry points:

1. The “normal” automated path, triggered by the output of the first channel.
2. A manual fallback path, triggered from assyst via an Action Destination (using an action template a case manager can run if something has gone out of sync).

The manual path is only meant as a safety net if ETM, the RPA process, or the external endpoint have been unavailable due to maintenance. In those cases, a user may need to temporarily control the event outside the normal sync process handled by channel 3 (not shown in the diagram).

The primary flow is the channel‑to‑channel integration via the destination. However, this “dual‑input channel” design is something we expect to reuse in other integrations as well, so if there are approaches that make it simpler or less fragile, I’m definitely interested.

We did consider letting ETM trigger the second destination via actions, but this is not practical for several reasons. Frequent automated actions risk interfering with case managers who may be editing the same event—leading to rejected saves if ETM updates the event’s version in the meantime. Our current ETM version also lacks retry logic, so destinations will likely serve as the retry mechanism for this project until we upgrade.

Regarding the assyst Message Body

Is the requirement to write XML in the Message Bus Request Mapper a limitation of ETM 1.8, or does this apply to more recent releases as well?

Elsewhere in ETM we normally work with JSON, but somewhere in the outgoing import‑stage pipeline converts the data to XML. I was therefore hoping that the same encoding layer was or could be used when generating the Message Bus Body? 
Manually converting JSON to XML in ECMAScript 5.1 is tedious, and this will become increasingly cumbersome if we implement more “action‑processor‑replacement” flows that rely on destinations passing assignment data to a different channel for processing. 

Makes sense.

There is no intrinsic requirement to use XML with the message bus. The limitation here is that a channel can only have 1 parser/format and that you want the channel to consume assyst Destination messages - which are always XML.  You could change the 2nd channel to be plain text parsed and send a JSON message from Channel 1  - but you wouldn’t be able to consume the manual fallback XML messages in that case. 

An alternative Design would be a separate channel (Destination Source/Plain Text Format) that read from a some queue (lets call that ‘Check Status’) 

Channel 1 (Destination Source/Destination Format) could send to ‘Check Status’, but use JSON.

Channel 2 (Destination Source/Destination Format) could send to ‘Check Status’ too. JSON.stringify(inbound) as the message body would convert the internal representation to JSON, suitable for the message bus)

Velocity (VTL) is available in recent versions and can make XML generation a bit easier. 

Internally all data is just handled as nested key/value pairs. We often present that to the user as JSON  -- but that’s a bit misleading. The XML conversion is done at a very low level and isn't user accessible.