More on this book
Kindle Notes & Highlights
by
Hohpe Gregor
we can use a Process Manager (312), which gives us more flexibility but requires the message to return to a central component after each function.
acquisitions or business partnerships often result in multiple systems performing the same business function. Also, many businesses that act as aggregators or resellers typically interface with multiple systems that perform the same functions (e.g., check inventory, place order).
in the world of message-based integration, passing messages through a chain of systems could mean significant overhead. Also, this approach would require collaboration of the individual systems, which may not be feasible if some systems are maintained by external business partners and are therefore not under our control.
Content-Based Router is a frequently used form of the more generic Message Router (78). It uses predictive routing—that is, it incorporates knowledge of the capabilities of all other systems. This makes for efficient routing because each outgoing message is sent directly to the correct system. The downside is that the Content-Based Router has to have knowledge of all possible recipients and their capabilities. As recipients are added, removed, or changed, the Content-Based Router has to be changed every time.
Use a special kind of Message Router, a Message Filter, to eliminate undesired messages from a channel based on a set of criteria.
The Message Filter is a Message Router (78) with a single output channel. If the content of an incoming message matches the criteria specified by the Message Filter, the message is routed to the output channel. If the message content does not match the criteria, the message is discarded.
define a single Publish-Subscribe Channel (106) that each customer is free to listen on. The customer can then use a Message Filter to eliminate messages based on criteria of his or her choosing, such as the type of item or the magnitude of the price change.
The Message Filter can be portrayed as a special case of a Content-Based Router (230) that routes the message either to the output channel or the null channel, a channel that discards any message published to it. The purpose of such a channel is similar to the /dev...
This highlight has been truncated due to consecutive passage length restrictions.
Stateless components have the advantage that they allow us to run multiple instances of the component in parallel to speed up processing.
some publish-subscribe systems allow you to define a hierarchical structure for Publish-Subscribe Channels
The Content-Based Router (230) evaluates each message’s content and routes it predictively to the appropriate receiver.
If we want to pass control to the recipients but need to use a router for reasons of network efficiency, we can employ a dynamic Recipient List (249). This type of Recipient List (249) acts as a Dynamic Router (243), allowing recipients to express their preferences via control messages to the router. The Recipient List (249) stores the recipient preferences in a database or a rule base. When a message arrives, the Recipient List (249) forwards the message to all interested recipients whose criteria match the message.
When a message arrives, the Dynamic Router evaluates all rules and routes the message to the recipient whose rules are fulfilled. This allows for efficient, predictive routing without the maintenance dependency of the Dynamic Router on each potential recipient.
A Dynamic Router is another example where message-based middleware performs similar functions to lower level IP networking.
A Dynamic Router works very similarly to the dynamic routing tables used in IP routing to route IP packets between networks. The protocol used by the recipients to configure the Dynamic Router is analogous to the IP Routing Information Protocol
A common use of the Dynamic Router is dynamic service discovery in service-oriented architectures. If a client application wants to access a service, it sends a message containing the name of the service to the Dynamic Router. The Dynamic Router maintains a service directory, a list of all services with their name and the channel they listen on. The router builds this directory based on control messages from each service provider. When a service request arrives, the Dynamic Router uses the service directory to look up the service by name, then routes the message to the correct channel. This
...more
Define a channel for each recipient. Then use a Recipient List to inspect an incoming message, determine the list of desired recipients, and forward the message to all channels associated with the recipients in the list.
The Recipient List component is responsible for sending the incoming message to each recipient specified in the recipient list. A robust implementation of the Recipient List must be able to process the incoming message but only “consume” it after all outbound messages have been successfully sent. As such, the Recipient List component has to ensure that the complete operation of receiving one message and sending multiple messages is atomic. If a Recipient List fails, it must be restartable, meaning it must be able to complete any operation that was in progress when the component failed.
Idempotent functions are those that do not change the state of the system if they are applied to themselves; that is, the state of the component is not affected if the same message is processed twice.
Idempotence is very handy because it allows us to simply resend messages when we are in doubt whether the recipient has received it. The TCP/IP protocol uses a similar mechanism to ensure reliable message delivery without unnecessary overhead
recipients can send their subscription preferences to the Recipient List via a special control channel. The Recipient List stores the preferences in a rules base and uses it to compile the recipient list for each message. This approach gives the subscribers control over the message filtering but leverages the efficiency of the Recipient List to distribute the messages. This solution combines the properties of a Dynamic Router (243) with a Recipient List to create a dynamic Recipient List
Generally, we can assume that the more recipients a message has, the more network traffic it causes. However, there are exceptions. Some publish-subscribe messaging systems are based on IP Multicast functionality and can route messages to multiple recipients with a single network transmission (requiring retransmission only for lost messages). IP Multicast takes advantage of Ethernet’s bus architecture. When an IP packet is sent across the network, all network adapters (NIC) on the same Ethernet segment receive the packet.
The DynamicRecipientList listens on two input queues, one for incoming messages (inQueue) and a control queue (controlQueue) where recipients can hand in their subscription preferences.
How can we process a message if it contains multiple elements, each of which may have to be processed in a different way?
We used the Pipes and Filters (70) architecture before to break out processing into well-defined, composable components as opposed to lumping multiple functions together, so we should be able to take advantage of this architecture here as well.
These extra elements may be required to make the resulting child message self-contained and enable stateless processing of each child message. It also allows reconciliation of associated child messages later on. For example, each order item message should contain a copy of the order number so we can properly associate the order item back to the order and all associated entities such as the customer placing the order
If we use message trees, the Splitter can easily be configured to iterate through all children under a specified node and send one message for each child node.
If message envelopes are used (see Envelope Wrapper [330]), each new message should be supplied with its own message envelope to make it compliant with the messaging infrastructure. For example, if the infrastructure requires a message to carry a timestamp in the message header, we would propagate the timestamp of the original message to each message’s header.
The inclusion of the customer ID and the order date makes the message self-contained and keeps the message consumer from having to store context across individual messages. This is important if the messages are to be processed by stateless servers.
A Splitter (259) is useful to break out a single message into a sequence of submessages that can be processed individually.
Likewise, a Recipient List (249) or a Publish-Subscribe Channel (106) is useful to forward a request message to multiple recipients in parallel in order to get multiple responses to choose from.
Use a stateful filter, an Aggregator, to collect and store individual messages until it receives a complete set of related messages. Then, the Aggregator publishes a single message distilled from the individual messages.
The Aggregator is a special filter (in a Pipes and Filters [70] architecture) that receives a stream of messages and identifies messages that are correlated. Once a complete set of messages has been received (more later on how to decide when a set is complete), the Aggregator collects information from each correlated message and publishes a single, aggregated message to the output channel for further processing.
Simple routing patterns like the Content-Based Router (230) are often stateless, which means the component processes incoming messages one by one and does not have to keep any information between messages.
When designing an Aggregator, we need to specify the following properties: 1. Correlation: Which incoming messages belong together? 2. Completeness Condition: When are we ready to publish the result message? 3. Aggregation Algorithm: How do we combine the received messages into a single result message?
allow the Aggregator to listen on a specific control channel that allows the manual purging of all active aggregates or a specific one.
Sometimes the aggregation is concluded by the arrival of an external business event. For example, in the financial industry, the end of the trading day may signal the end of an aggregation of incoming price quotes.
Select the “best” answer: This approach assumes that there is a single best answer, such as the lowest bid for an identical item. This makes it possible for the Aggregator to make the decision and only pass the “best” message on.
Condense data: An Aggregator can be used to reduce message traffic from a high-traffic source.
Collect data for later evaluation: It is not always possible for an Aggregator to make the decision of how to select the best answer.
the aggregation strategy is driven by parameters. For example, a strategy that waits for a specified amount of time can be configured with the maximum wait time. Likewise, if the strategy is to wait until an offer exceeds a specific threshold, we will most likely let the Aggregator know in advance what the desired threshold is. If these parameters are configurable at runtime, an Aggregator may feature an additional input that can receive control messages, such as these parameter settings. The control messages may also contain information such as the number of correlated messages to expect,
...more
It is very handy to have a simple listener tool that displays all messages traveling on a topic and also logs the messages into a file for later analysis.
A Message Router (78) can route messages from one channel to different channels based on message content or other criteria.
The obvious solution to the out-of-sequence problem is to keep messages in sequence in the first place. Keeping things in order is in fact easier than getting them back in order.
Use a stateful filter, a Resequencer, to collect and reorder messages so that they can be published to the output channel in a specified order.
The Resequencer can receive a stream of messages that may not arrive in order. It stores out-of-sequence messages in an internal buffer until a complete sequence is obtained, and then publishes the messages to the output channel in the proper sequence.
For the Resequencer to function, each message has to have a unique sequence number
let’s assume we have a configuration with multiple processing units, each of which deals with a specific message type. If one processing unit fails, we will get a long stream of out-of-sequence messages. A buffer overrun is almost certain. In some cases, we can use the message queue to absorb the pending messages. This works only if the messaging infrastructure allows us to read messages from the queue based on selection criteria as opposed to always reading the oldest message first.
One of the key features of the TCP protocol is to ensure in-sequence delivery of packets over the network. In reality, each packet may be routed through a different network path, so out-of-sequence packets occur quite frequently. The receiver maintains a circular buffer that is used as a sliding window. Receiver and sender negotiate on the number of packets to send before each acknowledgment. Because the sender waits for an acknowledgment from the receiver, a fast sender cannot outpace the receiver or cause the buffer to overflow. Specific rules also prevent the so-called Silly Window
...more
in voice over IP transmissions, filling in a blank packet results in a better user experience than issuing a re-request for a lost packet, which would cause a noticeable delay in the voice stream.
