Instead of waiting for clients to connect, a broker can initiate a connection to another endpoint on a specific protocol.

Currently, this feature supports only the AMQP protocol. However, in the future, it might be expanded to other protocols.

Broker connections are configured by the <broker-connections> XML element in the broker.xml configuration file.

<broker-connections>
  ...
</broker-connections>

1. AMQP Server Connections

An ActiveMQ Artemis broker can initiate connections using the AMQP protocol. This means that the broker can connect to another AMQP server (not necessarily ActiveMQ Artemis) and create elements on that connection.

To define an AMQP broker connection, add an <amqp-connection> element within the <broker-connections> element in the broker.xml configuration file. For example:

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="other-server" retry-interval="100" reconnect-attempts="-1" user="john" password="doe">
    ...
  </amqp-connection>
</broker-connections>
uri

tcp://host:myport[?options] (this is a required argument)

name

Name of the connection used for management purposes

user

Username with which to connect to the endpoint (this is an optional argument)

password

Password with which to connect to the endpoint (this is an optional argument)

retry-interval

Time, in milliseconds to wait before retrying a connection after an error. The default value is 5000.

reconnect-attempts

default is -1 meaning infinite

auto-start

Should the broker connection start automatically with the broker. Default is true. If false it is necessary to call a management operation to start it.

The connection URI options for transport settings, such as enabling and configuring TLS, are common with other Artemis connector URIs. See the transport doc for more. An example configuration for a TLS AMQP broker-connection can be found in the broker examples at ./examples/features/broker-connection/amqp-sending-overssl.
If auto-start is disabled on the broker connection, the start of the broker connection will only happen after the management method startBrokerConnection(connectionName) is called on the ServerController.
The target endpoint needs permission for all operations that are configured. Therefore, if a security manager is being used, ensure that you perform the configured operations with a user with sufficient permissions.

2. AMQP Server Connection Operations

The following types of operations are supported on an AMQP server connection:

  • Mirrors

    • The broker uses an AMQP connection to another broker and duplicates messages and sends acknowledgements over the wire.

  • Senders

    • Messages received on specific queues are transferred to another endpoint.

  • Receivers

    • The broker pulls messages from another endpoint.

  • Peers

    • The broker creates both senders and receivers on another endpoint that knows how to handle them. This is currently implemented by Apache Qpid Dispatch.

3. Reconnecting and Failover

It is possible to determine how reconnection will happen on a broker connection.

These are the attributes that are available on the amqp-connection XML element:

reconnect-attempts

default is -1 (infinite). How many attempts will be done after a failed connection

retry-interval

default is 5000, in milliseconds, the wait between each retry of a connection

It is also possible to specify alternate hosts on a broker connection by appending a comma separated list after a # at the end of the URI. The broker connection would keep trying on the alternate list until one of the targets is available to connect. Example:

<broker-connections>
  <amqp-connection uri="tcp://ServerA:5672#BackupA:5672,BackupB:5672" name="ServerA" reconnect-attempts="-1" retry-interval="5000">
    ...
  </amqp-connection>
</broker-connections>

Broker Connection Failover Figure 1. Broker Connection - Reconnecting and Failover.

The previous example portrays a case of connection failure towards ServerA. The system would try to connect to serverA, backupA, and backupB until it successfully connects to one of these nodes.

4. Mirroring

Mirroring will reproduce any operation that happened on the source brokers towards a target broker.

The following events are sent through mirroring:

  • Message sending

    • Messages sent to one broker will be "replicated" to the target broker.

  • Message acknowledgement

    • Acknowledgements removing messages at one broker will be sent to the target broker.

    • Note that if the message is pending for a consumer on the target mirror, the ack will not succeed and the message might be delivered by both brokers.

  • Queue and address creation.

  • Queue and address deletion.

By default, every operation is sent asynchronously without blocking any clients. However, if you set sync=true on the mirror configuration the clients will always wait for the mirror on every blocking operation.

4.1. Mirror configuration

Add a <mirror> element within the <amqp-connection> element to configure mirroring to the target broker.

The following optional arguments can be utilized:

message-acknowledgements

Specifies whether message acknowledgements are sent. The default value is true.

queue-creation

Specifies whether a queue- or address-creation event is sent. The default value is true.

queue-removal

Specifies whether a queue- or address-removal event is sent. The default value is true.

address-filter

An optional comma-separated list of inclusion and/or exclusion filter entries used to govern which addresses (and related queues) mirroring events will be created for on this broker-connection. That is, events will only be mirrored to the target broker for addresses that match the filter. An address is matched when it begins with an inclusion entry specified in this field, unless the address is also explicitly excluded by another entry.

An exclusion entry is prefixed with ! to denote any address beginning with that value does not match. If no inclusion entry is specified in the list, all addresses not explicitly excluded will match. If the address-filter attribute is not specified, then all addresses (and related queues) will match and be mirrored.

Examples:

  • eu matches all addresses starting with eu

  • !eu matches all address except for those starting with eu

  • eu.uk,eu.de matches all addresses starting with either eu.uk or eu.de

  • eu,!eu.uk matches all addresses starting with eu but not those starting with eu.uk

  • Address exclusion will always take precedence over address inclusion.

  • Address matching on mirror elements is prefix-based and does not support wild-card matching.
sync

If set to true any client blocking operation will be held until the mirror has confirmed receiving the operation. Default is false.

  • Notice that a disconnected node would hold all operations from the client. If you set sync=true you must reconnect a mirror before performing any operations.

An example of a mirror configuration is shown below:

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="mirror">
    <mirror/>
  </amqp-connection>
</broker-connections>

4.2. Store and Forward Queue

Mirror events are always stored on a local queue prefixed as $ACTIVEMQ_ARTEMIS_MIRROR_ and then concatenated with the broker connection’s configured name.

So, in the following configuration, mirror events will be stored on a queue named $ACTIVEMQ_ARTEMIS_MIRROR_brokerB.

<broker-connection>
  <amqp-connection uri="tcp://brokerB:5672" name="brokerB">
    <mirror/>
  </amqp-connection>
</broker-connection>

These messages are then transferred to brokerB:5672. A producer to the address $ACTIVEMQ_ARTEMIS_MIRROR_brokerB will be created towards brokerB. If there is a security manager configured, security roles must be provided to the user on the broker connection.

Notice the queue $ACTIVEMQ_ARTEMIS_MIRROR_brokerB will not actually exist on brokerB and so it will not be visible on the administration console. The target broker will treat these messages accordingly as mirror events and perform the appropriate operations at the target broker.

4.3. Pre Existing Messages

The broker will only mirror messages arriving from the point in time the mirror was configured. Previously existing messages will not be forwarded to other brokers.

5. Dual Mirror (Disaster Recovery)

ActiveMQ Artemis supports automatic fallback mirroring. Every sent message and every acknowledgement is asynchronously replicated to the mirrored broker.

On the following diagram there will be two servers called DataCenter1, and DataCenter2. In order to have a dual mirror configuration it is necessary to add the mirror broker connection on each broker.xml:

Broker Connection DR Figure 2. Broker Connection - Disaster Recovery.

On DataCenter1 the following code should be added on broker.xml:

<broker-connections>
   <amqp-connection uri="tcp://DataCenter2:5672" name="DC2">
      <mirror/>
   </amqp-connection>
</broker-connections>

The following xml snippet should be added on DataCenter2's broker.xml:

<broker-connections>
   <amqp-connection uri="tcp://DataCenter1:5672" name="DC1">
      <mirror/>
   </amqp-connection>
</broker-connections>

The broker connections will replicate sends and acknowledgements to the other broker, no matter where they originated. If messages are sent on DC1 (DataCenter1) these will be automatically transferred to DC2 (DataCenter2). Message acknowledgements received on DC2 will be automatically related back to DC1.

The only exception to that rule would be if there were already consumers with pending messages on any server, where a mirrored acknowledgement will not prevent the message being consumed by both consumers.

It is recommended to not have active consumers on both servers.

5.1. Example

There is an example as part of the distribution showing dual broker configuration (or disaster recovery) in ./examples/features/broker-connection/disaster-recovery.

On the provided example above, two brokers are configured to mirror each other and whatever happens in one broker is immediately copied over to the other broker.

6. Senders and Receivers

It is possible to connect an ActiveMQ Artemis broker to another AMQP endpoint simply by creating a sender or receiver broker connection element.

For a sender, the broker creates a message consumer on a queue that sends messages to another AMQP endpoint.

For a receiver, the broker creates a message producer on an address that receives messages from another AMQP endpoint.

Both elements function as a message bridge. However, there is no additional overhead required to process messages. Senders and receivers behave just like any other consumer or producer in ActiveMQ Artemis.

Specific queues can be configured by senders or receivers. Wildcard expressions can be used to match senders and receivers to specific addresses or sets of addresses. When configuring a sender or receiver, the following properties can be set:

address-match

Match the sender or receiver to a specific address or set of addresses, using a wildcard expression.

queue-name

Configure the sender or receiver for a specific queue.

6.1. Examples

Using address expressions:

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="other-server">
    <sender address-match="queues.#"/>
    <!-- notice the local queues for remotequeues.# need to be created on this broker -->
    <receiver address-match="remotequeues.#"/>
  </amqp-connection>
</broker-connections>

<addresses>
  <address name="remotequeues.A">
    <anycast>
      <queue name="remoteQueueA"/>
    </anycast>
  </address>
  <address name="queues.B">
    <anycast>
      <queue name="localQueueB"/>
    </anycast>
  </address>
</addresses>

Using queue names:

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="other-server">
    <receiver queue-name="remoteQueueA"/>
    <sender queue-name="localQueueB"/>
  </amqp-connection>
</broker-connections>

<addresses>
   <address name="remotequeues.A">
     <anycast>
       <queue name="remoteQueueA"/>
     </anycast>
   </address>
   <address name="queues.B">
     <anycast>
       <queue name="localQueueB"/>
     </anycast>
   </address>
</addresses>
Receivers can only be matched to a local queue that already exists. Therefore, if receivers are being used, ensure that queues are pre-created locally. Otherwise, the broker cannot match the remote queues and addresses.
Do not create a sender and a receiver to the same destination. This creates an infinite loop of sends and receives.

7. Peers

The broker can be configured as a peer which connects to the Apache Qpid Dispatch Router and instructs it that the broker will act as a store-and-forward queue for a given AMQP waypoint address configured on the router. In this scenario, clients connect to a router to send and receive messages using a waypointed address, and the router routes these messages to or from the queue on the broker.

The peer configuration causes ActiveMQ Artemis to create a sender and receiver pair for each destination matched in the broker-connection configuration, with these carrying special configuration to let Qpid Dispatch know to collaborate with the broker. This replaces the traditional need of a router-initiated connection and auto-links.

Qpid Dispatch Router offers a lot of advanced networking options that be used together with ActiveMQ Artemis.

With a peer configuration, the same properties are present as when there are senders and receivers. For example, a configuration where queues with names beginning queue. act as storage for the matching router waypoint address would be:

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="router">
    <peer address-match="queues.#"/>
  </amqp-connection>
</broker-connections>

<addresses>
   <address name="queues.A">
     <anycast>
       <queue name="queues.A"/>
     </anycast>
   </address>
   <address name="queues.B">
     <anycast>
       <queue name="queues.B"/>
     </anycast>
   </address>
</addresses>

There must be a matching address waypoint configuration on the router instructing it that the particular router addresses the broker attaches to should be treated as waypoints. For example, a similar prefix- based router address configuration would be:

address {
    prefix: queue
    waypoint: yes
}

For more information refer to the "brokered messaging" documentation for Apache Qpid Dispatch Router.

Do not use this feature to connect to another broker, otherwise any message sent will be immediately ready to consume creating an infinite echo of sends and receives.
It is not necessary to configure the router with a connector or auto-links to communicate with the broker. The brokers peer configuration replaces these aspects of the router waypoint usage.

8. Address Consideration

It is highly recommended that address name and queue name are the same. When a queue with its distinct name (as in the following example) is used senders and receivers will always use the address name when creating the remote endpoint.

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="other-server">
    <sender address-match="queues.#"/>
  </amqp-connection>
</broker-connections>
<addresses>
  <address name="queues.A">
    <anycast>
      <queue name="distinctNameQueue.A"/>
    </anycast>
  </address>
</addresses>

The above example would create an AMQP sender towards queues.A.

To avoid confusion it is recommended that address name and queue name are kept the same.

9. Federation

Broker federation allows the local broker to create remote receivers for addresses or queues that have local demand. Conversely, the broker connection can send federation configuration to the remote broker causing it to create receivers on the local broker based on remote demand on an address or queue over this same connection.

Add a <federation> element within the <amqp-connection> element to configure federation to the broker instance. The <amqp-connection> contains all the configuration for authentication and reconnection handling. See above sections to configure those values.

The broker connection federation configuration consists of one or more policies that define either local or remote federation configurations for addresses or queues.

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="federation-example">
    <federation>
       <local-address-policy name="example-local-address-policy">
         <include address-match="local-address.#" />
         <exclude address-match="local-address.excluded" />
       </local-address-policy>
       <local-queue-policy name="example-local-queue-policy">
         <include address-match="address" queue-match="local-queue" />
       </local-queue-policy>
       <remote-address-policy name="example-remote-address-policy">
         <include address-match="remote-address" />
       </remote-address-policy>
       <remote-queue-policy name="example-remote-queue-policy">
         <include address-match="#" queue-match="remote-queue" />
         <exclude address-match="excluded.#" queue-match="remote-queue-excluded" />
       </remote-queue-policy>
    </federation>
  </amqp-connection>
</broker-connections>

9.1. Local and remote address federation

Local or Remote address federation configures the local or remote broker to watch for demand on addresses. When demand exists it will create a consumer on the matching address on the opposing broker. Because the consumers are created on addresses on the opposing broker, the authentication credentials supplied to the broker connection must have sufficient access to read from (local address federation) or write to the matching address (remote address federation) on the opposing broker.

An example of address federation configuration is shown below. Local and remote address policies have identical configuration parameters, only the policy element names are different.

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="federation-example">
    <federation>
       <local-address-policy name="example-local-address-policy"
                             auto-delete="false"
                             auto-delete-delay="1"
                             auto-delete-message-count="12"
                             max-hops="1"
                             enable-divert-bindings="true">
         <include address-match="local-address.#" />
         <exclude address-match="local-address.excluded" />
       </local-address-policy>
    </federation>
  </amqp-connection>
</broker-connections>
name

The name of the policy, these names should be unique within a broker connection’s federation policy elements.

include

The address-match pattern to use to match included addresses, multiple of these can be set. If none are set all addresses are matched.

exclude

The address-match pattern to use to match excluded addresses, multiple of these can be set, or it can be omitted if no excludes are needed.

max-hops

The maximum number of hops that a message can perform to be federated. Default value is 0 and will work for most simple federation deployments. However, in certain topologies a higher value may be required to prevent looping.

auto-delete

For address federation, a durable queue is created on the broker from which messages are being federated. Set this property to true to mark the queue for automatic deletion once the initiating broker disconnects and the delay and message count parameters are met. This is a useful option if you want to automate the cleanup of dynamically-created queues. The default value is false, which means that the queue is not automatically deleted.

auto-delete-delay

The amount of time in milliseconds after the initiating broker has disconnected before the created queue can be eligible for auto-delete. The default value is 0 when option is omitted.

auto-delete-message-count

After the initiating broker has disconnected, the maximum number of messages allowed in the remote queue for the queue to be eligible for automatic deletion. The default value is 0.

enable-divert-bindings

Setting to true enables divert bindings to be listened-to for demand. If a divert binding with an address matches the included addresses for the address policy, any queue bindings that match the forwarding address of the divert creates demand. The default value is false.

9.2. Local and remote queue federation

Local or Remote queue federation configures the local or remote broker to watch for demand on queues and when demand exists it will create a consumer on the matching queue on the opposing broker. Because the consumers are created on queues on the opposing broker, the authentication credentials supplied to the broker connection must have sufficient access to read from (local queue federation) or write to the matching queue (remote queue federation) on the opposing broker.

An example of queue federation configuration is shown below. Local and remote queue policies have identical configuration parameters, only the policy element names are different.

<broker-connections>
  <amqp-connection uri="tcp://HOST:PORT" name="federation-example">
    <federation>
       <local-queue-policy name="example-local-queue-policy">
         <include address-match="#" queue-match="remote-queue" />
         <exclude address-match="excluded.#" queue-match="remote-queue-excluded" />
       </local-queue-policy>
    </federation>
  </amqp-connection>
</broker-connections>
name

The name of the policy. These names should be unique within a broker connection’s federation policy elements.

include

The queue-match pattern to use to match included queues. Multiple of these can be set. If none are set all queues are matched.

exclude

The queue-match pattern to use to match excluded queues. Multiple of these can be set, or it can be omitted if no excludes are needed.

priority-adjustment

When federation consumers are created this value can be used to ensure that those federation consumers have a lower priority value than other local consumers on the same queue. The default is -1.

include-federated

Controls if consumers on a queue which come from federation instances should be counted when observing a queue for demand, by default this value is false and federation consumers are not counted.