Questions[activemq](server)

Can someone clarify the difference between ActiveMQ and ActiveMQ-Artemis? From what I understood on Wikipedia, ActiveMQ-Artemis is essentially the successor to ActiveMQ, but if that is the case, I do not understand why Artemis is listed as a subproject of ActiveMQ.

Moreover, if I look at the versioning for ActiveMQ and ActiveMQ-Artemis, I see that the versions are completely different. ActiveMQ being at 5.15 and Artemis being at 2.2.

From what I understand, Artemis is a faster, more recent implementation, including support for JMS 2.0. Given a new installation & deployment of an MQ platform, is should ActiveMQ 5.15 be considered as deprecated in lieu of Artemis?

I've been trying all do to set up MCollective on my puppet cluster. No matter what I do, I can't seem to get the MCollective server to connect. The MCollective server(s) are running mostly on Ubnutu Xenial. The ActiveMQ broker (5.14.3) is running on Debian Stretch. I'm running puppet 4.x on all nodes. I've used every transport connector I can thing if, and they all fail to connect. Let me dump some log files at you.

In mcollective.log I'm getting Connection reset by peer:

I, [2017-01-27T15:43:59.869501 #18729]  INFO -- : activemq.rb:139:in `on_ssl_connecting' Establishing SSL session with stomp+ssl://[email protected]:61614
E, [2017-01-27T15:44:00.070995 #18729] ERROR -- : activemq.rb:149:in `on_ssl_connectfail' SSL session creation with stomp+ssl://[email protected]:61614 failed: Connection reset by peer - SSL_connect
I, [2017-01-27T15:44:00.071371 #18729]  INFO -- : activemq.rb:129:in `on_connectfail' TCP Connection to stomp+ssl://[email protected]:61614 failed on attempt 24

Oddly enough, in the ActiveMQ log, I also seem to be getting Connection reset by peer:

ERROR | Could not accept connection from null : {}
java.io.IOException: java.io.IOException: Connection reset by peer
    at org.apache.activemq.transport.nio.NIOSSLTransport.initializeStreams(NIOSSLTransport.java:188)[activemq-client.jar:]
    at org.apache.activemq.transport.stomp.StompNIOSSLTransport.initializeStreams(StompNIOSSLTransport.java:57)[activemq-stomp.jar:]
    at org.apache.activemq.transport.tcp.TcpTransport.connect(TcpTransport.java:543)[activemq-client.jar:]
    at org.apache.activemq.transport.nio.NIOTransport.doStart(NIOTransport.java:174)[activemq-client.jar:]
    at org.apache.activemq.transport.nio.NIOSSLTransport.doStart(NIOSSLTransport.java:462)[activemq-client.jar:]
    at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)[activemq-client.jar:]
    at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64)[activemq-client.jar:]
    at org.apache.activemq.transport.stomp.StompTransportFilter.start(StompTransportFilter.java:65)[activemq-stomp.jar:]
    at org.apache.activemq.transport.AbstractInactivityMonitor.start(AbstractInactivityMonitor.java:169)[activemq-client.jar:]
    at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64)[activemq-client.jar:]
    at org.apache.activemq.broker.TransportConnection.start(TransportConnection.java:1072)[activemq-broker.jar:]
    at org.apache.activemq.broker.TransportConnector$1$1.run(TransportConnector.java:218)[activemq-broker.jar:]
    at java.lang.Thread.run(Thread.java:745)[:1.8.0_111]

So they're both resetting the connection. Huh. Before you ask, no: there are no iptables rules, and yes: there is a route between the two nodes. Let's take a peak at lsof -i just to be sure and then I'll throw some config files at you.

java    20833 activemq   84u  IPv6  53552      0t0  TCP *:61614 (LISTEN)

activemq.xml:

<!DOCTYPE activemq [
  <!ENTITY keyStores SYSTEM "keyStores.xml">
]>
<beans
  xmlns="http://www.springframework.org/schema/beans"
  xmlns:amq="http://activemq.apache.org/schema/core"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
  http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">

    <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"/>

    <broker xmlns="http://activemq.apache.org/schema/core"
            useJmx="false"
            brokerName="broker"
            dataDirectory="${activemq.base}/data">

      <persistenceAdapter>
        <kahaDB directory="${activemq.base}/data/kahadb"/>
      </persistenceAdapter>

      <sslContext>
        &keyStores;
      </sslContext>

      <transportConnectors>
        <transportConnector
          name="stomp+nio"
          uri="stomp+nio+ssl://0.0.0.0:61614?needClientAuth=true&amp;transport.enabledProtocols=TLSv1,TLSv1.1,TLSv1.2"/>
      </transportConnectors>

      <plugins>
        <simpleAuthenticationPlugin>
          <users>
            <authenticationUser username="mcollective" password="password" groups="mcollective,everyone"/>
            <authenticationUser username="admin" password="password" groups="mcollective,admins,everyone"/>
          </users>
        </simpleAuthenticationPlugin>
        <authorizationPlugin>
          <map>
            <authorizationMap>
              <authorizationEntries>
                <authorizationEntry queue=">" write="admins" read="admins" admin="admins" />
                <authorizationEntry topic=">" write="admins" read="admins" admin="admins" />
                <authorizationEntry topic="mcollective.>" write="mcollective" read="mcollective" admin="mcollective" />
                <authorizationEntry queue="mcollective.>" write="mcollective" read="mcollective" admin="mcollective" />
                <authorizationEntry topic="ActiveMQ.Advisory.>" read="everyone" write="everyone" admin="everyone"/>
              </authorizationEntries>
            </authorizationMap>
          </map>
        </authorizationPlugin>
      </plugins>
    </broker>

</beans>

keyStores.xml:

<sslContext
    keyStore="/etc/activemq/keystore.jks"
    keyStorePassword="password"
    trustStore="/etc/activemq/truststore.jks"
    trustStorePassword="password" />

mcollective/server.cfg:

# /etc/mcollective/server.cfg

# ActiveMQ connector settings:
connector = activemq
direct_addressing = 1
plugin.activemq.pool.size = 1
plugin.activemq.pool.1.host = broker.example.com
plugin.activemq.pool.1.port = 61614
plugin.activemq.pool.1.user = mcollective
plugin.activemq.pool.1.password = password
plugin.activemq.pool.1.ssl = 1
plugin.activemq.pool.1.ssl.ca = /etc/puppetlabs/puppet/ssl/certs/ca.pem
plugin.activemq.pool.1.ssl.cert = /etc/puppetlabs/puppet/ssl/certs/mail.example.com.pem
plugin.activemq.pool.1.ssl.key = /etc/puppetlabs/puppet/ssl/private_keys/mail.example.com.pem
plugin.activemq.pool.1.ssl.fallback = 0

# SSL security plugin settings:
securityprovider = ssl
plugin.ssl_client_cert_dir = /etc/puppetlabs/mcollective/clients
plugin.ssl_server_private = /etc/puppetlabs/mcollective/server_private.pem
plugin.ssl_server_public = /etc/puppetlabs/mcollective/server_public.pem

# Facts, identity, and classes:
identity = mail.example.com
factsource = yaml
plugin.yaml = /etc/puppetlabs/mcollective/facts.yaml
classesfile = /var/lib/puppet/state/classes.txt

# No additional subcollectives:
collectives = mcollective
main_collective = mcollective

# Registration:
# We don't configure a listener, and only send these messages to keep the
# Stomp connection alive. This will use the default "agentlist" registration
# plugin.
registerinterval = 600

# Auditing (optional):
# If you turn this on, you must arrange to rotate the log file it creates.
rpcaudit = 1
rpcauditprovider = logfile
plugin.rpcaudit.logfile = /var/log/mcollective-audit.log

# Authorization:
# If you turn this on now, you won't be able to issue most MCollective
# commands, although `mco ping` will work. You should deploy the
# ActionPolicy plugin before uncommenting this; see "Deploy Plugins" below.

# rpcauthorization = 1
# rpcauthprovider = action_policy
# plugin.actionpolicy.allow_unconfigured = 1

# Logging:
logger_type = file
loglevel = debug
logfile = /var/log/mcollective.log
keeplogs = 5
max_log_size = 2097152
logfacility = user

# Platform defaults:
# These settings differ based on platform; the default config file created by
# the package should include correct values. If you are managing settings as
# resources, you can ignore them, but with a template you'll have to account
# for the differences.
libdir = /usr/share/mcollective/plugins
daemonize = 1

The keys/certs in keystore.jks and truststore.jks are correct and so is the password. The shared keys and certs are also available. So lets try connecting with openssl:

root@mail:/etc/puppetlabs/puppet/ssl# openssl s_client -connect broker.example.com:61614 -CAfile certs/ca.pem -cert certs/mail.example.com.pem -key private_keys/mail.example.com.pem
CONNECTED(00000003)
write:errno=104
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 305 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1485554633
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

I'm not exactly sure how to interpret this, maybe someone else knows. Any new good guesses would be helpful, I'm stuck.

This is very strange and I can't get my head around it. I have set up a nginx server to reverse proxy a docker container. Nginx seems to be able to server some pages, but not sub-directories, or perhaps it can't serve dynamic content. I am not entirely sure what's going on here.

For example I am trying to reverse proxy the activemq admin console as such with the following config, but it doesn't seem to be able to serve subdirectories:

server {
        listen 8162;
        server_name dockerdev1.site.com;

        ssl on;
        ssl_certificate /var/certificates/cert.com.crt;
        ssl_certificate_key /var/certificates/cert.com.key;

        ssl_session_timeout 5m;

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers "HIGH:!aNULL:!MD5 or HIGH:!aNULL:!MD5:!3DES";
        ssl_prefer_server_ciphers on;


        location / {
        include  /etc/nginx/mime.types;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass https://dockerdev1.site.com:11113;
                try_files $uri $uri/ =404;

        }


}

Aim

At the moment there are more than 100 systems in the office and the expectation is that the number will increase. All systems are managed by Puppet. If Puppet code changes on the Puppet Master and needs to be implemented on multiple systems, Puppet is run manually by executing sudo puppet agent -t on every system. Sometimes a change needs to be implemented on more than 10 systems. In order to safe time the idea is to use MCollective, i.e. run puppet one time on all systems instead of running puppet on every system.

Attempt

ActiveMQ, MCollective Server and MCollective Client have been installed on the Puppetmaster. Once mco ping is run the Puppetmaster can be found. After installing MCollective Puppet Agent and running mco rpc puppet runonce Puppet is run on the Puppetmaster.

In order to connect a remote mcollective-client the idea was that activemq and mcollective-client have to be installed and run in order to find the second host by issuing mco ping.

/var/log/mcollective.log

INFO -- : activemq.rb:114:in `on_connecting' TCP Connection attempt 23 to stomp://mcollective@stomp1:6163

/etc/activemq/activemq.xml

    <transportConnectors>
        <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
            <transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
        </transportConnectors>
</broker>

/etc/mcollective/server.cfg

connector = activemq
plugin.activemq.pool.size = 1
plugin.activemq.pool.1.host = localhost
plugin.activemq.pool.1.port = 61613
plugin.activemq.pool.1.user = mcollective
plugin.activemq.pool.1.password = helloworld

One of the assumptions was that multiple hosts should be configured in the server.cfg. While trying to make it work it became unclear how the architecture should look like. One ActiveMQ, one MCollective Server and multiple MCollective Clients (one on every node) are required or not?

Expected result

The expected result is that mco ping finds the Puppetmaster and the remote client. Once this works it would be possible to run Puppet on both systems using MCollective. When it is possible to execute Puppet on these two systems using MCollective, multiple nodes will be added and Puppet will run on these systems once mco rpc puppet runonce has been executed.

Current outcome

As it is not possible to find the remote client by executing mco ping it is not possible to run Puppet on both systems using MCollective.

Our company now uses Websphere MQ to facilitate communication with our business partners, but we're evaluating migration to ActiveMQ. I've read some documentation and articles and haven't found particular option we need - server-to-server connections.

Apache MQ has notion of broker networks, but as I understood it's more of a clustering then site-to-site communication.

We need to have a local message broker (AMQ server) and a remote one. Local is used by our applications and remote one is used by our partner's apps, there is a limited number of remote queues (on partner's broker) we need to put messages to and we also have some queues on our side which our partners have to put messages to. Is there a possibility to implement this workload with ApacheMQ?

From what I've read in the documentation the broker networks and distributed queues are a clustering solution to load balance messages between [geographically or otherwise] separated brokers. This presents HUGE security problems in our case.

Let's imagine we have two partners - A and B. we need to send messages to queue a.receive.q and b.receive.q on A and B respectively. From what I read A actually can subscribe to b.receive.q and vice versa. Also, if some hostile employee on our side (consider this just for the sake of risk assessment) launches local consumer on a.receive.q, messages won't be sent over the network at all (or only a fraction will be sent).

Feel free to ask questions if I'm a bit vague.