Uploaded image for project: 'MCollective'
  1. MCollective
  2. MCO-736

Reduce number of permanent subscriptions using the ActiveMQ connector

    Details

    • Template:
    • Story Points:
      1
    • Release Notes:
      New Feature
    • Release Notes Summary:
      Hide
      Add an activemq.agents_multiplex option to enable using a single destination /topic/<collection>.agents for all agents as opposed to creating a topic subscription for each agent. This is an optimization that may make sense when running collectives with several thousands of nodes in order to reduce the number of subscriptions in the message broker. Using the option is a trade-off between increasing network traffic by delivering messages to all nodes - and letting them select messages they care about - versus increasing work in the message broker to handle large numbers of subscriptions.
      Show
      Add an activemq.agents_multiplex option to enable using a single destination /topic/<collection>.agents for all agents as opposed to creating a topic subscription for each agent. This is an optimization that may make sense when running collectives with several thousands of nodes in order to reduce the number of subscriptions in the message broker. Using the option is a trade-off between increasing network traffic by delivering messages to all nodes - and letting them select messages they care about - versus increasing work in the message broker to handle large numbers of subscriptions.

      Description

      Hi,

      First of all, in case Jira is not the right place to report this, let me know where should I put it.

      In our MCollective setup, we have around 20k hosts and we plan to scale it up to at least 30k.

      If we understand correctly the internals of MCollective, each host creates one queue subscription (to /queue/<collective>.nodes) but also one topic subscription per agent (to /topic/<collective>.<agent>.agent). We are using 10 agents at the moment (discovery, filemgr, mgrep, nrpe, package, process, puppet, rpcutil, service and shell) so we have a total of 11 subscriptions per host. With 30k hosts, this would be 330k subscriptions, which is very high.

      What we propose to drastically reduce the number of subscriptions: use a single destination (/topic/<collective>.agents) for all the agents and use a header key (like "mc_agent") to identify which agent should get the message. In fact, this is similar to what is already done with the "nodes" queue, using an "mc_identity" header for targeted messages.

      This simple change would divide the number of topics used by 10 (in our case, from 1500 to 150) and the number of permanent subscriptions by 5.5 (from 330k to 60k). This would simplify a lot what ActiveMQ has to do for MCollective.


      Add an activemq.agents_multiplex option to enable using a single destination /topic/<collection>.agents for all agents as opposed to creating a topic subscription for each agent.

        Attachments

          Activity

            jsd-sla-details-panel

              People

              • Assignee:
                Unassigned
                Reporter:
                AlbertoPeon Alberto Rodriguez Peon
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: