Uploaded image for project: 'Puppet Server'
  1. Puppet Server
  2. SERVER-556

Generate function makes puppetserver crash with "java.lang.OutOfMemoryError: Java heap space" Exception

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: SERVER 2.0.0
    • Fix Version/s: SERVER 1.y, SERVER 2.y
    • Component/s: None
    • Labels:
      None
    • Environment:
    • Template:
    • Sub-team:
    • Story Points:
      5
    • Sprint:
      Server Jade 2015-05-27, Server Jade 2015-06-10

      Description

      Hi.
      The puppetserver crashes with a java.lang.OutOfMemoryError: Java heap space Exception when it calls an external program.

      Code
      $password = generate('/usr/bin/python2.7', "${settings::confdir}/opt/bin/bacula-genkey", "${settings::confdir}/opt/etc/baculaclient/${::fqdn}.key")

      Stacktrace:
      2015-04-17 13:32:57,604 INFO [puppet-server] access[^/puppet/v3/catalog/([^/]+)$] allowing $1 access
      2015-04-17 13:32:57,605 INFO [puppet-server] access[^/puppet/v3/node/([^/]+)$] allowing 'method' find
      2015-04-17 13:32:57,605 INFO [puppet-server] access[^/puppet/v3/node/([^/]+)$] allowing $1 access
      2015-04-17 13:32:57,606 INFO [puppet-server] access[^/puppet/v3/report/([^/]+)$] allowing 'method' save
      2015-04-17 13:32:57,606 INFO [puppet-server] access[^/puppet/v3/report/([^/]+)$] allowing $1 access
      2015-04-17 13:32:57,606 INFO [puppet-server] access[/puppet/v3/file] allowing * access
      2015-04-17 13:32:57,606 INFO [puppet-server] access[/puppet/v3/status] allowing 'method' find
      2015-04-17 13:32:57,607 INFO [puppet-server] access[/puppet/v3/status] allowing * access
      2015-04-17 13:32:57,607 INFO [puppet-server] access[/puppet-ca/v1/certificate_revocation_list/ca] allowing 'method' find
      2015-04-17 13:32:57,607 INFO [puppet-server] access[/puppet-ca/v1/certificate_revocation_list/ca] allowing * access
      2015-04-17 13:32:57,607 INFO [puppet-server] access[/puppet-ca/v1/certificate/ca] adding authentication any
      2015-04-17 13:32:57,607 INFO [puppet-server] access[/puppet-ca/v1/certificate/ca] allowing 'method' find
      2015-04-17 13:32:57,608 INFO [puppet-server] access[/puppet-ca/v1/certificate/ca] allowing * access
      2015-04-17 13:32:57,608 INFO [puppet-server] access[/puppet-ca/v1/certificate/] adding authentication any
      2015-04-17 13:32:57,608 INFO [puppet-server] access[/puppet-ca/v1/certificate/] allowing 'method' find
      2015-04-17 13:32:57,608 INFO [puppet-server] access[/puppet-ca/v1/certificate/] allowing * access
      2015-04-17 13:32:57,609 INFO [puppet-server] access[/puppet-ca/v1/certificate_request] adding authentication any
      2015-04-17 13:32:57,609 INFO [puppet-server] access[/puppet-ca/v1/certificate_request] allowing 'method' find
      2015-04-17 13:32:57,609 INFO [puppet-server] access[/puppet-ca/v1/certificate_request] allowing 'method' save
      2015-04-17 13:32:57,609 INFO [puppet-server] access[/puppet-ca/v1/certificate_request] allowing * access
      2015-04-17 13:32:57,610 INFO [puppet-server] access[/] adding authentication any
      2015-04-17 13:32:58,865 INFO [puppet-server] Puppet Caching node for puppet.puppet.local
      2015-04-17 13:35:29,970 WARN [o.e.j.u.t.QueuedThreadPool]
      java.lang.OutOfMemoryError: Java heap space
      at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57) ~[na:1.7.0_75]
      at java.nio.ByteBuffer.allocate(ByteBuffer.java:331) ~[na:1.7.0_75]
      at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:792) ~[na:1.7.0_75]
      at java.nio.charset.Charset.encode(Charset.java:842) ~[na:1.7.0_75]
      at java.nio.charset.Charset.encode(Charset.java:865) ~[na:1.7.0_75]
      at org.jruby.RubyEncoding.encode(RubyEncoding.java:236) ~[puppet-server-release.jar:na]
      at org.jruby.RubyString.encodeBytelist(RubyString.java:7684) ~[puppet-server-release.jar:na]
      at org.jruby.RubyString.<init>(RubyString.java:414) ~[puppet-server-release.jar:na]
      at org.jruby.RubyString.newString(RubyString.java:508) ~[puppet-server-release.jar:na]
      at org.jruby.Ruby.newString(Ruby.java:3378) ~[puppet-server-release.jar:na]
      at org.jruby.RubyBasicObject.hashyInspect(RubyBasicObject.java:1020) ~[puppet-server-release.jar:na]
      at org.jruby.RubyBasicObject.inspect(RubyBasicObject.java:999) ~[puppet-server-release.jar:na]
      at org.jruby.RubyKernel.inspect(RubyKernel.java:2107) ~[puppet-server-release.jar:na]
      at org.jruby.RubyKernel$INVOKER$s$0$0$inspect.call(RubyKernel$INVOKER$s$0$0$inspect.gen) ~[puppet-server-release.jar:na]
      at org.jruby.runtime.Helpers.invokedynamic(Helpers.java:2798) ~[puppet-server-release.jar:na]
      at org.jruby.RubyBasicObject.inspectObj(RubyBasicObject.java:1052) ~[puppet-server-release.jar:na]
      at org.jruby.RubyBasicObject.hashyInspect(RubyBasicObject.java:1020) ~[puppet-server-release.jar:na]
      at org.jruby.RubyBasicObject.inspect(RubyBasicObject.java:999) ~[puppet-server-release.jar:na]
      at org.jruby.RubyKernel.inspect(RubyKernel.java:2107) ~[puppet-server-release.jar:na]
      at org.jruby.RubyKernel$INVOKER$s$0$0$inspect.call(RubyKernel$INVOKER$s$0$0$inspect.gen) ~[puppet-server-release.jar:na]
      at org.jruby.runtime.Helpers.invokedynamic(Helpers.java:2798) ~[puppet-server-release.jar:na]
      at org.jruby.RubyBasicObject.inspectObj(RubyBasicObject.java:1052) ~[puppet-server-release.jar:na]
      at org.jruby.RubyBasicObject.hashyInspect(RubyBasicObject.java:1020) ~[puppet-server-release.jar:na]
      at org.jruby.RubyBasicObject.inspect(RubyBasicObject.java:999) ~[puppet-server-release.jar:na]
      at org.jruby.RubyKernel.inspect(RubyKernel.java:2107) ~[puppet-server-release.jar:na]
      at org.jruby.RubyKernel$INVOKER$s$0$0$inspect.call(RubyKernel$INVOKER$s$0$0$inspect.gen) ~[puppet-server-release.jar:na]
      at org.jruby.runtime.Helpers.invokedynamic(Helpers.java:2798) ~[puppet-server-release.jar:na]
      at org.jruby.RubyBasicObject.inspectObj(RubyBasicObject.java:1052) ~[puppet-server-release.jar:na]
      at org.jruby.RubyBasicObject.hashyInspect(RubyBasicObject.java:1020) ~[puppet-server-release.jar:na]
      at org.jruby.RubyBasicObject.inspect(RubyBasicObject.java:999) ~[puppet-server-release.jar:na]
      at org.jruby.RubyKernel.inspect(RubyKernel.java:2107) ~[puppet-server-release.jar:na]
      at org.jruby.RubyKernel$INVOKER$s$0$0$inspect.call(RubyKernel$INVOKER$s$0$0$inspect.gen) ~[puppet-server-release.jar:na]
      ^[[B^C2015-04-17 13:37:49,637 INFO [p.t.internal] Shutting down due to JVM shutdown hook.
      2015-04-17 13:37:49,638 INFO [p.t.internal] Beginning shutdown sequence
      2015-04-17 13:37:49,640 INFO [p.t.s.w.jetty9-service] Shutting down web server(s).
      2015-04-17 13:37:49,641 INFO [p.t.s.w.jetty9-core] Shutting down web server.
      2015-04-17 13:37:49,644 INFO [o.e.j.s.ServerConnector] Stopped ServerConnector@791bdd14

      {SSL-HTTP/1.1} {0.0.0.0:8140}

      2015-04-17 13:37:49,644 INFO [o.e.j.s.h.ContextHandler] Stopped o.e.j.s.h.ContextHandler@769fe49f

      {/puppet-admin-api,null,UNAVAILABLE}

      2015-04-17 13:37:49,644 INFO [o.e.j.s.h.ContextHandler] Stopped o.e.j.s.h.ContextHandler@5e35a03e

      {/,null,UNAVAILABLE}

      2015-04-17 13:37:49,645 INFO [o.e.j.s.h.ContextHandler] Stopped o.e.j.s.h.ContextHandler@a642e32

      {/puppet,null,UNAVAILABLE}

      2015-04-17 13:37:49,645 INFO [o.e.j.s.h.ContextHandler] Stopped o.e.j.s.h.ContextHandler@f268f64

      {/puppet-ca,null,UNAVAILABLE}

      2015-04-17 13:37:49,649 INFO [p.t.s.w.jetty9-core] Web server shutdown
      2015-04-17 13:37:49,651 INFO [p.t.internal] Finished shutdown sequence

      Call from shell:
      root@puppet:/home/vagrant# sudo -u puppet python /etc/puppetlabs/puppet/opt/bin/bacula-genkey /etc/puppetlabs/puppet/opt/etc/baculaclient/puppet.puppet.local.key
      H150L2364C0IL8KMFXNTUIGM2KPJ6MEXTroot@puppet:/home/vagrant#

      Memory allocation:
      root@puppet:/home/vagrant# cat /etc/default/puppetserver
      ###########################################

      1. Init settings for puppetserver
        ###########################################
      1. Location of your Java binary (version 7 or higher)
        JAVA_BIN="/usr/bin/java"
      1. Modify this if you'd like to change the memory allocation, enable JMX, etc
        JAVA_ARGS="-Xms7g -Xmx7g -XX:MaxPermSize=4g"

        Attachments

        1. 20150429_java_pid17668.hprof.part01.rar
          9.00 MB
        2. 20150429_java_pid17668.hprof.part02.rar
          9.00 MB
        3. 20150429_java_pid17668.hprof.part03.rar
          9.00 MB
        4. 20150429_java_pid17668.hprof.part04.rar
          9.00 MB
        5. 20150429_java_pid17668.hprof.part05.rar
          9.00 MB
        6. 20150429_java_pid17668.hprof.part06.rar
          9.00 MB
        7. 20150429_java_pid17668.hprof.part07.rar
          9.00 MB
        8. 20150429_java_pid17668.hprof.part08.rar
          9.00 MB
        9. 20150429_java_pid17668.hprof.part09.rar
          1.37 MB
        10. java_pid.part01.rar
          9.00 MB
        11. java_pid.part02.rar
          9.00 MB
        12. java_pid.part03.rar
          9.00 MB
        13. java_pid.part04.rar
          9.00 MB
        14. java_pid.part05.rar
          9.00 MB
        15. java_pid.part06.rar
          9.00 MB
        16. java_pid.part07.rar
          9.00 MB
        17. java_pid.part08.rar
          3.47 MB
        18. java_pid15802.hprof.part1.rar
          9.00 MB
        19. java_pid15802.hprof.part2.rar
          9.00 MB
        20. java_pid15802.hprof.part3.rar
          9.00 MB
        21. java_pid15802.hprof.part4.rar
          9.00 MB
        22. java_pid15802.hprof.part5.rar
          9.00 MB
        23. java_pid15802.hprof.part6.rar
          9.00 MB
        24. java_pid15802.hprof.part7.rar
          9.00 MB
        25. java_pid15802.hprof.part8.rar
          5.53 MB
        26. puppetlabs.tar.gz
          975 kB
        27. thread-stacks.txt
          176 kB

          Issue Links

            Activity

              People

              Assignee:
              preben Preben Ingvaldsen
              Reporter:
              olhe Olof Hellqvist
              QA Contact:
              Erik Dasher
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Zendesk Support