[PUP-8467] Starting to get broken pipe (HTTP 500) on certain manifest - gone after restarting puppetserver - Puppet Tickets

XML

Word

Printable

Details

Type: Bug
Status: Closed
Priority: Normal
Resolution: Fixed
Affects Version/s: None
Fix Version/s: PUP 4.10.11, PUP 5.3.6, PUP 5.5.0
Component/s: None
Labels:
None

Template:
PUP Bug Template customfield_10700 225926
Team:
Froyo
Method Found:
Needs Assessment

Release Notes:
Bug Fix
Release Notes Summary:
Warnings from internal poor coding practices should no longer be logged to the process' stderr (resolving an issue where log managers could cause a broken pipe).

QA Risk Assessment:
Needs Assessment

Description

After a while a certain manifest started firing HTTP 500 on the puppetserver side, though nothing had changed.

clients were aborting catalog compilation with:

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Method call, Broken pipe - <STDERR> at /srv/puppet/code/environments/production/modules/amodule/manifests/init.pp:16:35 on node node1.example.com

Server logs were showing the following:

2017-12-08 18:29:02,277 ERROR [qtp526724218-38738] [puppetserver] Puppet Evaluation Error: Error while evaluating a Method call, Broken pipe - <STDERR> at /srv/puppet/code/environments/production/modules/amodule/manifests/init.pp:16:35 on node node1.example.com

org/jruby/RubyIO.java:1457:in `write'

org/jruby/RubyIO.java:1428:in `write'

org/jruby/RubyKernel.java:1000:in `eval'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/ruby_function_instantiator.rb:22:in `create'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/module_loaders.rb:166:in `find'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/base_loader.rb:151:in `internal_load'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/base_loader.rb:32:in `load_typed'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/base_loader.rb:143:in `internal_load'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/base_loader.rb:32:in `load_typed'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/base_loader.rb:143:in `internal_load'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/base_loader.rb:32:in `load_typed'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/base_loader.rb:143:in `internal_load'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/base_loader.rb:32:in `load_typed'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/base_loader.rb:143:in `internal_load'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/base_loader.rb:32:in `load_typed'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/loader.rb:53:in `load'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:304:in `call_function'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:951:in `call_function_with_block'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:944:in `eval_CallMethodExpression'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:90:in `visit_this_1'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:81:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:358:in `eval_AssignmentExpression'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:90:in `visit_this_1'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:81:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:646:in `block in eval_BlockExpression'

org/jruby/RubyArray.java:1734:in `each'

org/jruby/RubyEnumerable.java:936:in `inject'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:646:in `eval_BlockExpression'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:90:in `visit_this_1'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:81:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/parser/evaluating_parser.rb:63:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:32:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:72:in `block in evaluate'

org/jruby/RubyKernel.java:1120:in `catch'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:71:in `block in evaluate'

org/jruby/RubyKernel.java:1120:in `catch'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:70:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast.rb:31:in `safeevaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/type.rb:136:in `evaluate_code'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:79:in `block in evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:71:in `evaluate'

org/jruby/RubyArray.java:1734:in `each'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:384:in `evaluate_classes'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/../../../puppet/functions/include.rb:16:in `include'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatch.rb:60:in `invoke'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:43:in `block in dispatch'

org/jruby/RubyKernel.java:1120:in `catch'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:42:in `dispatch'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/function.rb:46:in `block in call'

org/jruby/RubyKernel.java:1120:in `catch'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/function.rb:45:in `call'

/srv/puppet/code/environments/production/modules/site/configsets/manifests/types/immerx_10.pp:6:in `<eval>'

org/jruby/RubyKernel.java:1000:in `eval'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:307:in `block in call_function'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:305:in `call_function'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:949:in `call_function_with_block'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:918:in `eval_CallNamedFunctionExpression'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:90:in `visit_this_1'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:81:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:646:in `block in eval_BlockExpression'

org/jruby/RubyArray.java:1734:in `each'

org/jruby/RubyEnumerable.java:936:in `inject'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:646:in `eval_BlockExpression'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:90:in `visit_this_1'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:81:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/parser/evaluating_parser.rb:63:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:32:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:72:in `block in evaluate'

org/jruby/RubyKernel.java:1120:in `catch'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:71:in `block in evaluate'

org/jruby/RubyKernel.java:1120:in `catch'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:70:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast.rb:31:in `safeevaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/type.rb:136:in `evaluate_code'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:79:in `block in evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:71:in `evaluate'

org/jruby/RubyArray.java:1734:in `each'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:384:in `evaluate_classes'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/../../../puppet/functions/include.rb:16:in `include'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatch.rb:60:in `invoke'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:43:in `block in dispatch'

org/jruby/RubyKernel.java:1120:in `catch'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:42:in `dispatch'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/function.rb:46:in `block in call'

org/jruby/RubyKernel.java:1120:in `catch'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/function.rb:45:in `call'

/srv/puppet/code/environments/production/manifests/site.pp:121:in `<eval>'

org/jruby/RubyKernel.java:1000:in `eval'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:307:in `block in call_function'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:305:in `call_function'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:949:in `call_function_with_block'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:918:in `eval_CallNamedFunctionExpression'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:90:in `visit_this_1'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:81:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:646:in `block in eval_BlockExpression'

org/jruby/RubyArray.java:1734:in `each'

org/jruby/RubyEnumerable.java:936:in `inject'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:646:in `eval_BlockExpression'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:90:in `visit_this_1'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:81:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/parser/evaluating_parser.rb:63:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:32:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast.rb:31:in `safeevaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/type.rb:133:in `block in evaluate_code'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/scope.rb:953:in `with_guarded_scope'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/type.rb:131:in `evaluate_code'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:79:in `block in evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:71:in `evaluate'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:505:in `evaluate_ast_node'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:168:in `block in compile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:168:in `block in compile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:263:in `override'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:153:in `compile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:33:in `compile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:297:in `block in compile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:295:in `block in compile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:224:in `block in benchmark'

uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/benchmark.rb:308:in `realtime'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:223:in `benchmark'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:293:in `compile'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:52:in `find'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:195:in `find'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:121:in `do_find'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:48:in `block in call'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:263:in `override'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:47:in `call'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:82:in `block in process'

Restarting the puppetserver made the problem go away.

All hosts using that module were affected.

The offending line in the manifest looks like:

   $dump_sources = $hosts.map |$h| { "https://${acustomfcuntion('somevalue',$h)}.example.com" }

Attachments

Issue Links

relates to

PUP-8463 Remove existing warnings and run PRs through 'ruby -wc'

Closed

Activity

People

Assignee:: Peter Meier

Reporter:: Peter Meier

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 2017/12/08 10:02 AM

Updated:: 2018/09/12 1:36 AM

Resolved:: 2018/02/20 1:56 PM