Details
-
Bug
-
Status: Accepted
-
Normal
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
-
Needs Assessment
-
Needs Assessment
Description
Hey josh,
I did further debugging for https://tickets.puppetlabs.com/browse/PUP-10772. while doing so I discovered that I've 5 Ubuntu 16.04 machines, all of them currently show the
`Failed to deserialize Puppet::Resource::Catalog` error message that I show later
my client:
root~ # facter -p os
|
pup{
|
architecture => "amd64",
|
distro => {
|
codename => "xenial",
|
description => "Ubuntu 16.04.7 LTS",
|
id => "Ubuntu",
|
release => {
|
full => "16.04",
|
major => "16.04"
|
}
|
},
|
family => "Debian",
|
hardware => "x86_64",
|
name => "Ubuntu",
|
release => {
|
full => "16.04",
|
major => "16.04"
|
},
|
selinux => {
|
enabled => false
|
}
|
}
|
root~ # puppet --version
|
6.19.1
|
root ~ #
|
preferred_serialization_format is not set:
root ~ # cat /etc/puppetlabs/puppet/puppet.conf
|
|
|
[main]
|
certname = *
|
codedir = /etc/puppetlabs/code
|
hostprivkey = $privatekeydir/$certname.pem { mode = 640 }
|
logdir = /var/log/puppetlabs/puppet
|
pluginfactsource = puppet:///pluginfacts
|
pluginsource = puppet:///plugins
|
privatekeydir = $ssldir/private_keys { group = service }
|
rundir = /var/run/puppetlabs
|
server = *
|
show_diff = true
|
ssldir = /etc/puppetlabs/puppet/ssl
|
vardir = /opt/puppetlabs/puppet/cache
|
|
|
[agent]
|
classfile = $statedir/classes.txt
|
default_schedules = false
|
environment = production
|
localconfig = $vardir/localconfig
|
masterport = 8140
|
noop = false
|
report = true
|
runinterval = 7200
|
splay = true
|
splaylimit = 7200
|
usecacheonfailure = true
|
root ~ #
|
but msgpack is installed:
root ~ # /opt/puppetlabs/puppet/bin/gem list msgpack
|
|
|
*** LOCAL GEMS ***
|
|
|
msgpack (1.3.3, 1.3.1)
|
root ~ #
|
during a puppetrun to a 6.14.1 puppetserver with msgpack installed I see the following messages:
starting SSL for *:8140...
|
SSL established
|
<- "POST /puppet/v3/catalog/*?environment=production HTTP/1.1\r\nX-Puppet-Version: 6.19.1\r\nUser-Agent: Puppet/6.19.1 Ruby/2.5.8-p224 (x86_64-linux)\r\nAccept: application/vnd.puppet.rich+json, application/json, application/vnd.puppet.rich+msgpack, application/x-msgpack, text/pson\r\nContent-Type: application/x-www-form-urlencoded\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nHost: *:8140\r\nContent-Length: 40356\r\n\r\n"
|
...
|
Info: Unable to serialize catalog to json, retrying with pson
|
...
|
the agent run works fine. Now the same agent connects to a puppetserver without msgpack installed:
...
|
starting SSL for *:8140...
|
SSL established
|
<- "POST /puppet/v3/catalog/*?environment=production HTTP/1.1\r\nX-Puppet-Version: 6.19.1\r\nUser-Agent: Puppet/6.19.1 Ruby/2.5.8-p224 (x86_64-linux)\r\nAccept: application/vnd.puppet.rich+json, application/json, application/vnd.puppet.rich+msgpack, application/x-msgpack, text/pson\r\nContent-Type: application/x-www-form-urlencoded\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nHost: *:8140\r\nContent-Length: 40357\r\n\r\n"
|
...
|
-> "Content-Type: text/pson;charset=iso-8859-1\r\n"
|
-> "Transfer-Encoding: chunked\r\n"
|
-> "Connection: keep-alive\r\n"
|
-> "X-Puppet-Version: 6.19.1\r\n"
|
-> "Vary: Accept-Encoding, User-Agent\r\n"
|
-> "Content-Encoding: gzip\r\n"
|
...
|
Error: Could not retrieve catalog from remote server: Failed to deserialize Puppet::Resource::Catalog from pson: expected value in object at '"ELF\u0002\u0001\u0'!
|
Wrapped exception:
|
expected value in object at '"ELF\u0002\u0001\u0'!
|
Warning: Not using cache on failed catalog
|
Error: Could not retrieve catalog; skipping run
|
now it fails? Sadly I don't know since when this is the case. I suspect someone added a new file resource with a binary that's read in with file(). But this is really just a guess. Our codebase is pretty huge and digging through all of the modules takes some time. I cannot reproduce this on other agents with a different operating system. Sadly the code on those Ubuntu 16.04 boxes does only work on this specific OS. I don't know if this Ubuntu specific or indeed some puppet code.
Third run. msgpack installed again on the puppetserver. also enabled preferred_serialization_format = msgpack on the agent:
starting SSL for *:8140...
|
SSL established
|
<- "POST /puppet/v3/catalog/*?environment=production HTTP/1.1\r\nX-Puppet-Version: 6.19.1\r\nUser-Agent: Puppet/6.19.1 Ruby/2.5.8-p224 (x86_64-linux)\r\nAccept: application/vnd.puppet.rich+msgpack, application/x-msgpack, application/vnd.puppet.rich+json, application/json, text/pson\r\nContent-Type: application/x-www-form-urlencoded\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nHost: *:8140\r\nContent-Length: 40356\r\n\r\n"
|
...
|
-> "Date: Thu, 05 Nov 2020 17:14:27 GMT\r\n"
|
-> "Content-Type: application/vnd.puppet.rich+msgpack;charset=iso-8859-1\r\n"
|
-> "Transfer-Encoding: chunked\r\n"
|
-> "Connection: keep-alive\r\n"
|
-> "X-Puppet-Version: 6.19.1\r\n"
|
-> "Vary: Accept-Encoding, User-Agent\r\n"
|
-> "Content-Encoding: gzip\r\n"
|
...
|
Info: Unable to serialize catalog to json, retrying with pson
|
....
|
the agent run works as well with this setup. but I think the info message is wrong. While it might try pson, afterwards msgpack is used to decode the payload? Or the msgpack serialized to msgpack?
Attachments
Issue Links
- relates to
-
-
- Resolved
-