Uploaded image for project: 'PuppetDB'
  1. PuppetDB
  2. PDB-1605

puppet query to puppetdb fails on invalid cache + missing data

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: PDB 3.0.0
    • Fix Version/s: PDB 3.0.0
    • Component/s: None
    • Labels:
      None
    • Template:
    • Story Points:
      2
    • Sprint:
      PuppetDB 2015-06-17

      Description

      You can reproduce this by doing the following:

      rm ~/.puppet/var/yaml/facts/*
      

      then clear your database, start puppetdb, and

      puppet agent -t
      

      You should see something like this in the master log if you're running with --debug:

      Debug: Routes Registered:
      Debug: Route /^\/v2\.0/
      Debug: Route /.*/
      Debug: Evaluating match for Route /^\/v2\.0/
      Debug: Did not match path ("/production/node/mbp.local")
      Debug: Evaluating match for Route /.*/
      Debug: Creating new connection for https://mbp.local:8081
      Warning: Error connecting to mbp.local on 8081 at route /pdb/query/v4/nodes/mbp.local/facts, error message received was 'Not Found'.
      Warning: Unable to find results for mbp.local error: Failed to find '/pdb/query/v4/nodes/mbp.local/facts' on server: 'mbp.local' and port: '8081'
      Info: Caching node for mbp.local
      Debug: Failed to load library 'msgpack' for feature 'msgpack'
      Debug: Puppet::Network::Format[msgpack]: feature msgpack is missing
      Debug: node supports formats: pson b64_zlib_yaml yaml raw
      Debug: Routes Registered:
      Debug: Route /^\/v2\.0/
      Debug: Route /.*/
      Debug: Evaluating match for Route /^\/v2\.0/
      Debug: Did not match path ("/production/file_metadatas/plugins")
      Debug: Evaluating match for Route /.*/
      Debug: Failed to load library 'msgpack' for feature 'msgpack'
      Debug: Puppet::Network::Format[msgpack]: feature msgpack is missing
      Debug: file_metadata supports formats: pson b64_zlib_yaml yaml raw
      Debug: Routes Registered:
      Debug: Route /^\/v2\.0/
      Debug: Route /.*/
      Debug: Evaluating match for Route /^\/v2\.0/
      Debug: Did not match path ("/production/catalog/mbp.local")
      Debug: Evaluating match for Route /.*/
      Debug: Creating new connection for https://mbp.local:8081
      Info: 'replace facts' command for mbp.local submitted to PuppetDB with UUID 8ba17b7d-bf83-4201-ab2b-56ff49f79d93
      Debug: Using cached facts for mbp.local
      Info: Caching node for mbp.local
      Notice: Compiled catalog for mbp.local in environment production in 0.01 seconds
      Info: Caching catalog for mbp.local
      Debug: Creating new connection for https://mbp.local:8081
      Info: 'replace catalog' command for mbp.local submitted to PuppetDB with UUID fac80a88-78cd-448d-bce4-3a9b26a9b3c4
      Debug: Failed to load library 'msgpack' for feature 'msgpack'
      Debug: Puppet::Network::Format[msgpack]: feature msgpack is missing
      Debug: catalog supports formats: pson b64_zlib_yaml yaml dot raw
      Debug: Routes Registered:
      Debug: Route /^\/v2\.0/
      Debug: Route /.*/
      Debug: Evaluating match for Route /^\/v2\.0/
      Debug: Did not match path ("/production/report/mbp.local")
      Debug: Evaluating match for Route /.*/
      Debug: Received report to process from mbp.local
      Debug: Processing report from mbp.local with processor Puppet::Reports::Store
      Debug: Processing report from mbp.local with processor Puppet::Reports::Log
      Info: //mbp.local/Puppet: Retrieving plugin
      Info: //mbp.local/Puppet: Caching catalog for mbp.local
      Info: //mbp.local/Puppet: Applying configuration version '1433878349'
      Notice: //mbp.local/Puppet: Finished catalog run in 0.01 seconds
      Debug: Processing report from mbp.local with processor Puppet::Reports::Puppetdb
      Debug: Creating new connection for https://mbp.local:8081
      Info: 'store report' command for mbp.local submitted to PuppetDB with UUID 25e23645-9ee9-4276-9427-f1daee2e63dc
      

      despite the fact that it gets no data from PuppetDB, it's caching the correct facts data in that yaml file and nothing actually seems to be adversely affected. We need to understand why Puppet is connection to PDB instead of using the facts it's getting from the agent.

      This also may be affecting Puppet 4. In that case the yaml files are in ~/.puppetlabs instead of ~/.puppet.

      Maybe unrelated, but also notice that

      Info: Caching node for mbp.local
      

      is printed twice in that log. "Caching facts for mbp.local" is not printed at all, despite the fact that the run starts with the cache deleted and the master logs the line

      Debug: Using cached facts for mbp.local
      

      after the error occurs. Suspicious all around. This may be a puppet bug but we should understand it better before handing it off.

      Note that I've seen this under non-manufactured circumstances as well, where instead of being missing the cache is expired (and the master drops a log message to that effect.) I haven't been able to reproduce this by just winding back the expiration dates in the cache file, so I'm not sure what determines what provokes the error in that case.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              wyatt Wyatt Alt
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Zendesk Support