[PDB-4492] Subquery extractions of json fields throw NullPointerException - Puppet Jira Server

XML

Word

Printable

Details

Type: Bug
Status: Closed
Priority: Normal
Resolution: Won't Do
Affects Version/s: PDB 6.5.0
Fix Version/s: None
Component/s: PuppetDB
Labels:
None

Acceptance Criteria:

Hide

Either, a better error message saying "you can't/don't want to do that" if there's no reasonable use for json extraction in subqueries, or fix the type coercion matrix so that it handles json fields.

Show
Either, a better error message saying "you can't/don't want to do that" if there's no reasonable use for json extraction in subqueries, or fix the type coercion matrix so that it handles json fields.
Team:
PuppetDB
Method Found:
Manual Test

QA Risk Assessment:
Needs Assessment

Description

The type coercion matrix in the query engine (specifically engine.clj) does not properly handle json fields.

Attempting to run a query such as

curl -X POST http://localhost:8080/pdb/query/v4/facts -H 'Content-Type:application/json'   -d '{"query": ["and",

  ["=", "name", "ipaddress"],

  ["in", "value",

    ["extract", "facts", ["select_inventory", ["=", "certname", "host-9"]]]]] }'

Throws this stack trace.

2019-08-28 15:14:46,790 WARN  [qtp753236382-45] [o.e.j.s.HttpChannel] /pdb/query/v4/facts

java.lang.NullPointerException: null

        at puppetlabs.puppetdb.query_eng.engine$convert_type.invokeStatic(engine.clj:83)

        at puppetlabs.puppetdb.query_eng.engine$convert_type.invoke(engine.clj:81)

        at clojure.core$map$fn__5873.invoke(core.clj:2766)

        at clojure.lang.LazySeq.sval(LazySeq.java:42)

        at clojure.lang.LazySeq.seq(LazySeq.java:51)

        at clojure.lang.RT.seq(RT.java:535)

        at clojure.core$seq__5402.invokeStatic(core.clj:137)

        at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:24)

        at clojure.core.protocols$fn__8146.invokeStatic(protocols.clj:75)

        at clojure.core.protocols$fn__8146.invoke(protocols.clj:75)

        at clojure.core.protocols$fn__8088$G__8083__8101.invoke(protocols.clj:13)

        at clojure.core$reduce.invokeStatic(core.clj:6828)

        at clojure.core$into.invokeStatic(core.clj:6895)

        at clojure.core$mapv.invokeStatic(core.clj:6903)

        at clojure.core$mapv.invoke(core.clj:6903)

        at puppetlabs.puppetdb.query_eng.engine$eval33462$fn__33464.invoke(engine.clj:1279)

        at puppetlabs.puppetdb.query_eng.engine$eval33408$fn__33409$G__33399__33414.invoke(engine.clj:1242)

        at clojure.core$map$fn__5866.invoke(core.clj:2755)

        at clojure.lang.LazySeq.sval(LazySeq.java:42)

        at clojure.lang.LazySeq.seq(LazySeq.java:51)

        at clojure.lang.RT.seq(RT.java:535)

        at clojure.core$seq__5402.invokeStatic(core.clj:137)

        at clojure.core$map$fn__5866.invoke(core.clj:2746)

        at clojure.lang.LazySeq.sval(LazySeq.java:42)

        at clojure.lang.LazySeq.seq(LazySeq.java:51)

        at clojure.lang.Cons.next(Cons.java:39)

        at clojure.lang.LazySeq.next(LazySeq.java:83)

        at clojure.lang.RT.next(RT.java:713)

        at clojure.core$next__5386.invokeStatic(core.clj:64)

        at clojure.string$join.invokeStatic(string.clj:180)

        at clojure.string$join.invoke(string.clj:180)

        at honeysql.format$format_predicate_STAR_.invokeStatic(format.clj:366)

        at honeysql.format$format_predicate_STAR_.invoke(format.clj:356)

        at clojure.core$map$fn__5866.invoke(core.clj:2755)

        at clojure.lang.LazySeq.sval(LazySeq.java:42)

        at clojure.lang.LazySeq.seq(LazySeq.java:51)

        at clojure.lang.LazySeq.first(LazySeq.java:73)

        at clojure.lang.RT.first(RT.java:692)

        at clojure.core$first__5384.invokeStatic(core.clj:55)

        at clojure.string$join.invokeStatic(string.clj:180)

        at clojure.string$join.invoke(string.clj:180)

        at honeysql.format$format_predicate_STAR_.invokeStatic(format.clj:366)

        at honeysql.format$format_predicate_STAR_.invoke(format.clj:356)

        at honeysql.format$eval27602$fn__27604.invoke(format.clj:414)

        at clojure.lang.MultiFn.invoke(MultiFn.java:234)

        at honeysql.format$_format_clause.invokeStatic(format.clj:394)

        at honeysql.format$_format_clause.invoke(format.clj:391)

        at honeysql.format$eval27526$fn__27527$fn__27528$fn__27529.invoke(format.clj:326)

        at clojure.core$comp$fn__5807.invoke(core.clj:2569)

        at clojure.core$map$fn__5866.invoke(core.clj:2755)

        at clojure.lang.LazySeq.sval(LazySeq.java:42)

        at clojure.lang.LazySeq.seq(LazySeq.java:51)

        at clojure.lang.Cons.next(Cons.java:39)

        at clojure.lang.RT.next(RT.java:713)

        at clojure.core$next__5386.invokeStatic(core.clj:64)

        at clojure.string$join.invokeStatic(string.clj:191)

        at clojure.string$join.invoke(string.clj:180)

        at honeysql.format$space_join.invokeStatic(format.clj:15)

        at honeysql.format$space_join.invoke(format.clj:14)

        at honeysql.format$eval27526$fn__27527$fn__27528.invoke(format.clj:325)

        at honeysql.format$eval27526$fn__27527.invoke(format.clj:323)

        at honeysql.format$eval27306$fn__27307$G__27297__27312.invoke(format.clj:90)

        at honeysql.format$format.invokeStatic(format.clj:236)

        at honeysql.format$format.doInvoke(format.clj:208)

        at clojure.lang.RestFn.invoke(RestFn.java:410)

        at puppetlabs.puppetdb.query_eng.engine$eval33373$sql_from_query__33378$fn__33379.invoke(engine.clj:1236)

        at puppetlabs.puppetdb.query_eng.engine$eval33373$sql_from_query__33378.invoke(engine.clj:1233)

        at puppetlabs.puppetdb.query_eng.engine$eval33506$fn__33508.invoke(engine.clj:1251)

        at puppetlabs.puppetdb.query_eng.engine$eval33408$fn__33409$G__33399__33414.invoke(engine.clj:1242)

        at puppetlabs.puppetdb.query_eng.engine$plan__GT_sql.invokeStatic(engine.clj:1370)

        at puppetlabs.puppetdb.query_eng.engine$plan__GT_sql.invoke(engine.clj:1367)

        at puppetlabs.puppetdb.query_eng.engine$compile_user_query__GT_sql.invokeStatic(engine.clj:2458)

        at puppetlabs.puppetdb.query_eng.engine$compile_user_query__GT_sql.doInvoke(engine.clj:2440)

        at clojure.lang.RestFn.invoke(RestFn.java:442)

        at puppetlabs.puppetdb.query_eng$query__GT_sql.invokeStatic(query_eng.clj:129)

        at puppetlabs.puppetdb.query_eng$query__GT_sql.invoke(query_eng.clj:95)

        at puppetlabs.puppetdb.query_eng$eval34712$produce_streaming_body__34717$fn__34718$fn__34721.invoke(query_eng.clj:216)

        at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn$fn__25714$fn__25715.invoke(jdbc.clj:483)

        at clojure.java.jdbc$db_transaction_STAR_.invokeStatic(jdbc.clj:771)

        at clojure.java.jdbc$db_transaction_STAR_.invoke(jdbc.clj:741)

        at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn$fn__25714.invoke(jdbc.clj:482)

        at puppetlabs.puppetdb.jdbc$eval25688$retry_sql_STAR___25693$fn__25694$fn__25695.invoke(jdbc.clj:454)

        at puppetlabs.puppetdb.jdbc$eval25688$retry_sql_STAR___25693$fn__25694.invoke(jdbc.clj:453)

        at puppetlabs.puppetdb.jdbc$eval25688$retry_sql_STAR___25693.invoke(jdbc.clj:444)

        at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn.invokeStatic(jdbc.clj:480)

        at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn.invoke(jdbc.clj:469)

        at puppetlabs.puppetdb.query_eng$eval34712$produce_streaming_body__34717$fn__34718.invoke(query_eng.clj:212)

        at puppetlabs.puppetdb.query_eng$eval34712$produce_streaming_body__34717.invoke(query_eng.clj:198)

        at puppetlabs.puppetdb.http.query$query_handler$fn__37580.invoke(query.clj:388)

        at clojure.core$comp$fn__5807.invoke(core.clj:2569)

        at clojure.core$comp$fn__5807.invoke(core.clj:2569)

        at compojure.response$eval2299$fn__2300.invoke(response.clj:33)

        at compojure.response$eval2254$fn__2255$G__2245__2262.invoke(response.clj:6)

        at puppetlabs.puppetdb.http.handlers$eval37972$facts_routes__37977$fn__37978$fn__37979.invoke(handlers.clj:255)

        at puppetlabs.puppetdb.http.query$extract_query$fn__37562.invoke(query.clj:318)

        at bidi.ring$eval36000$fn__36001.invoke(ring.cljc:25)

        at bidi.ring$eval35979$fn__35980$G__35970__35989.invoke(ring.cljc:16)

        at puppetlabs.puppetdb.middleware$eval36736$make_pdb_handler__36745$fn__36748$fn__36750.invoke(middleware.clj:336)

        at puppetlabs.puppetdb.middleware$wrap_with_illegal_argument_catch$fn__36631.invoke(middleware.clj:99)

        at puppetlabs.puppetdb.middleware$verify_accepts_content_type$fn__36635.invoke(middleware.clj:114)

        at puppetlabs.puppetdb.middleware$verify_content_type$fn__36647.invoke(middleware.clj:144)

        at puppetlabs.puppetdb.middleware$wrap_with_metrics$fn__36663$fn__36672.invoke(middleware.clj:239)

        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__34789$fn__34790$fn__34791.invoke(metrics.clj:14)

        at puppetlabs.puppetdb.utils.metrics.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__34789$fn__34790.invoke(metrics.clj:14)

        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__34789$fn__34790$fn__34791.invoke(metrics.clj:14)

        at puppetlabs.puppetdb.utils.metrics.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__34789$fn__34790.invoke(metrics.clj:14)

        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__34789$fn__34790$fn__34791.invoke(metrics.clj:14)

        at puppetlabs.puppetdb.utils.metrics.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__34789$fn__34790.invoke(metrics.clj:14)

        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__34789$fn__34790$fn__34791.invoke(metrics.clj:14)

        at puppetlabs.puppetdb.utils.metrics.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__34789$fn__34790.invoke(metrics.clj:14)

        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_.invokeStatic(metrics.clj:17)

        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_.invoke(metrics.clj:6)

        at puppetlabs.puppetdb.middleware$wrap_with_metrics$fn__36663.invoke(middleware.clj:238)

        at puppetlabs.puppetdb.middleware$wrap_with_globals$fn__36626.invoke(middleware.clj:93)

        at puppetlabs.puppetdb.http.server$build_app$fn__38351.invoke(server.clj:76)

        at compojure.core$routing$fn__4129.invoke(core.clj:151)

        at clojure.core$some.invokeStatic(core.clj:2701)

        at clojure.core$some.invoke(core.clj:2692)

        at compojure.core$routing.invokeStatic(core.clj:151)

        at compojure.core$routing.doInvoke(core.clj:148)

        at clojure.lang.RestFn.invoke(RestFn.java:423)

        at puppetlabs.puppetdb.pdb_routing$wrap_with_context$fn__51634.invoke(pdb_routing.clj:35)

        at compojure.core$if_context$fn__4193.invoke(core.clj:218)

        at compojure.core$routing$fn__4129.invoke(core.clj:151)

        at clojure.core$some.invokeStatic(core.clj:2701)

        at clojure.core$some.invoke(core.clj:2692)

        at compojure.core$routing.invokeStatic(core.clj:151)

        at compojure.core$routing.doInvoke(core.clj:148)

        at clojure.lang.RestFn.applyTo(RestFn.java:139)

        at clojure.core$apply.invokeStatic(core.clj:667)

        at clojure.core$apply.invoke(core.clj:660)

        at compojure.core$routes$fn__4133.invoke(core.clj:156)

        at compojure.core$routing$fn__4129.invoke(core.clj:151)

        at clojure.core$some.invokeStatic(core.clj:2701)

        at clojure.core$some.invoke(core.clj:2692)

        at compojure.core$routing.invokeStatic(core.clj:151)

        at compojure.core$routing.doInvoke(core.clj:148)

        at clojure.lang.RestFn.invoke(RestFn.java:460)

        at puppetlabs.puppetdb.pdb_routing$pdb_app$fn__51645.invoke(pdb_routing.clj:62)

        at compojure.core$if_context$fn__4193.invoke(core.clj:218)

        at ring.middleware.params$wrap_params$fn__34893.invoke(params.clj:64)

        at puppetlabs.puppetdb.middleware$wrap_with_certificate_cn$fn__36616.invoke(middleware.clj:75)

        at puppetlabs.puppetdb.middleware$wrap_with_default_body$fn__36621.invoke(middleware.clj:82)

        at puppetlabs.puppetdb.middleware$wrap_with_debug_logging$fn__36599.invoke(middleware.clj:39)

        at puppetlabs.i18n.core$locale_negotiator$fn__124.invoke(core.clj:357)

        at puppetlabs.trapperkeeper.services.webserver.jetty9_core$ring_handler$fn__43852.invoke(jetty9_core.clj:433)

        at puppetlabs.trapperkeeper.services.webserver.jetty9_core.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a.handle(Unknown Source)

        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)

        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)

        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)

        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:205)

        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)

        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)

        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)

        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)

        at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:753)

        at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:174)

        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)

        at org.eclipse.jetty.server.Server.handle(Server.java:505)

        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)

        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)

        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)

        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)

        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)

        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698)

        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804)

        at java.base/java.lang.Thread.run(Thread.java:835)

Attachments

Issue Links

relates to

PDB-2634 PQL & Hashes: Provide hash projection capability for facts & resource params

Resolved

Activity

People

Assignee:: Unassigned

Reporter:: Austin Blatt

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 2019/08/28 3:28 PM

Updated:: 2022/01/17 4:39 PM

Resolved:: 2022/01/12 12:03 PM