[PDB-2262] NullPointerException while executing "extract" query against /pdb/query/v4/events Created: 2015/12/16  Updated: 2016/03/17  Resolved: 2015/12/23

Status: Closed
Project: PuppetDB
Component/s: None
Affects Version/s: PDB 3.1.1
Fix Version/s: PDB 4.0.0

Type: Bug Priority: Normal
Reporter: Lexa Whitehurst Assignee: Unassigned
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Template:
Story Points: 1
Sprint: PuppetDB 2015-12-30

 Description   

PDB 3.1.1 (from PE 2015.2.3)

I was experimenting to try to better understand how the extract operator works and what weird uses I can make for it in the v4 API. I tried something that I wasn't sure would work but was curious what it would do if it did, and PDB returned 500 referencing a NullPointerException.

Here is the query and output:

[root@pe-mom1-prod ~]# curl -XGET localhost:8080/pdb/query/v4/events -d 'query=["extract", ["certname"], ["and", ["=", "resource_type", "User"], ["=", "status", "noop"]], ["group_by", "certname", "resource_title"]]'
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /pdb/query/v4/events. Reason:
<pre>    java.lang.NullPointerException</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>

Here is what I believe is the relevant stacktrace from puppetdb.log

2015-12-16 19:02:58,889 WARN  [o.e.j.s.HttpChannel] /pdb/query/v4/events
java.lang.NullPointerException: null
        at clojure.core$name.invoke(core.clj:1524) ~[puppetdb.jar:na]
        at honeysql.format$eval2542$fn__2543.invoke(format.clj:306) ~[na:na]
        at honeysql.format$eval2496$fn__2497$G__2487__2502.invoke(format.clj:261) ~[na:na]
        at honeysql.format$to_sql.invoke(format.clj:340) ~[na:na]
        at honeysql.format$eval2564$fn__2565.invoke(format.clj:295) ~[na:na]
        at honeysql.format$eval2496$fn__2497$G__2487__2502.invoke(format.clj:261) ~[na:na]
        at honeysql.format$to_sql.invoke(format.clj:340) ~[na:na]
        at clojure.core$map$fn__4553.invoke(core.clj:2624) ~[puppetdb.jar:na]
        at clojure.lang.LazySeq.sval(LazySeq.java:40) ~[puppetdb.jar:na]
        at clojure.lang.LazySeq.seq(LazySeq.java:49) ~[puppetdb.jar:na]
        at clojure.lang.ChunkedCons.chunkedNext(ChunkedCons.java:59) ~[puppetdb.jar:na]
        at clojure.lang.ChunkedCons.next(ChunkedCons.java:43) ~[puppetdb.jar:na]
        at clojure.lang.LazySeq.next(LazySeq.java:81) ~[puppetdb.jar:na]
        at clojure.lang.RT.next(RT.java:674) ~[puppetdb.jar:na]
        at clojure.core$next__4112.invoke(core.clj:64) ~[puppetdb.jar:na]
        at clojure.string$join.invoke(string.clj:186) ~[puppetdb.jar:na]
        at honeysql.format$comma_join.invoke(format.clj:12) ~[na:na]
        at honeysql.format$eval2600$fn__2602.invoke(format.clj:375) ~[na:na]
        at clojure.lang.MultiFn.invoke(MultiFn.java:233) ~[puppetdb.jar:na]
        at honeysql.format$_format_clause.invoke(format.clj:364) ~[na:na]
        at honeysql.format$eval2554$fn__2555$fn__2556$fn__2557.invoke(format.clj:317) ~[na:na]
        at clojure.core$comp$fn__4495.invoke(core.clj:2438) ~[puppetdb.jar:na]
        at clojure.core$map$fn__4553.invoke(core.clj:2624) ~[puppetdb.jar:na]
        at clojure.lang.LazySeq.sval(LazySeq.java:40) ~[puppetdb.jar:na]
        at clojure.lang.LazySeq.seq(LazySeq.java:49) ~[puppetdb.jar:na]
        at clojure.lang.LazySeq.first(LazySeq.java:71) ~[puppetdb.jar:na]
        at clojure.lang.RT.first(RT.java:653) ~[puppetdb.jar:na]
        at clojure.core$first__4110.invoke(core.clj:55) ~[puppetdb.jar:na]
        at clojure.string$join.invoke(string.clj:185) ~[puppetdb.jar:na]
        at honeysql.format$space_join.invoke(format.clj:15) ~[na:na]
        at honeysql.format$eval2554$fn__2555$fn__2556.invoke(format.clj:316) ~[na:na]
        at honeysql.format$eval2554$fn__2555.invoke(format.clj:314) ~[na:na]
        at honeysql.format$eval2496$fn__2497$G__2487__2502.invoke(format.clj:261) ~[na:na]
        at honeysql.format$to_sql.invoke(format.clj:340) ~[na:na]
        at honeysql.format$format.doInvoke(format.clj:230) ~[na:na]
        at clojure.lang.RestFn.invoke(RestFn.java:410) ~[puppetdb.jar:na]
        at puppetlabs.puppetdb.query_eng.engine$eval18313$sql_from_query__18314$fn__18315.invoke(engine.clj:764) ~[na:na]
        at puppetlabs.puppetdb.query_eng.engine$eval18313$sql_from_query__18314.invoke(engine.clj:761) ~[na:na]
        at puppetlabs.puppetdb.query_eng.engine$eval18402$fn__18403.invoke(engine.clj:778) ~[na:na]
        at puppetlabs.puppetdb.query_eng.engine$eval18343$fn__18344$G__18334__18349.invoke(engine.clj:770) ~[na:na]
        at puppetlabs.puppetdb.query_eng.engine$plan__GT_sql.invoke(engine.clj:838) ~[na:na]
        at puppetlabs.puppetdb.query_eng.engine$compile_user_query__GT_sql.doInvoke(engine.clj:1376) ~[na:na]
        at clojure.lang.RestFn.invoke(RestFn.java:442) ~[puppetdb.jar:na]
        at puppetlabs.puppetdb.query.events$query__GT_sql.invoke(events.clj:179) ~[na:na]
        at puppetlabs.puppetdb.query.events$query__GT_sql.invoke(events.clj:163) ~[na:na]
        at puppetlabs.puppetdb.query_eng$entity__GT_sql_fns$fn__19888.invoke(query_eng.clj:46) ~[na:na]
        at puppetlabs.puppetdb.query_eng$produce_streaming_body$fn__19898.invoke(query_eng.clj:75) ~[na:na]
        at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn$fn__6264$fn__6265$fn__6266.invoke(jdbc.clj:233) ~[na:na]
        at clojure.java.jdbc.deprecated$transaction_STAR_.invoke(deprecated.clj:377) ~[na:na]
        at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn$fn__6264$fn__6265.invoke(jdbc.clj:233) ~[na:na]
        at clojure.java.jdbc.deprecated$with_connection_STAR_.invoke(deprecated.clj:307) ~[na:na]
        at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn$fn__6264.invoke(jdbc.clj:230) ~[na:na]
        at puppetlabs.puppetdb.jdbc$eval6242$retry_sql_STAR___6243$fn__6244$fn__6245.invoke(jdbc.clj:202) ~[na:na]
        at puppetlabs.puppetdb.jdbc$eval6242$retry_sql_STAR___6243$fn__6244.invoke(jdbc.clj:201) ~[na:na]
        at puppetlabs.puppetdb.jdbc$eval6242$retry_sql_STAR___6243.invoke(jdbc.clj:193) ~[na:na]
        at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn.invoke(jdbc.clj:229) ~[na:na]
        at puppetlabs.puppetdb.query_eng$produce_streaming_body.invoke(query_eng.clj:73) ~[na:na]
        at puppetlabs.puppetdb.http.events$routes$fn__20552$fn__20556$fn__20561.invoke(events.clj:54) ~[na:na]
        at puppetlabs.puppetdb.http.events$routes$fn__20552$fn__20556.invoke(events.clj:49) ~[na:na]
        at puppetlabs.puppetdb.http.events$routes$fn__20552.invoke(events.clj:49) ~[na:na]
        at puppetlabs.puppetdb.middleware$verify_accepts_content_type$fn__20305.invoke(middleware.clj:112) ~[na:na]
        at puppetlabs.puppetdb.middleware$validate_query_params$fn__20316.invoke(middleware.clj:153) ~[na:na]
        at puppetlabs.puppetdb.middleware$wrap_with_paging_options$fn__20300.invoke(middleware.clj:90) ~[na:na]
        at puppetlabs.puppetdb.http.v4$v4_app$fn__21467.invoke(v4.clj:21) ~[na:na]
        at net.cgrand.moustache$alter_request$fn__20436.invoke(moustache.clj:54) ~[na:na]
        at puppetlabs.puppetdb.http.v4$v4_app.invoke(v4.clj:21) ~[na:na]
        at puppetlabs.puppetdb.http.server$routes$fn__21568.invoke(server.clj:34) ~[na:na]
        at net.cgrand.moustache$alter_request$fn__20436.invoke(moustache.clj:54) ~[na:na]
        at puppetlabs.puppetdb.http.server$routes.invoke(server.clj:34) ~[na:na]
        at puppetlabs.puppetdb.middleware$wrap_with_metrics_STAR_$fn__20332$fn__20341.invoke(middleware.clj:206) ~[na:na]
        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__19961$fn__19962$fn__19963.invoke(metrics.clj:14) ~[na:na]
        at puppetlabs.puppetdb.utils.metrics.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na]
        at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na]
        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__19961$fn__19962.invoke(metrics.clj:14) ~[na:na]
        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__19961$fn__19962$fn__19963.invoke(metrics.clj:14) ~[na:na]
        at puppetlabs.puppetdb.utils.metrics.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na]
        at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na]
        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__19961$fn__19962.invoke(metrics.clj:14) ~[na:na]
        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__19961$fn__19962$fn__19963.invoke(metrics.clj:14) ~[na:na]
        at puppetlabs.puppetdb.utils.metrics.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na]
        at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na]
        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__19961$fn__19962.invoke(metrics.clj:14) ~[na:na]
        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__19961$fn__19962$fn__19963.invoke(metrics.clj:14) ~[na:na]
        at puppetlabs.puppetdb.utils.metrics.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source) ~[na:na]
        at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na]
        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_$fn__19961$fn__19962.invoke(metrics.clj:14) ~[na:na]
        at puppetlabs.puppetdb.utils.metrics$multitime_BANG__STAR_.invoke(metrics.clj:17) ~[na:na]
        at puppetlabs.puppetdb.middleware$wrap_with_metrics_STAR_$fn__20332.invoke(middleware.clj:205) ~[na:na]
        at puppetlabs.puppetdb.middleware$wrap_with_globals$fn__20296.invoke(middleware.clj:78) ~[na:na]
        at compojure.core$routing$fn__9579.invoke(core.clj:127) ~[na:na]
        at clojure.core$some.invoke(core.clj:2570) ~[puppetdb.jar:na]
        at compojure.core$routing.doInvoke(core.clj:127) ~[na:na]
        at clojure.lang.RestFn.invoke(RestFn.java:423) ~[puppetdb.jar:na]
        at puppetlabs.puppetdb.pdb_routing$wrap_with_context$fn__35855.invoke(pdb_routing.clj:27) ~[na:na]
        at compojure.core$if_context$fn__9641.invoke(core.clj:194) ~[na:na]
        at compojure.core$routing$fn__9579.invoke(core.clj:127) ~[na:na]
        at clojure.core$some.invoke(core.clj:2570) ~[puppetdb.jar:na]
        at compojure.core$routing.doInvoke(core.clj:127) ~[na:na]
        at clojure.lang.RestFn.applyTo(RestFn.java:139) ~[puppetdb.jar:na]
        at clojure.core$apply.invoke(core.clj:632) ~[puppetdb.jar:na]
        at compojure.core$routes$fn__9583.invoke(core.clj:132) ~[na:na]
        at compojure.core$routing$fn__9579.invoke(core.clj:127) ~[na:na]
        at clojure.core$some.invoke(core.clj:2570) ~[puppetdb.jar:na]
        at compojure.core$routing.doInvoke(core.clj:127) ~[na:na]
        at clojure.lang.RestFn.invoke(RestFn.java:486) ~[puppetdb.jar:na]
        at puppetlabs.puppetdb.pdb_routing$pdb_app$fn__35868.invoke(pdb_routing.clj:46) ~[na:na]
        at compojure.core$if_context$fn__9641.invoke(core.clj:194) ~[na:na]
        at ring.middleware.params$wrap_params$fn__20055.invoke(params.clj:64) ~[na:na]
        at puppetlabs.puppetdb.middleware$wrap_with_authorization$fn__20283.invoke(middleware.clj:45) ~[na:na]
        at puppetlabs.puppetdb.middleware$wrap_with_certificate_cn$fn__20288.invoke(middleware.clj:60) ~[na:na]
        at puppetlabs.puppetdb.middleware$wrap_with_default_body$fn__20292.invoke(middleware.clj:67) ~[na:na]
        at puppetlabs.puppetdb.middleware$wrap_with_debug_logging$fn__20279.invoke(middleware.clj:29) ~[na:na]
        at puppetlabs.trapperkeeper.services.webserver.jetty9_core$ring_handler$fn__29420.invoke(jetty9_core.clj:408) ~[na:na]
        at puppetlabs.trapperkeeper.services.webserver.jetty9_core.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a.handle(Unknown Source) ~[na:na]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1129) ~[puppetdb.jar:na]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1065) ~[puppetdb.jar:na]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[puppetdb.jar:na]
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) ~[puppetdb.jar:na]
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) ~[puppetdb.jar:na]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) ~[puppetdb.jar:na]
        at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95) ~[puppetdb.jar:na]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) ~[puppetdb.jar:na]
        at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:159) ~[puppetdb.jar:na]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) ~[puppetdb.jar:na]
        at org.eclipse.jetty.server.Server.handle(Server.java:497) ~[puppetdb.jar:na]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) ~[puppetdb.jar:na]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) [puppetdb.jar:na]
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) [puppetdb.jar:na]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) [puppetdb.jar:na]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) [puppetdb.jar:na]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]



 Comments   
Comment by Kurt Wall [ 2015/12/29 ]

Verified in the latest 4.0 snapshot (4.0.0.SNAPSHOT.2015.12.29T0941). From PDB 3.2.1, showing the error:

# puppetdb --version
puppetdb version: 3.2.1
# curl -XGET localhost:8080/pdb/query/v4/events -d 'query=["extract", ["certname"], ["and", ["=", "resource_type", "User"], ["=", "status", "noop"]], ["group_by", "certname", "resource_title"]]'
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /pdb/query/v4/events. Reason:
<pre>    java.lang.NullPointerException</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>

After updating puppetdb to 4.0.0.SNAPSHOT.2015.12.29T0941:

# puppetdb --version
puppetdb version: 4.0.0.SNAPSHOT.2015.12.29T0941
# curl -XGET localhost:8080/pdb/query/v4/events -d 'query=["extract", ["certname"], ["and", ["=", "resource_type", "User"], ["=", "status", "noop"]], ["group_by", "certname", "resource_title"]]'
[]

Generated at Thu Aug 22 11:10:17 PDT 2019 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.