[PDB-1249] PR (1270): (maint) benchmark improvements - wkalt Created: 2015/02/23  Updated: 2015/07/10  Resolved: 2015/03/10

Status: Closed
Project: PuppetDB
Component/s: None
Affects Version/s: None
Fix Version/s: PDB 3.0.0

Type: Improvement Priority: Normal
Reporter: gepetto-bot Assignee: Wyatt Alt
Resolution: Fixed Votes: 0
Labels: github
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Template:
Story Points: 0
Sprint: PuppetDB 2015-02-25, PuppetDB 2015-03-11

 Description   

(maint) benchmark improvements

Pull Request Description


This adds two optional flags to benchmark:

A/-archive is passed a path to a PDB export, and will use whatever
catalogs/reports/facts are inside.

D/-default is used without a parameter and will use the sample data in
acceptance/benchmark/. Currently this is just facts but will be expanded when
the upcoming command changes land.


(webhooks-id: 38bd3294be2c62d8c4abc73150b1bb7a)



 Comments   
Comment by gepetto-bot [ 2015/02/23 ]

pljenkinsro commented:

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/656/

Comment by gepetto-bot [ 2015/02/23 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/662/

Comment by gepetto-bot [ 2015/02/23 ]

wkalt commented:

@pljenkinsro retest this please

Comment by gepetto-bot [ 2015/02/23 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/668/

Comment by gepetto-bot [ 2015/02/24 ]

kbarber commented:

@wkalt there is a docs page where this tool's flags are documented in load_testing_tool fyi.

Comment by gepetto-bot [ 2015/02/24 ]

kbarber commented:

@wkalt oh yeah, and release notes .

Comment by gepetto-bot [ 2015/02/24 ]

wkalt commented:

@kbarber dealt with docs and release notes

Comment by gepetto-bot [ 2015/02/24 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/675/

Comment by gepetto-bot [ 2015/02/24 ]

wkalt commented:

@pljenkinsro retest this please

Comment by gepetto-bot [ 2015/02/24 ]

pljenkinsro commented:

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/677/

Comment by gepetto-bot [ 2015/02/25 ]

kbarber commented:

This seems pretty cool.

Found a bug: If you pass in a tarball with -A, that has no reports, this will break. In fact I bet this happens for any missing object type. I only found this because I had disabled/broken reports in my test env, so I was lucky .

I think also, leading from this, this is something worth fixing. Think about -D ... it probably also suffers the same issue, I presume this is why you skipped out the other objects in the defaults there.

I think we should become resilient to missing object types, it helps both places.

Totally future stuff beyond this though:

There is another side to this, we could potentially drop the use of the 3 object types switches for paths, but just expecting a root dir (like an export root with facts/reports/catalogs in it). This way, we reduce the amount of switches, and force people to pass in 'export' format objects only, either a tarball, or an unpacked tarball directory basically ..

This would lead to the need for another switch, that allows us to control what objects get sent though possibly .

Comment by gepetto-bot [ 2015/02/25 ]

ajroetker commented:

@kbarber
------
There is another side to this, we could potentially drop the use of the 3 object types switches for paths, but just expecting a root dir (like an export root with facts/reports/catalogs in it). This way, we reduce the amount of switches, and force people to pass in 'export' format objects only, either a tarball, or an unpacked tarball directory basically ..

This would lead to the need for another switch, that allows us to control what objects get sent though possibly .
--------

Do we not have switches for this kind of thing already in import/export? if not it seems useful to have on either end there as well, though export seems better if we had to choose.

Comment by gepetto-bot [ 2015/02/25 ]

wkalt commented:

@kbarber fixed that archive issue and tightened up the handling on empty directories

Comment by gepetto-bot [ 2015/02/25 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/715/

Comment by gepetto-bot [ 2015/02/25 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/716/

Comment by gepetto-bot [ 2015/02/25 ]

wkalt commented:

@pljenkinsro retest this please

Comment by gepetto-bot [ 2015/02/25 ]

pljenkinsro commented:

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/719/

Comment by gepetto-bot [ 2015/02/26 ]

wkalt commented:

@senior has raised the issue that we don't ship the acceptance directory, so as written package users won't be able to use the -D flag. This shouldn't be merged until we figure out what to do there.

Comment by gepetto-bot [ 2015/02/26 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/726/

Comment by gepetto-bot [ 2015/02/26 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/728/

Comment by gepetto-bot [ 2015/03/03 ]

kbarber commented:

Still blocked on a decision about what to do with -D. Right now we're proposing storing it in the resource area, so its shipped out with our jar, and works the same way in dev mode.

Comment by gepetto-bot [ 2015/03/06 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/818/

Comment by gepetto-bot [ 2015/03/06 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/819/

Comment by gepetto-bot [ 2015/03/06 ]

wkalt commented:

I've augmented the sample data with reports and catalogs and the -D flag now loads the data from the classpath.

Comment by gepetto-bot [ 2015/03/06 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/820/

Comment by gepetto-bot [ 2015/03/06 ]

wkalt commented:

@pljenkinsro retest this please

Comment by gepetto-bot [ 2015/03/06 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/821/

Comment by gepetto-bot [ 2015/03/06 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/824/

Comment by gepetto-bot [ 2015/03/09 ]

senior commented:

@wkalt I think you're missing facts from this PR, you removed it from acceptance, but I don't see it in test. For me this shows up as an NPE when I run:
```
lein run benchmark -D -c postgres.ini -r 10 -i 10 -n 10
```

Comment by gepetto-bot [ 2015/03/09 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/825/

Comment by gepetto-bot [ 2015/03/09 ]

wkalt commented:

@senior addressed your comments

Comment by gepetto-bot [ 2015/03/09 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/827/

Comment by gepetto-bot [ 2015/03/09 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/831/

Comment by gepetto-bot [ 2015/03/09 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/833/

Comment by gepetto-bot [ 2015/03/09 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/834/

Comment by gepetto-bot [ 2015/03/09 ]

wkalt commented:

I realize that having a flag for --default was a little weird. Now default is inferred if none of -F, -C, -R, or -A are supplied.

Comment by gepetto-bot [ 2015/03/09 ]

pljenkinsro commented:

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/835/

Comment by gepetto-bot [ 2015/03/10 ]

senior commented:

This is a nice change, I don't know how many times I've had to go fishing for an export tarball for benchmark.

Comment by gepetto-bot [ 2015/03/10 ]

Pull request (maint) benchmark improvements has been closed.

Generated at Tue Jun 25 08:30:51 PDT 2019 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.