[PUP-9647] Pip package provider does not handle pip executable paths with spaces Created: 2019/04/17  Updated: 2019/12/12  Resolved: 2019/10/15

Status: Resolved
Project: Puppet
Component/s: Types and Providers
Affects Version/s: PUP 5.5.10, PUP 6.4.0
Fix Version/s: PUP 5.5.18, PUP 6.4.5, PUP 6.11.0

Type: Bug Priority: Normal
Reporter: Sean Millichamp Assignee: Josh Cooper
Resolution: Fixed Votes: 0
Labels: maintenance, resolved-issue-added, triage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Template: PUP Bug Template
Team: Coremunity
Sprint: Platform Core KANBAN
Method Found: Customer Feedback
Zendesk Ticket IDs: 34604
Zendesk Ticket Count: 1
Release Notes: Bug Fix
Release Notes Summary: Puppet could not manage pip resources if the pip command was in a directory containing spaces, like C:\Program Files\Python27 on Windows.
QA Risk Assessment: Needs Assessment


The package pip provider has some suggestion of Windows support. However, it uses "execpipe" to invoke commands, which puts a shell redirection of "2>&1" into the final command which is not compatible with Windows.

On a Windows system with Python installed when the "pip_version" method is called it results in an error: "Cannot collect packages for Puppet::Type::Package::ProviderPip3 provider; undefined method `[]' for nil:NilClass"

Reworking this in terms of execute allows the pip provider to work on Windows as well.


Comment by Sean Millichamp [ 2019/04/17 ]

I have created two PRs which seem to resolve the issue:

against master: https://github.com/puppetlabs/puppet/pull/7487

against 5.5.x: https://github.com/puppetlabs/puppet/pull/7488


Comment by Sean Millichamp [ 2019/06/01 ]

I have pushed a rebase required due to recent changes in the provider to https://github.com/puppetlabs/puppet/pull/7488

Comment by Josh Cooper [ 2019/06/25 ]

I think the issue occurs if pip --version returns output that doesn't match the expected regex. To repro, created pip executable shell script in current working directory:

echo whoops

PATH=.:$PATH bundle exec puppet apply -e "package { 'foo': provider => pip }" --trace
Notice: Compiled catalog for neptune in environment production in 0.02 seconds
Error: Failed to apply catalog: undefined method `[]' for nil:NilClass
/Users/josh/work/puppet/lib/puppet/provider/package/pip.rb:73:in `block (2 levels) in pip_version'
/Users/josh/work/puppet/lib/puppet/provider/package/pip.rb:72:in `each'
/Users/josh/work/puppet/lib/puppet/provider/package/pip.rb:72:in `collect'
/Users/josh/work/puppet/lib/puppet/provider/package/pip.rb:72:in `block in pip_version'
/Users/josh/work/puppet/lib/puppet/util/execution.rb:81:in `block (2 levels) in execpipe'
/usr/local/Cellar/rbenv/1.1.2/versions/2.3.8/lib/ruby/2.3.0/open-uri.rb:37:in `open'
/usr/local/Cellar/rbenv/1.1.2/versions/2.3.8/lib/ruby/2.3.0/open-uri.rb:37:in `open'
/Users/josh/work/puppet/lib/puppet/util/execution.rb:80:in `block in execpipe'
/Users/josh/work/puppet/lib/puppet/util.rb:129:in `withenv'
/Users/josh/work/puppet/lib/puppet/util/execution.rb:79:in `execpipe'
/Users/josh/work/puppet/lib/puppet/provider.rb:117:in `execpipe'
/Users/josh/work/puppet/lib/puppet/provider/package/pip.rb:71:in `pip_version'
/Users/josh/work/puppet/lib/puppet/provider/package/pip.rb:35:in `instances'

Since the pip provider combines stdout and stderr, it's possible something is being written to stderr. This can be simulated using:

echo whoops >/dev/stderr
echo pip 1.2.3

The pip provider shouldn't assume each line of output from pip --version matches the regex, see https://github.com/puppetlabs/puppet/blob/179bd10781978ac631eb13fd5b311b420e2c609c/lib/puppet/provider/package/pip.rb#L45

Comment by Sean Millichamp [ 2019/06/26 ]

I dig a bit more digging around. I believe this is the output I'm getting which is causing the version match to fail:

'C:/Program' is not recognized as an internal or external command, operable program or batch file.

The actual path (as returned by Puppet::Util.which('pip.exe')) is "C:/Program Files/Python35/Scripts/pip.exe"

It looks like the command returned needs to have double-quotes around it for it to work properly on Windows, and the double quotes don't seem to negatively affect running it on Linux.

Comment by Josh Cooper [ 2019/07/15 ]

Ah nice find Sean Millichamp!

Comment by Josh Cooper [ 2019/10/07 ]

Merged to 5.5.x in https://github.com/puppetlabs/puppet/commit/9a2305bb527c93b25c2e6bf7db958f87860cc8a4

Generated at Mon Jan 27 12:09:35 PST 2020 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.