[PUP-2905] When installing PIP packages from sources different than pypi, the install and any notifies are triggered during every run. Created: 2014/07/08  Updated: 2019/10/01

Status: Accepted
Project: Puppet
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: redmine.exporter Assignee: Unassigned
Resolution: Unresolved Votes: 6
Labels: cern, git, package, pip, redmine, source, type_and_provider
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Template:
Epic Link: Pip Package Improvements
Team: Night's Watch

 Description   

I have a plugin for a Python application, Sentry, which I am installing via GitHub like so:

    package{ 'sentry-hipchat':
      ensure   => 'cfac72b33dc8468745669ea65998a5b911e9aae0',
      provider => 'pip',
      source   => 'git+git://github.com/invisiblehand/sentry-hipchat.git'
    }

When running the Puppet agent with --debug --trace --test, I can see Puppet checks the output of 'pip freeze', however the output from this command, for packages installed from GitHub seems to differ, and I don't believe it will match the regex used here:

https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/package/pip.rb#L16-L22

Here is some sample output from 'pip freeze' on my system:

    raven==3.5.1
    redis==2.8.0
    sentry==6.3.3
    -e git://github.com/invisiblehand/sentry-hipchat.git@cfac72b33dc8468745669ea65998a5b911e9aae0#egg=sentry_hipchat-master
    setproctitle==1.1.8
    simplejson==3.3.1

I'm using Puppet 2.7.13, installed via RubyGems, on Ubuntu 12.04 LTS 64-bit, running Ruby 1.9.3p448.

The output from the Puppet agent run is as follows:

    debug: Puppet::Type::Package::ProviderPip: Executing '/usr/bin/pip freeze'
    debug: Puppet::Type::Package::ProviderPip: Executing '/usr/bin/pip freeze'
    debug: Puppet::Type::Package::ProviderPip: Executing '/usr/bin/pip install -q -e git+git://github.com/invisiblehand/sentry-hipchat.git@cfac72b33dc8468745669ea65998a5b911e9aae0#egg=sentry-hipchat'
    notice: /Stage[main]/Ih-sentry/Package[sentry-hipchat]/ensure: created
    info: /Stage[main]/Ih-sentry/Package[sentry-hipchat]: Scheduling refresh of Service[sentry]
    info: /Stage[main]/Ih-sentry/Package[sentry-hipchat]: Scheduling refresh of Service[sentry-udp]
    info: /Stage[main]/Ih-sentry/Package[sentry-hipchat]: Scheduling refresh of Service[sentry-celery]

As you can see, the package is 'created', which triggers a refresh of the services, even though there is no need.

Does anyone have any advice?

Thanks!

Ryan



 Comments   
Comment by Samuele Kaplun [ 2014/07/08 ]

Same here:

Certain packages loaded with the pip provider get reinstalled every-time:
[...]
Info: Applying configuration version '1404822902'
Notice: /Stage[main]/Hg_inspire::Wn/Package[setuptools]/ensure: created
Notice: /Stage[main]/Hg_inspire::Wn/Packagehttp://www.reportlab.com/ftp/pyRXP-1.16-daily-unix.tar.gz/ensure: created
Notice: /Stage[main]/Hg_inspire::Wn/Packagehttps://py-editdist.googlecode.com/files/py-editdist-0.3.tar.gz/ensure: created
Notice: /Stage[main]/Hg_inspire::Wn::Invenio/Packagegit+git://github.com/inspirehep/harvesting-kit.git#master/ensure: created
Notice: Finished catalog run in 66.76 seconds
[...]

Comment by Samuele Kaplun [ 2014/07/08 ]

BTW, I have updated the title of this ticket, because the bug exists for any sources different than pypi (e.g. HTTP/HTTPS).

Comment by Samuele Kaplun [ 2014/07/08 ]

Bumping the priority, because in our use case it potentially causes continuous unwanted restarting of services.

Comment by Alessio [ 2014/07/08 ]

I have this problem too.

Comment by Alessio [ 2014/07/17 ]

I found that specifying the package name everything works properly again.
e.g.
package

{"editdist": ensure => present, provider => 'pip', source => "https://py-editdist.googlecode.com/files/py-editdist-0.3.tar.gz", }

Can properly check with pip freeze that is installed.

This is a bit of workaround but we changed our configuration to something like this:
package

{ "HarvestingKit": ensure => present, provider => pip source => "https://github.com/inspirehep/harvesting-kit/tarball/e163844543a6f3cf04369975645659334dac144c", }

And now the pip provider will work again because pip freeze will show the "<package name>==<version>" instead of the url

Comment by Bryan Andrews [ 2016/04/20 ]

I thought I had this same problem but it turns out I was installing the package "cx_Oracle" with an underscore because that was the name found with pip3 search. Which also successfully installs it. However `pip3 freeze` was showing it as "cx-Oracle" with a dash. I changed the name of the package to match what `pip3 freeze` showed and everything is fine.

Comment by Mark McKinstry [ 2016/07/06 ]

Alessio,

With your workaround, if you ever want to uprade to a newer version, say the tarball from 6b311b0da17a57e2f2317ed161055b3b9bd1ab04, puppet won't upgrade it because the ensure just checks to see if any version is installed, not a specific version.

package{ "HarvestingKit": 
  ensure   => present, 
  provider => pip,
  source   => "https://github.com/inspirehep/harvesting-kit/tarball/e163844543a6f3cf04369975645659334dac144c",
}

Generated at Thu Nov 14 11:06:29 PST 2019 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.