[SERVER-377] "puppetserver gem" command doesn't work from behind a proxy server Created: 2015/02/17  Updated: 2018/09/18  Resolved: 2016/11/30

Status: Closed
Project: Puppet Server
Component/s: DOCS, Puppet Server
Affects Version/s: SERVER 1.0.8
Fix Version/s: SERVER 5.0.0

Type: Bug Priority: Normal
Reporter: Geoff Williams Assignee: Ruth Linehan
Resolution: Fixed Votes: 8
Labels: docs_reviewed, maintenance
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

puppet master accesses internet via a proxy server


Attachments: HTML File gem    
Issue Links:
Duplicate
Relates
relates to SERVER-262 `puppetserver gem env` does not work,... Closed
relates to SERVER-522 Temporarily undo `puppetserver gem en... Closed
relates to SERVER-584 Ability to specify arbitrary values f... Closed
relates to SERVER-156 No http client support for http_proxy... Closed
relates to SERVER-540 puppetserver 1.0.8-1puppetlabs1 no lo... Closed
relates to SERVER-297 Consolidate environment variable hand... Closed
Support
Template:
Epic Link: Improve handling of environment variables
Team: Enterprise in the Cloud
Sub-team: emerald
Story Points: 3
Sprint: Server Emerald 2015-05-13, Server Emerald 2015-07-08, Server Emerald 2015-07-22, WC 2016-11-16, WC 2016-11-30
CS Priority: Reviewed
Release Notes: New Feature
Release Notes Summary: (unsure whether this is a bug fix or new feature) `puppetserver gem` command now respects HTTP_PROXY, HTTPS_PROXY, NO_PROXY, http_proxy, https_proxy, and no_proxy environment variables. This means that puppetserver can now install gems when behind a proxy if these environment variables are set.
QA Contact: Erik Dasher

 Description   

Updated description: puppetserver gem command should use system defined proxies http_proxy, https_proxy, no_proxy, HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables if set and notify the user that it is doing so.

-----------------------------------------------------

Original Description:

Description

When a puppet master must access the internet via a proxy server, it is not possible to use the puppetserver gem command to install gems.

This command is required to be usable to install gems into the JVM for systems such as hiera-eyaml.

Expected result

puppetserver gem command should use system defined proxies http_proxy, https_proxy environment variables if set and notify the user that it is doing so.

Actual result

puppetserver gem install ... fails with a timeout error when attempting to install gems.

Workaround

http://docs.puppetlabs.com/pe/3.7/release_notes_known_issues.html#installing-gems-when-puppet-server-is-behind-a-proxy-requires-manual-download-of-gems

Take a backup of the file /opt/puppet/share/puppetserver/cli/apps/gem
Edit the file to supply JVM arguments to support http(s) proxies by adding the parameters

  • -Dhttps.proxyHost=XXX
  • -Dhttp.proxyHost=XXX
  • -Dhttp.proxyPort=XXX
  • -Dhttps.proxyPort=XXX

Suggested fix

puppetserver 'gem' script defined in /opt/puppet/share/puppetserver/cli/apps/gem should be aware of and re-export formatted (eg with awk) contents of environment variables http_proxy and https_proxy. When doing so it should notify the user by printing a message to the screen.

QA:

Risk Assessment: High
Probability: Medium Low
Severity: High



 Comments   
Comment by Geoff Williams [ 2015/02/17 ]

Example of in-place fix to /opt/puppet/share/puppetserver/cli/apps/gem:

#!/usr/bin/env bash
PROXY_PORT=8080
PROXY_HOST=myproxy.xyz.com
 
JVM_EXTRA=" -Dhttp.proxyHost=$PROXY_HOST -Dhttp.proxyPort=$PROXY_PORT -Dhttps.proxyHost=$PROXY_HOST -Dhttps.proxyPort=$PROXY_PORT "
 
"${JAVA_BIN}" ${JVM_EXTRA} \
    -cp "${INSTALL_DIR}/puppet-server-release.jar" \
   clojure.main -m puppetlabs.puppetserver.cli.gem \
    --config "${CONFIG}" -- "$@" 

This fix could be enhanced by splitting the http_proxy and https_proxy environment variables into the required components using awk so that they are automatically resolved without having to edit the file in the future.

Comment by Dimitri Tischenko [ 2015/03/18 ]

You could also consider supporting the proxy-related variables in puppet.conf.
Also please make sure to also support and test a scenario with authenticated proxies (the above example would not work for those cases).

Comment by Chris Price [ 2015/03/18 ]

Dimitri Tischenko thanks for the extra info. What sort of authentication mechanism are you using to connect to your proxy?

Comment by Dimitri Tischenko [ 2015/03/18 ]

I am using

http_proxy=http://$username:$password@$hostname:$port

note: $username and $password need to be urlencoded.

Alternatively, the puppet proxy config variables (https://docs.puppetlabs.com/references/latest/configuration.html) support

  • http_proxy_host
  • http_proxy_port
  • http_proxy_user
  • http_proxy_password
Comment by Chris Price [ 2015/03/18 ]

Great, thanks. Just wanted to make sure we were talking about pw auth and not something more complex, like cert-based. We'll make sure we get in a fix that includes pw auth.

In the meantime, I've found a few stack overflow articles that suggest that you could extend the workaround proposed in this ticket with:

-Dhttp.proxyUser=someUserName
-Dhttp.proxyPassword=somePassword

Would be interested in hearing whether that works for you, just as a workaround until we get a real fix in place.

Comment by Dimitri Tischenko [ 2015/03/18 ]

One more issue which crops up regularly is the proxy exclusion patterns (the no_proxy setting). How can that be configured?

Comment by Chris Price [ 2015/03/18 ]

I'm not familiar with that; is that a Ruby setting?

Comment by Dimitri Tischenko [ 2015/03/18 ]

No, that's actually a Linux-ism, see here: https://wiki.archlinux.org/index.php/proxy_settings
Different configuration scopes use a different syntax to describe proxy exceptions. Java world has http.nonProxyHosts, see http://docs.oracle.com/javase/7/docs/technotes/guides/net/proxies.html

Comment by Chris Price [ 2015/03/18 ]

Great, that's helpful info. Thanks.

Comment by Geoff Williams [ 2015/03/23 ]

Updated gem script file:

  • Parses proxy variables from puppet.conf
  • Allows puppet.conf variables to be overridden by environment variable http_proxy
  • No need to code the proxy information into the script file any more
  • Nice(er) drop-in replacement until this can be addressed system-wide
Comment by Geoff Williams [ 2015/03/23 ]

PR of above file: https://github.com/puppetlabs/puppet-server/pull/460

Comment by Geoff Williams [ 2015/03/24 ]

Dimitri Tischenko Unfortunately its not possible to pass proxy usernames and passwords that JVM will automatically use via JVM environment variables. There is (Java/Clojure) coding effort required to get this to work so the fix outlined here won't work for you, you need support in-product

Comment by Jeremy Barlow [ 2015/04/09 ]

As a historical note...

Per some work that was done in SERVER-262, we, for the Puppet Server 1.0.2 release only, had enabled many environment variables to flow through from the shell environment to the running Ruby code. For only Puppet Server 1.0.2, then, the http_proxy and https_proxy environment variables would have been honored for gem installs.

SERVER-262 was later reverted for work done in SERVER-522. The original behavior which disallowed environment variables to flow through would have been restored in Puppet Server 1.0.8. Note that no PE releases have used Puppet Server 1.0.2.

Comment by Jeremy Barlow [ 2015/07/01 ]

I spent some time playing around with different proxy parameters and methods for specifying those parameters. I wanted to take the current environment variable filtering question out of the equation, so I just ran gem through jruby from the command line, like:

jruby -e 'load "<jruby_home_dir>/bin/gem" list nokogiri --remote --all

I setup a local squid proxy via SquidMan, http://squidman.net/squidman/, on my Mac, enabling proxy authentication so that I could see the effect when manipulating the corresponding authentication parameters when performing gem commands.

Here are some observations:

1) The "http*" proxy environment variables (e.g., http_proxy) supersede any corresponding values that may have been set via the Java system properties (e.g., http.proxyHost and http.proxyPort).

2) I couldn’t get any requests to actually use the http.proxyUser, http.proxyPassword, or http.nonProxyHosts values. For the http.proxyUser or http.proxyPassword to be used, maybe it is necessary to set a default global Authenticator in code like is mentioned in the stackoverflow article that Geoff Williams mentioned earlier, http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java, not sure. For http.nonProxyHosts, I don’t have a good explanation, although it does seem that others have encountered similar problems - e.g., http://stackoverflow.com/questions/17902830/jvm-arguments-for-https-nonproxyhosts. Maybe it’s just a case of “Just doesn’t work” on my Mac.

3) I didn’t run into any problems / unexpected behaviors with respect to how rubygems utilizes the different proxy-related environment variables - http(s)_proxy, http(s)_proxy_user, http(s)_proxy_pass, or no_proxy. There is an order of precedence that rubygems follows for these options which seems sane to me. This includes:

a) If the target of the request is an “https” url, rubygems will look for an https_* variable first. If it doesn’t find one, it falls back to a http_* variable. If the target of the request is an “http” url, rubygems will only look for an http_* variable.

b) Rubygems will look for an environment variable in all lowercase first, e.g., http_proxy. If that is not found, rubygems will fall back to looking for an all uppercase variable, e.g., HTTP_PROXY.

c) For the proxy username and password to use in the HTTP Proxy-Authorization header for the request, rubygems will look for a username and password in the URL for the http(s)_proxy first, falling back to any values in http(s)_proxy_user and http(s)_proxy_pass if not found. So for example, if you had a value of “http://user1:pass1@127.0.0.1:8080” for http_proxy, “user2” for http_proxy_user, and “pass2” for http_proxy_pass, the Proxy-Authorization header would include the Base64-encoded form of “user1:pass1”. If you had a value of “http://127.0.0.1:8080” for http_proxy with the same values for http_proxy_user and http_proxy_pass, the Proxy-Authorization header would include the Base64-encoded form of “user2:pass2”.

We are also trying to move toward a world where Puppet Server no longer has a dependency on puppet.conf settings. So, I'd been imagining that we would expose the required proxy settings in the puppet server config files and then wire up the puppet server gem command to use those.
There are some mixed opinions about the various pros and cons about allowing environment variables through, so we are expecting to have a design meeting to cover SERVER-297 pretty soon, and the output of that conversation will have an impact on what we decide to do as the final solution for this one.

Where we ended up as a result of SERVER-297 is that we are still whitelisting the HOME and PATH environment variables. We set aside the broader conversation about possibly opening the door for whitelisting environment variables more generally, like for custom ones that might be specific to a particular gem, to SERVER-584.

I'm pretty sold at this point on the value of allowing the proxy-related variables flow all the way from the parent shell environment into Ruby code when running under JRuby - not just for gem operations but more generally for any Ruby code that would run under Puppet Server. This just seems like the right thing to do from a *nix tool perspective.

I'm also onboard with Chris Price' thinking that we should define a mechanism for proxy settings to be specified in Puppet Server’s configuration files and that any integration between “puppetserver gem” proxy configuration settings should be built on the Puppet Server configuration and not “puppet.conf”, as we’ll be more generally trying to migrate “master” operations away from “puppet.conf” dependencies over time.

Assuming we eventually get to a model where the proxy settings could be derived either from Puppet Server configuration settings or from shell environment variables, the order of precedence would need to be established. I’ve thought about this a fair amount - and changed my opinion multiple times in the process. At this point, I’d suggest that we go with having any shell environment variable that is defined override any corresponding setting which may be present in the Puppet Server configuration. This approach would seem most consistent with how *nix tools typically seem to work - including legacy Puppet (https://docs.puppetlabs.com/references/latest/configuration.html#httpproxyhost). Also, it would provide users with a straightforward way to set a one-off value for a gem install. For example, users might want to manage a standard group of proxy settings in configuration - for use by Puppet extensions in the production Puppet Server stack and for Puppet Server CLI commands - but, for just a single “gem install”, want to use a different proxy server. By defining a different value for that environment variable in the context of that one “gem install” invocation, the unique proxy value could be used once without affecting the “global” configuration for future commands.

For this ticket, I’d propose that we do the following:

1) Add the lowercase and uppercase variants of http_proxy, https_proxy, http_proxy_user, https_proxy_user, http_proxy_pass, https_proxy_pass, and no_proxy to the current environment variable whitelist used for the construction of JRuby ScriptingContainers.

2) Add support for an optional “proxy” section to the “http-client” section of “puppetserver.conf” - http://docs.puppetlabs.com/puppetserver/2.1/configuration.html#puppetserverconf.

Proposed Clojure schema:

(def ProxyCredentials
  {:user schema/Str
 :password schema/Str})
(def ProxyInfo
  {:host schema/Str

  (schema/optional-key :credentials) ProxyCredentials

  (schema/optional-key :no-proxy-hosts) [schema/Str]})

Example:

http-client: {
  proxy: {
    host: my-proxy-host
    port: 8080
    credentials: {
      user: proxyuser
      password: proxypassword
    }
    no-proxy-hosts: { “*.mydomain.com”, “*.myotherdomain.com” }
  }
}

3) For any of the following environment variables not set from the environment, inject a value from the “http-client” configuration section (if one is available) into the JRuby ScriptingContainer's environment.

  • HTTP_PROXY -> http://[proxy.host]:[proxy.port]
  • HTTP_PROXY_USER -> [proxy.credentials.user]
  • HTTP_PROXY_PASS -> [proxy.credentials.password]
  • NO_PROXY -> Flattened, comma-delimited form of [proxy.no-proxy-hosts].

Assuming no environment variables or "http-client" proxy configuration were available, rubygems should just fallback to using the Java system properties for proxy (where they work anyway), if specified, else no proxy at all.

Thoughts from others - Chris Price, in particular?

Comment by Jeff McCune [ 2015/07/01 ]

Thoughts from others

I'm +1 in that I've read the whole thing and have no qualms with anything proposed.

To bring this back around to Nan Liu's issue where he uses the Faraday library from a report processor, am I correct in my understanding that this proposal will support his use case by allowing the extension to read proxy settings from the JRuby ENV object? From the user perspective, they will be able to set the proxy settings either in the environment or in the puppetserver configuration file and those settings will flow through to the custom extension without the user or Nan (as the extension author) having to do anything special?

Comment by Jeremy Barlow [ 2015/07/01 ]

To bring this back around to Nan Liu's issue where he uses the Faraday library from a report processor, am I correct in my understanding that this proposal will support his use case by allowing the extension to read proxy settings from the JRuby ENV object?

Thanks, Jeff McCune. Glad you brought up a different example to consider.

Yeah, this should allow for any Ruby code - not just rubygems - that runs under Puppet Server to reference ENV['http_proxy*'] and get a value which would have been defined in the shell environment.

It looks like Faraday, unlike rubygems, only looks for an environment variable named http_proxy (lowercase) and not HTTP_PROXY (uppercase) and would only be able to derive the proxy user and password from whatever is in the http_proxy variable as it does not look at the http_proxy_user or http_proxy_password environment variables:

https://github.com/lostisland/faraday/blob/v0.9.1/lib/faraday/connection.rb#L84

Outside of rubygems, I actually haven't found a case yet for where http_proxy_user and http_proxy_password environment variables are used directly. The mapping I proposed for the "http-client" configuration variables onto the environment variables that could be seen in process by Ruby code probably doesn't make sense for more general purposes, then. Maybe a better mapping would be:

If we were to apply this configuration override across all Ruby execution paths under Puppet Server, this approach would take away a little of the flexibility that you'd have with how environment variables are used in rubygems. It seems like this would still be flexible enough for general purposes, though, and consistency in definition across Ruby execution paths seems like it would lead to more predictable results for users.

I'd still be okay with having the http_proxy_user and http_proxy_pass shell environment variables be flowed through to Ruby code - just with the caveat that those could be trumped via the population of credentials into the http_proxy variable from configuration file settings, depending upon specific Ruby extension's use of those variables. Although maybe if those are truly only rubygems specific, it wouldn't be worth whitelisting those. Thoughts?

Comment by Jeff McCune [ 2015/07/02 ]

I'd still be okay with having the http_proxy_user and http_proxy_pass shell environment variables be flowed through to Ruby code - just with the caveat that those could be trumped via the population of credentials into the http_proxy variable from configuration file settings, depending upon specific Ruby extension's use of those variables. Although maybe if those are truly only rubygems specific, it wouldn't be worth whitelisting those. Thoughts?

I vote we just ignore http_proxy_user and http_proxy_pass and not include them in the whitelist at all. They'll be relatively easy to add if there's a compelling reason to do so, but I think the answer of "just put the username and password in the http_proxy variable to make that work will be acceptable to nearly everyone.

I also really like the new mapping of lower case variables instead of upper case variables. In my opinion the lower case form is more conventional for proxy configuration (as we observed with Faraday) than the upper case form, even though environment variables are usually upper case.

Thoughts, particularly on whether or not we should simply ignore the upper case forms or not?

Comment by Jeremy Barlow [ 2015/07/02 ]

Thanks, Jeff McCune.

I vote we just ignore http_proxy_user and http_proxy_pass and not include them in the whitelist at all. They'll be relatively easy to add if there's a compelling reason to do so, but I think the answer of "just put the username and password in the http_proxy variable to make that work will be acceptable to nearly everyone.

I'm good with that. Can revisit later if we need to.

I also really like the new mapping of lower case variables instead of upper case variables. In my opinion the lower case form is more conventional for proxy configuration (as we observed with Faraday) than the upper case form, even though environment variables are usually upper case.

Yeah, I don't have a good feel for that. Anecdotally, I've seen various references to both all uppercase and all lowercase - fortunately never mixed case - in google searches. About the proxy variables specifically, https://wiki.archlinux.org/index.php/Proxy_settings says:

Some programs look for the all caps version of the environment variables.

For programs that support both, it does generally seem like a lowercase version is given precedence - e.g., rubygems as mentioned earlier, curl (http://curl.haxx.se/docs/manpage.html). I also have to wonder, though, that since many programs seem to support both formats about the likelihood of users having systems that use the uppercase format - especially, as you mentioned, since it seems conventional for environment variables more generally to be in uppercase.

With respect to our whitelist into the JRuby ScriptingContainer, I'd be fine with just playing dumb and letting either an all uppercase or all lowercase version of any variable that we want to whitelist come through. Then we wouldn't have to get into guessing games about what third-party Ruby libraries might have only had support for reading an uppercase form of the variable and what-not.

With respect to the mapping from any specified "http-client" configuration settings, I'd also be fine with mapping onto an uppercase and lowercase form of the variable, i.e.:

What do you think?

Comment by Jeff McCune [ 2015/07/02 ]

What do you think?

+1, sounds close to ideal. Only detailed-level question remaining in my mind is if mapping http proxy on top of HTTPS_PROXY and https_proxy could cause an issue for someone who wants to use two different proxies for the two different protocols.

Comment by Chris Price [ 2015/07/03 ]

Just read this. Sounds OK to me. My only question is: if this is just intended to solve the 'gem' and http proxy issues, then it doesn't solve the general issue around allowing users to opt-in to additional environment variables. I think that's fine, I do think we should treat that as two discrete sets of work (especially since I believe that the clj-http-client code will need to be modified in order to support what you're describing here, right?).

It might be worth mapping out the plan for the more general environment variable issue a little further, though, just to make 100% sure that it will fit in nicely with your proposal here.

I don't mind deferring to you guys' judgment on that, though. This proposal sounds fine for this ticket.

Comment by Jeremy Barlow [ 2015/07/09 ]

Only detailed-level question remaining in my mind is if mapping http proxy on top of HTTPS_PROXY and https_proxy could cause an issue for someone who wants to use two different proxies for the two different protocols.

I was only thinking that the mapping would be done for cases where the environment variable were not already set. So if someone had set the https_proxy and/or HTTPS_PROXY variables in the shell environment, the proxy specified in the configuration file would not override those.

This would raise the question as well, though, about whether we should have a scheme-specific distinction in the proxy configuration file. So, rather than my relatively simpler example from earlier, you might be able to do something like:

http-client: {
  proxy: {
    http: {
      host: my-http-proxy-host
      port: 8080
      credentials: {
        user: httpproxyuser
        password: httpproxypassword
      }
    }
    https: {
      host: my-https-proxy-host
      port: 8080
      credentials: {
        user: httpsproxyuser
        password: httpsproxypassword
      }
    }
    no-proxy-hosts: { “*.mydomain.com”, “*.myotherdomain.com” }
  }
}

Given that it seems like we've survived without having to make that distinction in the legacy Puppet settings for proxy so far, though, I hesitate to add that level of complexity to the schema at this point. Also, the inconsistent support for even referencing the scheme-specific settings among tools/libraries, e.g., faraday not even trying to look at https_proxy, makes me even less inclined to try to do some sort of mapping for it. Hoping we could just add that distinction later when/if it is needed. Having the flexibility to make that distinction via the shell environment variables, if really needed, might be enough for now.

I'd also be good with just having the override from proxy configuration be applied to the http_proxy and HTTP_PROXY variables, when not set in the shell environment, and forego any mapping over the https_proxy and/or HTTPS_PROXY variables. Sound sane?

Comment by Jeremy Barlow [ 2015/07/09 ]

My only question is: if this is just intended to solve the 'gem' and http proxy issues, then it doesn't solve the general issue around allowing users to opt-in to additional environment variables. I think that's fine, I do think we should treat that as two discrete sets of work (especially since I believe that the clj-http-client code will need to be modified in order to support what you're describing here, right?).

Yeah, what I had in mind for this specific ticket was just to provide a solution for the http client proxy issues for puppetserver gem and other pure-Ruby extension points running in Puppet Server. This wouldn't cover proxy in the context of HTTP client calls that Puppet Server would make via com.puppetlabs.puppetserver.ExecutionStubImpl, which go back through our Ruby/Java Interop layer. For the latter, work I'd been thinking we'd do around SERVER-156, I'd been thinking we'd need to do a Java / clj-http-client equivalent implementation - hopefully building upon the configuration setting approach that we lay out in this ticket.

It might be worth mapping out the plan for the more general environment variable issue a little further, though, just to make 100% sure that it will fit in nicely with your proposal here.

Yeah, that's a fair point. The approach I have in mind for the more general environment variable issue, work that would be done for SERVER-584, would be compatible with the changes that we would be starting with here. I'll take a cut at describing my thoughts on that one and see if we can reach some level of consensus before we get too deep into implementation toward the approach we've been talking about for this one.

Comment by Chris Price [ 2015/07/09 ]

Yeah, what I had in mind for this specific ticket was just to provide a solution for the http client proxy issues for puppetserver gem and other pure-Ruby extension points running in Puppet Server. This wouldn't cover proxy in the context of HTTP client calls that Puppet Server would make via com.puppetlabs.puppetserver.ExecutionStubImpl, which go back through our Ruby/Java Interop layer. For the latter, work I'd been thinking we'd do around SERVER-156, I'd been thinking we'd need to do a Java / clj-http-client equivalent implementation - hopefully building upon the configuration setting approach that we lay out in this ticket.

Ideally we could just re-use the settings that you're defining in this ticket, right?

Yeah, that's a fair point. The approach I have in mind for the more general environment variable issue, work that would be done for SERVER-584, would be compatible with the changes that we would be starting with here. I'll take a cut at describing my thoughts on that one and see if we can reach some level of consensus before we get too deep into implementation toward the approach we've been talking about for this one.

In terms of order of operations, it's up to you how far you want to get into the weeds of writing up notes and debating the implementation details for SERVER-584 right now. As long as you've thought it through enough to where you feel like the UX (in terms of config files) is going to feel reasonably cohesive, and that the new sections of settings for the two things aren't going to conflict with one another I'm happy.

Comment by Jeremy Barlow [ 2015/07/17 ]

This would raise the question as well, though, about whether we should have a scheme-specific distinction in the proxy configuration file. ... Given that it seems like we've survived without having to make that distinction in the legacy Puppet settings for proxy so far, though, I hesitate to add that level of complexity to the schema at this point.

So after spending some time looking at how Apache / Java handles proxy settings (documented in SERVER-156), I've changed my mind on this point. In particular it appears that the default Java ProxySelector is pretty rigid about only using the <scheme>.proxyHost and <scheme>.proxyPort associated with the scheme in the target URI for a request - see http://www.docjar.com/html/api/sun/net/spi/DefaultProxySelector.java.html. So if your target URI has a scheme of "https", only the https.proxyHost and https.proxyPort settings would be used. If neither of those were defined but the http.proxyHost and https.proxyPort were, the ProxySelector would not fall back to using the http.* ones for a target URI having a scheme of "https".

Since the scheme distinction seems to be common with respect to environment variables and Java system properties, I think it would be better for us to just have our proxy configuration allow for that distinction as well. So, that could look like the example I'd listed earlier:

http-client: {
  proxy: {
    http: {
      host: my-http-proxy-host
      port: 8080
      credentials: {
        user: httpproxyuser
        password: httpproxypassword
      }
    }
    https: {
      host: my-https-proxy-host
      port: 8080
      credentials: {
        user: httpsproxyuser
        password: httpsproxypassword
      }
    }
    no-proxy-hosts: { “*.mydomain.com”, “*.myotherdomain.com” }
  }
}

I'd propose we only try to map configuration for http.* to http.* and https.* to https.* with respect to environment variables – and later, for SERVER-156, to proxy option setting in the client libraries – and not try to get into substituting any app-layer https to http fallbacks. We can just let the various utility libraries take care of that sort of fallback mapping however they already do.

Jeff McCune - what do you think?

Comment by Jeremy Barlow [ 2015/07/17 ]

Ideally we could just re-use the settings that you're defining in this ticket, right?

Definitely. That's why I'm hoping to get the design for this right so that we can use the same in both places.

In terms of order of operations, it's up to you how far you want to get into the weeds of writing up notes and debating the implementation details for SERVER-584 right now. As long as you've thought it through enough to where you feel like the UX (in terms of config files) is going to feel reasonably cohesive, and that the new sections of settings for the two things aren't going to conflict with one another I'm happy.

We have a separate conversation going on SERVER-584 now. I'm hoping that we're pretty close to making a decision on that. Given that we have a couple of directions proposed there that would affect the way we'd express the environment variable overrides for this ticket, I'm inclined to just try to let SERVER-584 be completed first before trying to continue forward on the development for this one.

Comment by Anoop Kumar [ 2015/08/12 ]

I hit this today in PE 3.8.2 on SLES 11, I ended up using the '-p' option in 'gem install' and it worked flawlessly. Is it already fixed in PE 3.8.2 or is this another workable option?

p, -​[no-]http-proxy [URL] - Use HTTP proxy for remote operations

Comment by Jeremy Barlow [ 2016/11/29 ]

Merged to puppetserver#master at 7cb154a.

Comment by Ruth Linehan [ 2016/11/30 ]

Acceptance tests for this passed in https://jenkins-master-prod-1.delivery.puppetlabs.net/view/puppetserver/view/all/job/platform_puppetserver_integration-system_no-conditional_full-master/42/.

Comment by Jeff McCune [ 2017/03/07 ]

For posterity the work-around example looks like so:

puppetserver gem install -p http://webproxy.acme.com:8080 hiera-eyaml

Comment by Gary Larizza [ 2017/05/03 ]

Jeff McCune Posterity thanks you for that!

Comment by Joshua McClintock [ 2018/03/27 ]

I have a custom function which is using 'net/http' and I want it to inherit the line 'http_proxy=<some value>' from /etc/sysconfig/pe-puppetserver, but it doesn't seem to work.  Is there a way to make this pass through so my custom function can see it?

Comment by Melissa Amos [ 2018/09/18 ]

Removed from PE known issues in a long-overdue cleanup.

Generated at Sat Dec 14 15:16:59 PST 2019 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.