[CPR-379] Puppet server 4 not packaged properly (and not in Debian/OpenBSD) Created: 2016/09/12  Updated: 2017/12/28  Resolved: 2017/12/28

Status: Closed
Project: Community Package Repository
Component/s: None
Affects Version/s: None
Fix Version/s: 2017/08/02

Type: Bug Priority: Normal
Reporter: Kilian Krause Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Debian/Ubuntu or OpenBSD - probably all


Template:

Error rendering 'issue-templates-customfield'. Please contact your Jira administrators.

Team: Release Engineering

 Description   

When installing puppet server in an open source production environment, it'd be perfect to have proper distro packages and not have the puppetlabs repo as the only alternative.

For this, Debian recommends these rules (https://wiki.debian.org/UpstreamGuide) to upstreams to understand the requirements of a package build. In short the buildd is offline and taking a source and a transformation ruleset to (preferably reproducibly) build a binary package of this source. All that's available is either pre-installed in the build environment (by means of Build-Depends) or carried in the source.

Right now the documentation isn't allowing to see the required steps through since leiningen hides the dependency chain and "automatically" pulls requirements in from external sources and puts them into the jar directly. Thus https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=830904 is still open and no proper Debian package exists.

Once having a real distro package, it'd be preferable if the puppetlabs db would only ship the minimal update to the distro packages.

TIA!

Best,
Kilian



 Comments   
Comment by Chris Price [ 2016/09/12 ]

Michael Stahnke should this ticket go under "RE"?

Comment by Michael Stahnke [ 2016/09/13 ]

Hey, thanks for the ticket.

We don't control how the clojure build system works. (Leinigen). We use it, because that's the best tool for the job. Most linux distributions have never really been too great at packaging up things that run on the JVM due to their requirements. Those requirements make sense for the Distribution, but in the end make shipping something using a clojure pretty difficult. Something like puppetdb would require 300+ packages in Debian/Fedora world. Nobody wants to maintain that, plus we can't be sure all versions of all those libs always work with our software.

At this time, we think it's best for folks to get their puppet software directly from Puppet.

Comment by Chris Price [ 2016/09/15 ]

Kilian Krause one minor correction, for what it's worth: neither puppetserver nor puppetdb ever runs as root - not even during bootstrapping. I recognize that that does not address your other concerns, but it does mean that an attack vector based on code execution is significantly smaller than what you hinted at.

Beyond that, Michael Stahnke knows far more about all of this kind of thing than I do, so I defer to him.

Comment by Kilian Krause [ 2016/09/15 ]

Chris,

The point is not that puppetmaster itself may be exploitable within the host/environment it runs in (and potentially directly compromising it). It's about what the function of a puppetmaster is. It's implicitly root on all systems it manages (potentially including its own machine). Thus, if the module/manifest translation engine is successfully hacked, a puppetmaster will indeed instruct up to all of the servers that connect to it (i.e. every node that's not broken and periodically refreshing its state sync with the puppet agent).

It's pretty much the same point as deploying a continuous integration engine and not ensuring that this framework is the most secure building block of your architecture. Any attacker will start to focus there since it promises the maximum penetration/efficiency with just being one single target on one single platform. Ignoring that fact is plain igrnoring the reality of any real world attackers considerations.

In short: it's not so much about what a puppetserver is, it's about what a puppetserver does!

Best,
Kilian

Comment by Chris Price [ 2016/09/15 ]

Fair enough.

Comment by Michael Stahnke [ 2016/09/15 ]

There's nothing like a binary blob in pupeptserver. There is a jar with dependencies, but they are all open source components. Distros are welcome to repackage our software to fit within their guidelines. We don't package things that way because it's much much more effort for almost no benefit. It many cases, it's a hindrance since old version stick around in distros for a long time.

The source code is available. It's open source. The source for the dependencies is available.

I don't see how we, as an upstream, should be held accountable for how it gets into Distro FOO. I don't see GNU providing GCC for my raspberry Pi ARM device. I do see them putting up source code and I can build it myself. How is this different? It;'s different because Debian (and many other distros) have tooling to deal with C code, building it and packaging it. Debian (and many other distros) could choose to build tooling like that around clojure if they so desired. When golang came on the scene, many distros had to build/modify their tooling to allow for packages written in go. This is basically the same as when golang came on the scene, except that maintainers jumped right in to work on golang.

I sympathize with your position, as a long-time distro package maintainer. I wish there was tooling in distros for lots of things on the JVM. There isn't, and hasn't been for the last 20 years. Until Linux distributions want to change that, you won't see lots of things written scala, clojure, java, jruby, groovy, packaged up. You'll see some, but not complex projects. (For example the vast majority of JBoss software still isn't packaged for RHEL/Fedora properly even though RH owns them).

In short, all source is provided. You can build the jar on your own machine if you so desire. You can use our init scripts, or provide your own. I'm not sure what else we could do other than completely rewrite our software in another language, and that doesn't seem to be a good course of action for us.

Comment by Kilian Krause [ 2016/09/19 ]

Michael,

First, I wasn't holding you accountable, I was trying to see how a distro asking for help can be supported to have a clean, yet compliant package in their repo. The fact that you consider this package too old and dated isn't really how I would see things, but I'll just accept this for now. From my experience there can be a very good chance that a distro package is totally up-to-date and just as frictionless to install like the solution you already offer, if only upstream and the distro maintainer have found their packaging path and proper interface for fast and effective communication.

The part with the binary blob was just summarizing the fact that if you consider building the clojure from source too complex for a distro and that trusting a pre-built binary file (in a package format or not) is the recommendation for the installation to work well, this no longer adds much difference whether it's open source in the backend or not. If the build process and thus the reproducibility isn't verifyable (i.e. I cannot download the source and compile my own package with the same checksums) I'm effectively trusting the clojure just like the nvidia driver. Nothing else was my point.

Even if the reproducible build part isn't possible, I may still wish/hope/expect a distro maintainer to be a second pair of eyes to look over a release to not blow up my system. Usually this should never happen with a new upstream release, but it may just be a poilcy issue.

So, if you say the build process with the average distro isn't optimal, I'd like to hear, what your ideal build process would look like if it were following JVM/clojure rather than C like you say. For me the distro build process has always been flexible enough to accomodate any kind of build chain with whatever rules any upstream came up with - it was just a matter of tearing it apart and piecing it together to make it "single stepped" and thereby reproducible on your own infrastructure. Maybe I'm wrong here, but from my understanding it's a totally scripted configure&&build&&make install style mimic that's only waiting to be filled with the right commands (just like you would manually building things on the console). The only "difficult" part is to find out what automagic is happening in the background and pulling in depends live from the internet and how to disable that.

Rewriting the puppetserver in another language is surely an option - not a very realistic one I'd guess though. So, as Stig Sandbeck Mathisen already expressed, we'd like to tackle the point you may with distros not being good at packaging JVM/clojure style sources. Because the maintainers don't understand Java. They understand the regular C-style build toolchain. And we'd like to find out how this "other" toolbox works. Maybe you can just shed some light there and give us a kick off in the right direction to then stand up a proper package.

TIA!

Best,
Kilian

Comment by Michael Stahnke [ 2016/09/20 ]

Firstly, I appreciate the discussion. It's awesome that you want to help improve things. I'm traveling the next two weeks, so I won't have a ton of time to dig in, but I am not ignoring you.

How does packaging up things that use maven-based projects work for Debian? I think that's the closest analog I can think of.

Comment by Kilian Krause [ 2016/09/21 ]

Michael,

What would most probably get us closest is this:
https://wiki.debian.org/Java/MavenDebianHelper
https://wiki.debian.org/Java/MavenBuilder
https://wiki.debian.org/Java/Packaging
https://packages.debian.org/de/sid/maven-debian-helper
https://packages.debian.org/sid/maven-repo-helper

Not that I'd have worked with any of those before, but well, I guess I said that already. =)

Yet, just for my curiosity: I thought puppetserver doesn't use maven? Is that hidden somewhere below the leiningen/ezbake/rake stuff somewhere?

Cheers,
Kilian

Comment by Past Haus [ 2016/09/21 ]

Kilian Krause Leiningen uses maven under the hood to resolve clojure dependencies.

Generated at Sat Aug 08 17:45:37 PDT 2020 using Jira 8.5.2#805002-sha1:a66f9354b9e12ac788984e5d84669c903a370049.