[PUP-3524] Switch from YAML (et al) to JSON Created: 2014/10/23 Updated: 2018/03/07 Resolved: 2017/02/23
|Remaining Estimate:||Not Specified|
|Time Spent:||Not Specified|
|Original Estimate:||Not Specified|
|Epic Name:||JSON All The Things!|
I'd like to move us away from PSON and onto a standard format. YAML is out of the question because it is either slow and unsafe (all of the YAML vulnerabilities) or extremely slow and safe (safe_yaml). MessagePack might be nice. It is pretty well specified, has a fairly large number of libraries written for it, but it doesn't do much to help us solve the wild west of encoding in puppet. In MessagePack there aren't really any enforcements of string encodings and everything is treated as an array of bytes.
In order to keep consistency across various puppet projects we'll be going with JSON. JSON requires that everything is valid UTF-8, which gives us a nice deliberateness to handling data. JSON is pretty fast (not as fast as MessagePack) and there are a lot of libraries if it turns out that the built in json isn't fast enough (puppet-server could use jrjackson, for instance).
|Comment by Henrik Lindberg [ 2014/10/23 ]|
One note that I also sent out on puppet dev.
BTW: MsgPack is essentially binary-JSON with compression. We can easily support a msgpack/json format in addition to text/json. Where the text/json for file content is either a String or Hash, where the hash is
and the Msgpack is either String or a Hash with
Without any other changes, the only place we support the Binary is in File's content so that serialization/deserialization needs to be special. When we have typed parameters, the serialization is instead driven by the type.
|Comment by Andrew Parker [ 2014/10/27 ]|
I'm leaving out things like config files that puppet doesn't produce. This means things like the certificate extensions on an agent aren't on this list of items. I did create a ticket for the exec ENC, but it isn't something that really has to be done and we can decide to just close it "won't fix" with no detriment to other tickets.
|Comment by Chris Price [ 2015/01/27 ]|
After discussion with Kylo Ginsberg (and a bit with Henrik Lindberg as well, we decided to break this into two chunks of work. The ticket was formerly titled "Switch from YAML and PSON to JSON". We're going to break out the PSON/networking layer subtasks into a separate epic/story (
The plan for now is to have the Puppet Server team own the PSON epic in the run-up to Puppet 5.0, and the Client/Language teams would own the remaining tickets.
|Comment by Moses Mendoza [ 2016/12/05 ]|
|Comment by Henrik Lindberg [ 2016/12/06 ]|
Still high value - we are not ready to get this done for 5.0.0 even if we made progress; there is a Pcore implementation for JVM now that supports Pcore serialization over Json and MsgPack. Think we can make that an opt-in during the 5.x series. Changing version to 5.y