Define PCore - the Puppet Meta Meta Model

The PCore model is the puppet meta meta model which will be used to describe all puppet meta models.

If your head is spinning:

• A meta model is like a schema e.g. the json-schema for a puppet catalog
• A meta meta model is like the jsonschema json-schema, that defines what you can define in a json-schema

In Puppet we now have a very rich Type System. PCore takes advantage of that and this allows us to define very compact models.
The first model that we express is the PCore meta meta model as it defines what follows. The first real model we aim to express using PCore
is the AST model.

We are doing this because existing modeling Ecore, jsonschema, yaml-schema, etc. are based on assumptions about the target language (java, json, yaml) and thus contain features suitable for each of those technologies (and naturally also lacking features). While it is possible to model anything with those, it requires yet another abstraction layer since the basic types available are not enough to describe the puppet types.

With our own model we require fewer abstraction layers. What we process is already typed by the type system, and it should be just as easy to describe a model as it is to define a value's type. When something is typed by the type system, enough information is known to enable that the value is serialized. This means that objects do not have to support a variety of serialization protocols (to_yaml, to_json, to_msgpack, to_this, to_that). To work around that, in Puppet the protocol is now "to_data_hash" and "from_data_hash" and that step reduces the information to a hash of serializable values and that is in turned serialized with a to_yaml etc. When doing this we created another problem for us; this is a lossy transformation ! (This is one reason it is hard to serialize anything but the most rudimentary data types9).

The work consists of:

• Specifying PCore
• Implementing PCore in PCore
• Specifying how an instance model (a catalog, an AST, etc) + a PCore model is used to serialize and deserialize that model,
• It is expected that this work will spin off requirements for enhancements to the type system (Some already identified and worked on).

Other tickets will contain the more concrete things (like writing a serializer)

PCore is an enabler for many future features:

• partitioning results to make what is required in each communication to be smaller
• efficient sending of a machine state (the RAL) as a fact
• a more efficient replacement for plugin-sync
• describing resource types in puppet
• polyglot implementations (native resource types)
• functions on the agent (without distributing the pp source to all agents)
• and many more

This ticket is about none of the future things - only the basic processing needed to support the "Rich Data Types" needed for various purposes (binary data, encrypted data, rich resource attributes values in general), but with the required concepts in place for higher order functionality (or at least that the implementation does not block the higher order functionality by being so simplistic that it has to be immediately redone).

The design has to be as future proof as we can reasonably make it. As this sits at the core it is both hard and takes a long time to change when done wrong. We should rather err on the side of being too cautious and e.g. include extra meta information than skip it in the belief that what we implement first will last for ever).