[PUP-5914] data binding: a way to set the merge behaviour (like in the lookup() function) ? Created: 2016/02/17  Updated: 2016/02/22  Resolved: 2016/02/17

Status: Closed
Project: Puppet
Component/s: Language
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: Francois Lafont Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates PUP-5395 Set resolution_type when using data b... Closed
Template:

 Description   

Hi,

I use puppet 4 and data binding with much satisfaction. Just for memory, data binding for instance it's:

1. A manifest foo/manifests/init.pp like that in a foo module:

class foo (
    String[1]                  $param1,
    Hash[String[1], String[1]] $param2,
    Array[Data, 1]             $param3,
) {
    # The body of the class...
}

2. This boilerplate function foo/lib/puppet/bindings/foo/default.rb:

Puppet::Bindings.newbindings('foo::default') do
  bind {
    name         'foo'
    to           'function'
    in_multibind 'puppet::module_data'
  }
end

3. And a puppet function in foo/functions/data.pp:

function foo::data {
 
  # Code where $default_param1, $default_param2 and $default_param3 are set...
 
  {
    foo::param1: $default_param1,
    foo::param2: $default_param2,
    foo::param3: $default_param3,
  }
}

And now, in hiera, I can override the default value of a parameter like that:

---
# Here I just want to override the default value of param2.
foo::param2:
   key1: 'specific-value1'
   key2: 'specific-value2'

The wish

It's very useful but the merge behavior in the data binding mechanism is not changeable and it's necessarily the first behavior, ie the behavior where there is no merge (the first value found wins).

It could be amazing if it was possible to set the merge behavior like in the lookup() function. I have no idea how the syntax could be but the first way I see is something like that in the declaration of the foo class:

class foo (
           String[1]                  $param1,
    deep   Hash[String[1], String[1]] $param2,
    unique Array[Data, 1]             $param3,
) {
    # The body of the class...
}

The syntax will be

[<merge behavior>] [<data type>] $<variable-name>

where <merge behavior> will be a string like in the lookup() function, ie 'first', 'unique', 'hash' or 'deep' (cf here) etc. To keep compatibility with the "old" syntax:

    String[1] $param1,

will be equivalent to:

    first String[1] $param1,

I have absolutely no idea if the implementation of this feature will be possible and/or complicated (I'm unable to evaluate that and unable to implement that... unfortunately) but it seems to me that it could be a very useful feature and I'm ready to test an implementation of that when you want of course.

Of course, it's just a wish. No problem if this ticket is rejected.

Regards.
Fran├žois Lafont

PS: message edited (to just fix typo)



 Comments   
Comment by R.I.Pienaar [ 2016/02/17 ]

Francois Lafont sounds like you want the lookup_options stuff, see https://www.devco.net/archives/2016/02/03/puppet-4-data-lookup-strategies.php

Comment by Henrik Lindberg [ 2016/02/17 ]

The lookup_options functionality should address this. If that is not working for you when using a function to deliver the data, please open a ticket for that. Closing this as a duplicate.

Comment by Francois Lafont [ 2016/02/17 ]

To: R.I.Pienaar, thanks for your link. I have added the site in my bookmarks. Just a remark, in the common.yaml, you have put that:

lookup_options:
  users::local:
    strategy: deep
    merge_hash_arrays: true

But the correct yaml seems to be:

lookup_options:
  users::local:
    merge:
      strategy: deep
      merge_hash_arrays: true

or something like that.

To Henrik Lindberg: Ok thanks for your answer. Indeed, the lookup_options feature is what I want. That's cool.

During my tests, I have just noticed that the merge parameter of the lookup() function is not well taken into account when I use the hiera-eyaml backend but It's probably a problem which concerns hiera-eyam and not puppet directly. So I have made an issue here https://github.com/TomPoulton/hiera-eyaml/issues/186.

Thanks again.
Regards.

PS: message edited (typo)

Comment by Francois Lafont [ 2016/02/19 ]

Hi,

I'm sorry I have again a question concerning lookup_options. Do you know if it's possible to set the default merge policy of a specific parameter of a module, directly in the module itself when the module has "data_provider" = "function"?

According to the article of R.I.Pienaar it's possible for a module where "data_provider" = "hiera" but is it possible for a module where "data_provider" = "function"?

Comment by Francois Lafont [ 2016/02/19 ]

And what is the merge policy of lookup_options itself?

Comment by Henrik Lindberg [ 2016/02/19 ]

The choice of data_provider should not matter, the lookup_options feature sits in a layer above that.

Thomas Hallgren I don't recall where the merge behavior of the lookup_options key itself is specified. Can you help answer that question.

Comment by Francois Lafont [ 2016/02/19 ]

Henrik Lindberg ok but it seemed to me that the functions/data.pp function could only use keys qualified with the name of the module. Hence my question. But after your answer, I have made a quick test and I have realized that the lookup_options key is accepted in the functions/data.pp function. So I imagine this key is an exception.

So that's cool, thx.

Indeed I'm curious to know the merge policy of the very specific lookup_options key.

Comment by Henrik Lindberg [ 2016/02/19 ]

IIRC, lookup_options are merged, but if you specify a key, it's content wins over lower levels. This means that all keys that are specified at any layer contribute to the overall lookup_options, but specification at a higher level of the same key as at a lower level wins. (I may have missed some detail, but that is the principle).

Comment by Henrik Lindberg [ 2016/02/19 ]

And you are right lookup_options is an exception to the rule that you can only bind values in the module's name space. It is also a reserved key - you cannot use lookup to get the value of lookup_options.

Comment by Francois Lafont [ 2016/02/19 ]

Ok, so finally, for the lookup_options key itself, I don't see any difference between the merge you describe and the deep merge, unless I have missed something.

Edit: after thought, in fact it's not exactly a deep merge...

Anyway, now I have lot of personal modules to rewrite to use this excellent data binding mechanism (which I have well understood now I think). Thx again.

Generated at Thu Nov 14 01:50:32 PST 2019 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.