Document merging and lookup paths¶
The basics¶
Documents (and sub-documents!) can reference other files. These files will be loaded first and then the documents are merged on top of them. This allows you to overwrite and merge configuration files.
The following examples will use the Parent
and Example
document types from the previous chapters.
Let’s say you have the following configuration file (type Parent
):
# fixtures/parent_with_ref.yml
parent:
$ref: /referenced-document
name: overwritten
direct:
int: 1234
map:
key_from_parent_with_ref:
this: is from parent_with_ref
This document contains a $ref
entry. When calling the resolve_and_merge_references()
on a YamlConfigDocument
Configcrunch looks for files with this relative
path in fhe array you provide it. Let’s say you have the following file (note the path):
# fixtures/repo/referenced-document.yml
parent:
name: this will be lost
direct:
this: foo
map:
key:
this: bar
list:
- entry1
- entry2
- entry3
You can tell Configcrunch to look in the directory fixtures/repo
while trying to load parent_with_ref.yml
.
Configcrunch will notice the $ref
key, look for referenced-document.yml
and merge the two files:
>>> document = Parent.from_yaml("fixtures/parent_with_ref.yml")
>>> document.resolve_and_merge_references(["./fixtures/repo"])
Parent(...)
>>> document.freeze()
>>> print(document['name'])
overwritten
>>> print(document['direct']['this'])
foo
>>> print(document['direct']['int'])
1234
The list passed to resolve_and_merge_references()
is a list of lookup paths.
As you can see the resulting document is a combination of the two documents. All values in referenced-document.yml
were replaced with values from parent_with_refs.yml
. This also spans sub-documents.
The resulting document is:
parent:
name: overwritten
direct:
this: foo
int: 1234
map:
key:
$name: key
this: bar
key_from_parent_with_ref:
this: is from parent_with_ref
list:
- entry1
- entry2
- entry3
Chaining references¶
You can chain $ref-Entries. If a document is $ref’erenced this document can contain a $ref-entry as well. It’s $ref reference will be processed first.
Documents that are referenced can reference other documents relative to their position within the reference paths.
Example document in lookup paths:
# fixtures/repo/referenced-document.yml
parent:
$ref: ./referenced-document
map:
key2:
this: bar2
Example document that loads this document:
parent:
$ref: /referenced-document-with-reference
name: overwritten
Example after merge:
parent:
name: overwritten
direct:
this: foo
map:
key:
$name: key
this: bar
key2:
$name: key2
this: bar2
list:
- entry1
- entry2
- entry3
References in sub-documents¶
Sub documents can also contain $ref
entries. They will be merged as expected. For this to work
you only need to set up sub-documents as explained in the last chapter.
You place these documents to include also in the lookup paths:
# fixtures/repo/examples/referenced.yml
example:
this: can be changed
int: 1
You can then use it with $ref
-entries:
parent:
$ref: /referenced-document
name: overwritten
direct:
$ref: /examples/referenced
this: was changed
This will result in the following document being merged:
parent:
name: overwritten
direct:
this: foo
int: 1234
map:
key:
$name: key
this: bar
key_from_parent_with_ref:
this: is from parent_with_ref
list:
- entry1
- entry2
- entry3
Warning
Sub-documents are loaded after parent documents are merged, so $ref-Entries in sub-documents can be overwritten if they are present in referenced parent documents. They are NOT chained in this case.
As an example, imagine parent_with_ref.yml
and referenced-document.yml
would both
contain a $ref
-entry for the sub-document under direct. Only the $ref
-entry from parent_with_ref.yml
will be processed.
Multiple lookup paths¶
You can provide multiple lookup paths.
If a document is found in multiple lookup paths, the documents will be processed in the order of the entries in the lookup paths list. They will then be merged as explained under “Chaining references”.
This allows a lookup path to “override” another. The first lookup path is the base lookup path and the documents in the other lookup paths can extend and change definitions in the lookup paths that come before them.
Removing entries¶
During merging, a document can remove something from a referenced document with the special
$remove
keyword.
To remove string entries from lists, add an entry to the list which has the original value and prefix it with “$remove::”. Only removing strings from lists is supported.
Example document:
# fixtures/parent_with_ref.yml
parent:
$ref: /referenced-document
name: overwritten
direct:
int: 1234
map:
key: $remove
key_from_parent_with_ref:
this: is from parent_with_ref
list:
- "$remove::entry2"
Merge result:
parent:
name: overwritten
direct:
this: foo
int: 1234
map:
key_from_parent_with_ref:
this: is from parent_with_ref
list:
- entry1
- entry3