Keep the Yaml files in your project tidy is useful. Not only it improves code readability but also helps avoiding misunderstanding and bugs.
Luckily there are many tools available that helps with “linting” and formatting yaml file, I will introduce two of those tools in this blog post. I cover the issue of handling long strings thoroughly in another blog post titled: YAML Files – Formatting Long Strings.
You can install it using
pip install --user yamllint
Once installed, the tool can be invoked directly from the command line with:
The tool checks for the validity of syntax, additionally it also highlights multiple usage of the same key, as well as non-breaking formatting issues such as lines’ width, trailing spaces, indentation, spaces before the comments.
The output would look like the following example depending on the content of the file:
24:81 error line too long (152 > 80 characters) (line-length)
26:81 error line too long (156 > 80 characters) (line-length)
39:30 warning truthy value should be one of [false, true] (truthy)
52:81 error line too long (144 > 80 characters) (line-length)
The tool is well maintained and well documented, and the docs can be found under https://yamllint.readthedocs.io/en/stable/.
yamllint consistently, it is recommended to add a git hook that invokes yamllint before committing the code to the git repository.
One way to do that is to integrate yamllint with
pre-commit a took that magically add git pre-commit hooks to any project. https://pre-commit.com/ is a wonderful tool that I encourage all developers to use to ensure simple mistakes, and bugs are not introduced accidentally to the code. It can be used to enforce code formatting styles. Integrating yamllint with pre-commit is detailed here: https://yamllint.readthedocs.io/en/stable/integration.html.
I you are into automation like me, you would want to know that yamllint exits with 0 when no errors nor warnings have been found and exists with non-zero otherwise.
It is also important to mention that yamllint can be imported as a python package and can be used programmatically to lint yaml files. In fact I use this technique as part of my test-suite to ensure yaml files in a project are passing linting tests for example.
The output of yamllint can be used to direct manual formatting. Let’s take the following example:
1:1 error too many blank lines (1 > 0) (empty-lines)
2:1 warning missing document start "---" (document-start)
26:14 warning missing starting space in comment (comments)
26:13 warning missing starting space in comment (comments)
28:14 warning comment not indented like content (comments-indentation)
277:81 error line too long (90 > 80 characters) (line-length)
299:27 error no new line character at the end of file (new-line-at-end-of-file)
The first two issues indicates that the some_file.yaml does not starts with “—” but with an empty line. To fix this issue, we can replace the empty line with “—” (without the quote). The next warnings concentrates on comments, missing starting space in comment which means the comment token # and the actual comment string after it are not separated by a space. To fix the issue we could manually introduce the space. If we fix the indentation of comments to match the indentation of the content, then we resolve the next warning related to comments at 28:14.
The error in 277:81 is stating that the line is simply too long for the rules set to the yamllint where the limit is 80 characters. This issue is not trivial to fix and that’s why we have dedicated a section for it later in this post, however, the general idea is to break the long string over several lines without introducing newline characters where those are not needed.
The last error can be fixed by adding a new line at the very end of the document.
Manual formatting works. What would be nice if yamllint would fix these issues as they are detected. Like
black or other similar tools. Sadly this is not the plan for yamllint.
Luckily, someone thought of this issue already and wrote a hook that integrates with pre-commit and formats broken yaml files. The open-source pre-commit hook is called
yamlfmt and can be found here: https://github.com/jumanjihouse/pre-commit-hook-yamlfmt. To use hook add it to the .pre-commit file in combination with yamllint as explained in the documentation
- repo: https://github.com/adrienverge/yamllint.git
rev: v1.27.1 # or higher tag
- id: yamllint
args: [--format, parsable, --strict]
- repo: https://github.com/jumanjihouse/pre-commit-hook-yamlfmt
rev: 0.2.2 # or other specific tag
- id: yamlfmt
The auto-formatting hook can fix most issues highlighted by
yamllint but not all of them. Notably the long strings errors cannot be resolved with
yamlfmt hence the need for more on manual formatting of long strings.
There are other tools that focuses on fixing yaml issues such as
yamlfix that can be found under https://github.com/lyz-code/yamlfix, and
yamlfixer that can be found under https://github.com/opt-nc/yamlfixer. I might alter or expand this post to include my experience with both tools once I have finished testing them thoroughly.
Handling long strings
Now let us focus on handling long strings. But what does the error actually means? In short it means there are more characters in one line that the rule defined by yamllint allows.
The default value is 80 characters. Any line that has more characters than 80, will trigger the error.
To learn more about handling long string, please refer to my other post: YAML Files – Formatting Long Strings.