imag/doc/src/02000-store.md

The Store

The store is where all the good things happen. The store is basically just a directory on the filesystem imag manages and keeps its state in.

One could say that the store is simply a databases, and it really is. We opted to go for plain text, though, as we believe that plain text is the only sane way to do such a thing. A user should always be able to read her data without great effort and putting everything in a real database like sqlite or even postgresql would need a user to install additional software just to read his own data. We don't want that. Text is readable until the worlds end and we think it is therefor better to store the data in plain text.

The following sections describe the store and the file format we use to store data. One may skip the following sections, they are included for users who want to dig into the store with their editors.

File Format

The contents of the store are encoded in either UTF-8 or ASCII. Either way, a normal text editor (like vim or the other one) will always be sufficient to dog into the store and modify files. For simple viewing even a pager (like less) is sufficient.

Each entry in the store consists of two parts:

1. Header
2. Content

The following section describe their purpose.

Header Format

The header format is where imag stores its data. The header is an area at the top of every file which is seperated from the content part by three dashes (---). Between these three dashes there is structured data. imag uses TOML as data format for this structured data, because it fits best and the available TOML parser for the rust programming language is really good.

The header can contain any amount of data, but modules (see @sec:modules) are restricted in their way of altering the data.

So normally there are several sections in the header. One section ([imag]) is always present. It contains a version field, which tells imag which version this file was created with (the version information is also encoded in the filename, just in case things change in the future). It also contains a links field which is an Array of values. This links field is for linking (see @sec:thestore:linking) to other entries in the store.

Other sections are named like the modules which created them. Every module is allowed to store arbitrary data under its own section and a module may never read other sections than its own. This is not enforced by imag itself, though.

Content Format

The content is the part of the file where the user is free to enter any textual content. The content may be rendered as Markdown or other markup format for the users convenience. The store does never expect and specific markup and actually the markup implementation is not inside the very code of imag.

Technically it would be possible that the content part of a file is used to store binary data. We don't want this, though.

Example

An example for a file in the store follows.

---
[imag]
links = ["/home/user/more_kittens.mpeg"]
version = "0.2.0"

[note]
name = "foo"
---

This is an example text, written by the user.

File organization

The "Entries" are stored as files in the "Store", which is a directory the user has access to. The store may exist in the users Home-directory or any other directory the user has read-write-Access to.

Each module stores its data in an own subdirectory in the store. This is because we like to keep things ordered and clean, not because it is technically necessary.

We name the path to a file in the store "Store id" or "Storepath" and we often refer to it by using the store location as root. So if the store exists in /home/user/store/, a file with the storepath /example.file is (on the filesystem) located at /home/user/store/example.file.

A storepath contains predefined parts:

• The module name of the Module the Entry belongs to, as said above. This part is always a directory.
• The version (semantic versioning applies) of the module storing the entry. This part is a postfix to the filename.

The pattern for the storepath is

/<module name>/<optional sub-folders>/<file name>~<sem version>

So if a module named "example-module" with version "0.1.0" stores a file in the Store, the storepath for a file with the name "example" is "/example-module/example~0.1.0".

Any number of subdirectories may be used, so creating folder hierarchies is possible and valid. A file "example" for a module "module" in version "0.1.0" would be stored in sub-folders like this:

/module/some/sub/folder/example~0.1.0

For example, it is valid if these files exist at the same time:

• /foo/bar~0.2
• /foo/bar~1.3

It might not be sane, though.

To future-proof the system it is necessary to provide a way for modules to differentiate in their versions on the store level. Thus if a module wants to retrieve a file from the store it must at least accept files from it's current advertised version. It may accept older files and it may transform them and resubmit them in the newer version.

Store path links

Linking entries is version independent.

This means if an entry "a" from a module "A" gets written to the store, it may link to an entry "b" from a module "B", which is in version "0.1.0" at the moment. If the module "B" gets updated, it might update its entries in the store as well. The link from the "a" should never get invalid in this case, though it is not ensured by the core of imag itself.