doc: Rewrite "The Store" chapter
This commit is contained in:
parent
28a9c5ffc9
commit
7a8432123b
1 changed files with 86 additions and 116 deletions
|
|
@ -1,64 +1,67 @@
|
|||
# The Store {#sec:thestore}
|
||||
|
||||
The store is where all the good things happen.
|
||||
The store is basically just a directory on the filesystem imag manages and keeps
|
||||
its state in.
|
||||
|
||||
One could say that the store is simply a databases, and it really is. We opted
|
||||
to go for plain text, though, as we believe that plain text is the only sane way
|
||||
to do such a thing.
|
||||
A user should always be able to read her data without great effort and putting
|
||||
everything in a _real_ database like sqlite or even postgresql would need a user
|
||||
to install additional software just to read his own data. We don't want that.
|
||||
Text is readable until the worlds end and we think it is therefor better to
|
||||
store the data in plain text.
|
||||
|
||||
The following sections describe the store and the file format we use to store
|
||||
data. One may skip the following sections, they are included for users who want
|
||||
to dig into the store with their editors.
|
||||
|
||||
## File Format {#sec:thestore:fileformat}
|
||||
|
||||
The content in the store MUST BE encoded in either Unicode UTF-8 or ASCII.
|
||||
Each "Entry" (File) MUST HAVE a "Header" component as well as a "Content"
|
||||
component.
|
||||
Each "Entry" in the store MUST start with three single dashes ("-") followed
|
||||
by a newline character, named "initial marker" in the following chapter.
|
||||
The Header follows the initial marker (@sec:thestore:fileformat:header).
|
||||
The Header MUST BE followed by a line which contains three single dashes ("-")
|
||||
and a newline character, called "header-close marker" in the following
|
||||
chapter.
|
||||
The content follows the header-close marker (@sec:thestore:fileformat:content).
|
||||
The contents of the store are encoded in either UTF-8 or ASCII. Either way, a
|
||||
normal text editor (like `vim` or the other one) will always be sufficient to
|
||||
dog into the store and modify files. For simple viewing even a pager (like
|
||||
`less`) is sufficient.
|
||||
|
||||
Each entry in the store consists of two parts:
|
||||
|
||||
1. Header
|
||||
1. Content
|
||||
|
||||
The following section describe their purpose.
|
||||
|
||||
### Header Format {#sec:thestore:fileformat:header}
|
||||
|
||||
The header format MUST BE "TOML".
|
||||
The sections which MAY or MUST be in the header are defined in the following
|
||||
chapters.
|
||||
The header format is where imag stores its data. The header is an area at the
|
||||
top of every file which is seperated from the content part by three dashes
|
||||
(`---`). Between these three dashes there is structured data. imag uses `TOML`
|
||||
as data format for this structured data, because it fits best and the available
|
||||
`TOML` parser for the rust programming language is really good.
|
||||
|
||||
#### Header section: "imag" {#sec:thestore:fileformat:header:imag}
|
||||
The header can contain any amount of data, but modules (see @sec:modules) are
|
||||
restricted in their way of altering the data.
|
||||
|
||||
The header MUST contain a section called "imag", where the automatically by the
|
||||
program generated data is stored in.
|
||||
The contents of this section is edited via commandline calls or by the
|
||||
program implicitely and SHOULD NOT be edited by the user.
|
||||
So normally there are several sections in the header. One section (`[imag]`) is
|
||||
always present. It contains a `version` field, which tells imag which version
|
||||
this file was created with (the version information is _also_ encoded in the
|
||||
filename, just in case things change in the future). It also contains a `links`
|
||||
field which is an Array of values. This `links` field is for linking (see
|
||||
@sec:thestore:linking) to other entries in the store.
|
||||
|
||||
This "imag" section MUST contain the following keys
|
||||
|
||||
1. A "version" Key. The version stored here is the version of the Store, the
|
||||
Entry was created with.
|
||||
|
||||
The "imag" section MAY contain
|
||||
|
||||
1. A section "imag.links" where a module is allowed to store URIs in a flat
|
||||
list
|
||||
1. A section "imag.content", used for referring to external content.
|
||||
The key "uri" is the only one which is required, it refers to external
|
||||
content.
|
||||
An explicitely suggested key is "file" for referring to a _local Mirror_ of
|
||||
the content.
|
||||
|
||||
#### Header section: "custom" {#sec:thestore:fileformat:header:custom}
|
||||
|
||||
The header MAY contain a section named "custom".
|
||||
The user is free to store arbitrary data here.
|
||||
The user is also free to edit this section by either commandline or editor.
|
||||
|
||||
#### Module Header section {#sec:thestore:fileformat:header:module}
|
||||
|
||||
The header MAY contain a section named after a module.
|
||||
The corrosponding module is allowed to store arbitrary data in this section.
|
||||
Other sections are named like the modules which created them. Every module is
|
||||
allowed to store arbitrary data under its own section and a module may never
|
||||
read other sections than its own. This is not enforced by imag itself, though.
|
||||
|
||||
### Content Format {#sec:thestore:fileformat:content}
|
||||
|
||||
The content is the part of the file where the user is free to enter any
|
||||
textual content.
|
||||
The content MAY BE rendered as Markdown or other markup format for the users
|
||||
convenience.
|
||||
The Store library MUST NOT expect any particular markup format.
|
||||
The content is the part of the file where the user is free to enter any textual
|
||||
content. The content may be rendered as Markdown or other markup format for the
|
||||
users convenience. The store does never expect and specific markup and actually
|
||||
the markup implementation is not inside the very code of imag.
|
||||
|
||||
Technically it would be possible that the content part of a file is used to
|
||||
store binary data. We don't want this, though.
|
||||
|
||||
### Example {#sec:thestore:fileformat:example}
|
||||
|
||||
|
|
@ -69,17 +72,10 @@ An example for a file in the store follows.
|
|||
[imag]
|
||||
version = "0.1.0"
|
||||
|
||||
[imag.content]
|
||||
url = "file://home/user/kittens.mpeg"
|
||||
|
||||
imag.links = [
|
||||
"imag://home/user/more_kittens.mpeg"
|
||||
]
|
||||
imag.links = ["/home/user/more_kittens.mpeg"]
|
||||
|
||||
[examplemodule]
|
||||
arbitrary = data
|
||||
[custom]
|
||||
truth = 42
|
||||
arbitrary = "data"
|
||||
---
|
||||
|
||||
This is an example text, written by the user.
|
||||
|
|
@ -89,23 +85,25 @@ This is an example text, written by the user.
|
|||
## File organization {#sec:thestore:fileorganization}
|
||||
|
||||
The "Entries" are stored as files in the "Store", which is a directory the
|
||||
user has access to.
|
||||
The store MAY exist in the users Home-directory or any other directory the
|
||||
user has Read-Write-Access to.
|
||||
user has access to. The store may exist in the users Home-directory or any
|
||||
other directory the user has read-write-Access to.
|
||||
|
||||
The Path of each File is shown as absolute path in this paper, while the root
|
||||
is always the store directory.
|
||||
This Path is named "Storepath".
|
||||
So if the store exists in `/home/user/store/`, a file with the Storepath
|
||||
Each module stores its data in an own subdirectory in the store. This is because
|
||||
we like to keep things ordered and clean, not because it is technically
|
||||
necessary.
|
||||
|
||||
We name the path to a file in the store "Store id" or "Storepath" and we often
|
||||
refer to it by using the store location as root.
|
||||
So if the store exists in `/home/user/store/`, a file with the storepath
|
||||
`/example.file` is (on the filesystem) located at
|
||||
`/home/user/store/example.file`.
|
||||
|
||||
A Storepath contains predefined parts:
|
||||
A storepath contains predefined parts:
|
||||
|
||||
* The module name of the Module the Entry belongs to.
|
||||
This part is a directory.
|
||||
* The version (semantic versioning applies) of the module storing the Entry
|
||||
This part is a postfix to the filename
|
||||
* The module name of the Module the Entry belongs to, as said above.
|
||||
This part is always a directory.
|
||||
* The version (semantic versioning applies) of the module storing the entry.
|
||||
This part is a postfix to the filename.
|
||||
|
||||
The pattern for the storepath is
|
||||
|
||||
|
|
@ -113,66 +111,38 @@ The pattern for the storepath is
|
|||
/<module name>/<optional sub-folders>/<file name>~<sem version>
|
||||
```
|
||||
|
||||
So if a Module named "ExampleModule" with version "0.1" stores a file in the
|
||||
Store, the Storepath for a file with the name "example" is
|
||||
"/ExampleModule/example~0.1".
|
||||
So if a module named "example-module" with version "0.1.0" stores a file in the
|
||||
Store, the storepath for a file with the name "example" is
|
||||
"/example-module/example~0.1.0".
|
||||
|
||||
Any number of subdirectories MAY BE used, so creating folder hierarchies is
|
||||
possible and valid.
|
||||
A file "example" for a module "module" in version "0.1" would be stored in
|
||||
sub-folders like this:
|
||||
Any number of subdirectories may be used, so creating folder hierarchies is
|
||||
possible and valid. A file "example" for a module "module" in version "0.1.0"
|
||||
would be stored in sub-folders like this:
|
||||
|
||||
```
|
||||
/module/some/sub/folder/example~0.1
|
||||
/module/some/sub/folder/example~0.1.0
|
||||
```
|
||||
|
||||
A given file MUST only exist once with the same path.
|
||||
For example, it is invalid if these files exist at the same time:
|
||||
For example, it is valid if these files exist at the same time:
|
||||
|
||||
* /foo/bar~0.2
|
||||
* /foo/bar~1.3
|
||||
|
||||
To future-proof the System it is thus necessary to create a disambiguation at
|
||||
the store level. Thus if a library wants to retrieve a file from the Store
|
||||
it MUST at least accept files from it's current advertised version. It MAY
|
||||
accept older files and it MAY transform them and resubmit them in the newer
|
||||
version.
|
||||
It might not be sane, though.
|
||||
|
||||
For this there will be an enum returned for each given Entry. It will have these
|
||||
members:
|
||||
|
||||
- `Compatible`, for version matches
|
||||
- `PossiblyIncompatible`, if the current version is at least a major number
|
||||
further
|
||||
- `Incompatible`, if the file is a at least a major number further
|
||||
To future-proof the system it is necessary to provide a way for modules to
|
||||
differentiate in their versions on the store level. Thus if a module wants to
|
||||
retrieve a file from the store it must at least accept files from it's current
|
||||
advertised version. It may accept older files and it may transform them and
|
||||
resubmit them in the newer version.
|
||||
|
||||
## Store path links {#sec:thestore:links}
|
||||
|
||||
Linking entries MUST BE version independent.
|
||||
Linking entries is version independent.
|
||||
|
||||
This means if an entry "a" from a module "A" gets written to the store, it may
|
||||
link to an entry "b" from a module "B", which is in version "0.1" at the moment.
|
||||
If the module "B" gets updated, it might update its entries in the store as
|
||||
well.
|
||||
The link from the "a" MUST NOT get invalid in this case.
|
||||
|
||||
This is accomplished by linking without the version number: So a link for the
|
||||
entry
|
||||
|
||||
```
|
||||
/module/some/sub/folder/example~0.1
|
||||
```
|
||||
|
||||
is
|
||||
|
||||
```
|
||||
imag://module/some/sub/folder/example
|
||||
```
|
||||
|
||||
As shown in the example, a link to imag-internal entries, the link is prefixed
|
||||
with a "imag://" identifier.
|
||||
This prefix is optional.
|
||||
A link to external content MUST NEVER be prefixed this way.
|
||||
The path of the internal link MUST NEVER be relative, but always absolute from
|
||||
the root directory of the store.
|
||||
link to an entry "b" from a module "B", which is in version "0.1.0" at the
|
||||
moment. If the module "B" gets updated, it might update its entries in the store
|
||||
as well. The link from the "a" should never get invalid in this case, though it
|
||||
is not ensured by the core of imag itself.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue