Commit graph

4084 commits

Author SHA1 Message Date
266311d743 Change backends to do less ser-/deserialization
This commit changes the backends to do less de/ser, as it now stores the
Entry objects in the backend and does the de/serialization there.

This means the store does only serialize things once from json to toml
in the io backend.

See the diff of the documentation for more details.
2017-06-18 12:32:04 +02:00
52011a59b2 Move serialize/deserialize calls of Entry to backend 2017-06-18 12:32:03 +02:00
cd99873f17 Merge pull request #972 from matthiasbeyer/libimagstore/io-backend
Libimagstore/io backend
2017-06-18 12:31:38 +02:00
3af1042593 Fix fs_to_write() to serialize headers correctly 2017-06-18 11:33:18 +02:00
c013ca8025 Outsource header/content parsing from store.rs to util.rs for reusability 2017-06-18 11:33:18 +02:00
91c427925b Make backend generic over Read/Write 2017-06-18 11:33:18 +02:00
73c1e79084 Implement high-level store test with IO backend 2017-06-18 11:33:18 +02:00
c4b2287876 Move the documentation to the docs 2017-06-18 11:33:18 +02:00
76dfe23f65 Add StdIoFileAbstraction module documentation 2017-06-18 11:33:18 +02:00
e5c8e9a1ac Add simple tests 2017-06-18 11:33:18 +02:00
aede9a112b Add first code draft for JSON mapper implementation 2017-06-18 11:33:17 +02:00
08f9eb3d83 Add new dependencies: Serde* 2017-06-18 11:31:55 +02:00
af2b629a86 Add VersionError kind 2017-06-18 11:31:55 +02:00
4abd9dd7cf Split into module-files 2017-06-18 11:31:55 +02:00
447f2610ef Make StdIo backend abstract over IO->"fs"->IO mapper 2017-06-18 11:31:55 +02:00
146e2a1140 Implement backend code for in-memory IO backend
This commit implements a backend which reads from a Read when created
and writes to a Write when dropped.

This way, one can initialize stores which are build from commandline
feeded JSON or TOML (currently JSON is implemented/about to be
implemented).
2017-06-18 11:31:55 +02:00
f487550f81 Merge pull request #971 from matthiasbeyer/libimagstore/storeid-local-part-altering
Add fn StoreId::local_push() for altering the local part of a StoreId
2017-06-17 21:14:44 +02:00
5b8c138922 Add fn StoreId::local_push() for altering the local part of a StoreId 2017-06-17 17:21:27 +02:00
bc95c5615d Merge pull request #970 from matthiasbeyer/libimagstore/fs-abstraction-pub
Make libimagstore::file_abstraction pub
2017-06-17 14:07:01 +02:00
71e3d3d2d1 Merge pull request #968 from matthiasbeyer/libimagentrytag/validator-helper-enhancement
Add is_tag_str(&str) wrapper for is_tag(String)
2017-06-17 12:54:13 +02:00
1f26460b90 Make libimagstore::file_abstraction pub 2017-06-17 12:51:29 +02:00
ac805dcef7 Add is_tag_str(&str) wrapper for is_tag(String) 2017-06-17 12:34:13 +02:00
c4d4fe9389 Merge pull request #967 from matthiasbeyer/libimagstore/remove-todo-comment
Remove "TODO"-comment
2017-06-13 09:04:14 +02:00
ac8ea872cb Remove "TODO"-comment 2017-06-12 19:29:57 +02:00
a9d2d7c354 Merge pull request #962 from matthiasbeyer/libimagstore/fs-memory-backend-as-dependency-injection
Libimagstore/fs memory backend as dependency injection
2017-06-10 21:01:57 +02:00
15b77ac2c1 Merge pull request #965 from matthiasbeyer/libimagentrytag/clap-validators
Add clap validator for tag string
2017-06-09 14:55:34 +02:00
98c33f7571 Add clap validator for tag string 2017-06-09 13:48:24 +02:00
bec59c1f8a Remove warnings by returning errors explicitely 2017-06-09 13:27:06 +02:00
81449f48d6 Re-export file abstraction types 2017-06-09 13:22:45 +02:00
f82813bc3d Use in-memory backend in store tests 2017-06-09 11:15:55 +02:00
eeccd148f3 Transform backend code from compiletime-injection to dependency-injection
The title might be a bit inaccurate, but I cannot think of something
better.

Before the change
=================

Before this change, we had a compiletime backend for the store. This
means that the actual filesystem operations were compiled into the store
either as real filesystem operations (in a normal debug or release
build) but as a in-memory variant in the 'test' case.
So tests did not hit the filesystem when running.
This gave us us the possibility to run tests concurrently with multiple
stores that did not interfere with eachother.

Problem
=======

This approach worked perfectly well until we started to test not the
store itself but crates that depend on the store implementation.
When running tests in a crate A that depends on the store, the store
itself was compiled with the filesystem-hitting-backend.
This was problematic, as tests could not be implemented without hitting
the filesystem.

After the change
================

After this change, the backend code is injected into the store via
dependency injection (the `Store::new()` function automatically uses the
filesystem-backend).
The store can be created with a the in-memory backend when running tests
now.

Implementation
==============

The implementation of this is rather stupid, despite the big diff in
this commit.

Lets have a look at the `Store` code changes first and then we'll
discuss the `file_abstraction` changes this commit introduces.

libimagstore::fs_abstraction
----------------------------

The filesystem was abstracted via a trait `FileAbstraction` which
contains the essential functions for working with the filesystem.

Two implementations are provided in the code:

    * FSFileAbstraction
    * InMemoryFileAbstraction

whereas the first actually works with the filesystem and the latter
works with an in-memory HashMap that is used as filesystem.

Further, the trait `FileAbstractionInstance` was introduced for
functions which are executed on actual instances of content from the
filesystem, which was previousely tied into the general abstraction
mechanism.

So, the `FileAbstraction` trait is for working with the filesystem, the
`FileAbstractionInstance` trait is for working with instances of content
from the filesystem (speak: actual Files).

The `FileAbstraction` trait requires a function to be implemented that
can be used to create a `FileAbstractionInstance` object from it.

In case of the `FSFileAbstractionInstance`, which is the implementation
of the `FileAbstractionInstance` for the actual filesystem-hitting code,
the underlying resource is managed like with the old code before.
The `InMemoryFileAbstractionInstance` implementation is corrosponding to
the `InMemoryFileAbstraction` implementation - for the in-memory
"filesystem".

The implementation of the `get_file_content()` function had to be
changed to return a `String` rather than a `&mut Read` because of
lifetime issues.
This change is store-internally and the API of the store itself was not
affected.

libimagstore::store::Store changes
----------------------------------

So, first we need to make sure that the store knows the actual backend.
Therefor, the `new()` method was renamed to `new_with_backend()` and got
another parameter: the backend implementation.
A new `new()` function was created for convenience and
backwards-compatibility with the rest of the imag codebase (and also
because nobody wants to pass the backend manually all the time).

As the backend is abstracted via a trait, and the store should not
change its interface, the new `Store` member was introduced inside a
`Box`.

All calls (`remove_file()`, `copy()`, `rename()` and `create_dir_all()`)
were refactored from `FileAbstraction::*` calls into `self.backend.*`
calls.

libimagstore::store::StoreEntry changes
---------------------------------------

The `StoreEntry` type is constructed in the store internally for holding
a `StoreId` object as well as some status and the file abstraction code.

This object is constructed via a `new()` function that got a new
parameter: a `&Box<FileAbstraction>` to the backend abstraction the
store uses.
This backend is now used to create a new `FileAbstractionInstance`
object for the `StoreEntry`.

Also the `StoreEntry::get_entry()` code had to be adapted to the new
`Entry::from_str()` function interface.

This commit message is partially added as comment in the code.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2017-06-09 11:15:55 +02:00
ef82b2ab41 Merge pull request #961 from matthiasbeyer/add-missing-license-header
Add missing license headers
2017-06-07 21:13:14 +02:00
795df63679 Add missing license headers 2017-06-07 20:45:38 +02:00
4fff92e7c0 Merge pull request #960 from matthiasbeyer/libimagmail/use-email-crate
Libimagmail/use email crate
2017-06-07 17:52:40 +02:00
1b15d45e7c Merge pull request #959 from matthiasbeyer/libimagentrydate/fix-header-location
Fix header location for datetime values
2017-06-07 17:40:57 +02:00
a106b8c1f4 Add missing license headers 2017-06-07 17:40:45 +02:00
226c939ab8 Remove unused imports 2017-06-07 17:39:49 +02:00
0219f00ec9 Make MailIter pub 2017-06-07 17:39:23 +02:00
41a12db528 Use "email" instead of "mailparse" 2017-06-07 17:38:50 +02:00
ceda2643c0 Fix header location for datetime values 2017-06-07 17:25:58 +02:00
16a12af873 Merge pull request #941 from matthiasbeyer/libimagentrydate/init
libimagentrydate/init
2017-06-07 17:00:52 +02:00
9982195714 Add more tests 2017-06-07 16:47:52 +02:00
c7bbc83572 Update toml-query: 0.1 -> 0.2 2017-06-07 16:47:50 +02:00
33f700dd1c Add test for testing Entry::set_date() 2017-06-07 16:47:17 +02:00
08f43c8811 Merge pull request #957 from matthiasbeyer/update-toml-query
dependency: toml-query: 0.1.0 -> 0.2.0
2017-06-07 13:37:02 +02:00
820037bba6 dependency: toml-query: 0.1.0 -> 0.2.0 2017-06-07 12:29:57 +02:00
bb9ff5bfd8 Merge pull request #951 from matthiasbeyer/remove-hooks
libimagstore: Remove hook support
2017-06-07 11:45:02 +02:00
72f0664976 Merge pull request #952 from matthiasbeyer/remove-hooks-impl-link-consistency-check-in-libimagentrylink
Remove hooks impl link consistency check in libimagentrylink
2017-06-06 22:30:00 +02:00
2b3eab31bd Add tests for range type 2017-06-06 19:37:30 +02:00
fb228c65b1 Move range type to own module 2017-06-06 19:37:30 +02:00