Commit graph

293 commits

Author SHA1 Message Date
feea57679d Remove scope
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-11-06 20:04:47 +01:00
21d411f57b Remove call to format_err!(), use err_msg() instead
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-11-06 20:04:47 +01:00
09f0968755 Remove repetitive code by abstracting it
With this patch, the log level aggregation is abstracted away by using
zero sized types and a macro to implement the whole thing.

This does not alter functionality, but makes the code more readable.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-11-06 19:39:41 +01:00
1ff1d85428 Remove unnecessary scope
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-11-06 19:08:00 +01:00
1a70c523dc Remove indirection
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-11-06 19:03:19 +01:00
d493b04555 Remove scope for better readability
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-11-06 19:03:10 +01:00
0e4cb59772 Format code
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-11-06 19:02:57 +01:00
582fd10acb FS backend: Safe allocation of new PathBuf object
::stf::fs::create_dir_all() takes any ref to `Path`, which is what we
have here, so we can leave out the allocation of a new PathBuf object
here.

Also remove the match by a `if let Some(_)`, which increases the
readability a bit.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-11-06 18:37:53 +01:00
88a4eee087 Adapt Store::get()/Store::create() to check cache before FS
With this change, the cache is tested before accessing the filesystem,
which probably increases the speed if the cache has the entry, because
we avoid the slow IO operation.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-11-06 18:35:17 +01:00
becfcd4180 Fix: In-Memory test backend: Actually remove the old entry on "move"
This fixes a really ugly bug where the in-memory backend for the store
did not remove the entry from the in-memory hashmap on "move", but
simply copied it from the old location to the new one.

That caused tests to fail after the fixes introduced for the
Store::get() function which checked the filesystem and the internal
cache whether an entry exists before the actual "get" operation, because
an old entry would still exist after a move (only in the testcases).

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
Fixes: 09e8619cf5 ("libimagstore: Move from error-chain to failure")
2018-11-06 18:34:33 +01:00
daaa4fd9ca Make assert!() output more verbose
By printing the actual `Result<>` object that failed the assertion.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-11-06 18:17:46 +01:00
d982a657f8 Rewrite Debug for FileLockEntry
This changes the implementation of Debug for the FileLockEntry to be
more explanatory of how the entry actually looks like. It does not only
print the path of the store anymore, but also the location of the Entry.

Printing header and content would be still too much, tho.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-11-06 18:12:58 +01:00
e919435504 Fix: Remove call to deprecated function 2018-11-06 14:48:49 +01:00
cb0fcaffde Remove logging calls in favour of debug calls
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-11-01 21:35:40 +01:00
f01a262bdf Add debugging output in test via logger 2018-11-01 20:37:00 +01:00
3bf3a8890b Add debugging output in test via logger 2018-11-01 20:36:46 +01:00
993f83a31b Print Runtime errors to stderr 2018-11-01 20:26:25 +01:00
1a1cc885cc Merge branch 'rt-remove-default-value-verbosity'
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-11-01 00:05:42 +01:00
351849e349 Fix: Remove default value for --verbose flag
This actually caused an error.

When executing `imag ids --verbose trace` for example, the `imag` binary
got the default value (as in `--verbose info`) supplied from clap. So
far, so good. Problem is that the implementation then forwarded that
flag to `imag-ids`, which resulted in the `--verbose` flag to be passed
_two times_ to `imag-ids`.

Somehow this is really strange, but it does not really matter. First of
all: A default of "Info" is still too high IMO. Default should be
warnings and errors, but no information printed. We like silent tools,
don't we?

Second is that the commandline argument forwarding mechanism of `imag`
is broken and this was a fix which helped debugging of the brokenness,
so this is acutally a step forward in this regard as well.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-10-31 23:13:07 +01:00
9636cef7a8 Print (debugging) CLI when initializing runtime
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-10-31 23:11:06 +01:00
59f16d0eab Add feature: Public logging initialization
This feature is required for the `imag` binary. It allows the binary to
use the imag internal logger for logging its own log output.

We need to be able to initialize the logger from an external module (in
all imag modules, the Runtime::new() implementation takes care of this,
but as we cannot use that in the `imag` binary itself, we allow this
method to be public behind a feature flag).

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-10-31 23:10:26 +01:00
8236e73402 libimagrt: Move from error-chain to failure
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-10-30 18:46:28 +01:00
09e8619cf5 libimagstore: Move from error-chain to failure
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-10-30 18:46:28 +01:00
c2f674cc29 libimagerror: Move from error-chain to failure
Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-10-30 18:46:28 +01:00
c160a967ec Add error kinds 2018-10-30 18:46:28 +01:00
22c13fed42 Fix: --debug on CLI should enable logging and override it
When specifying the `--debug` flag on the commandline, logging was not
enabled. That was because the config file parsing did not consider the
args.

Now, if `--debug` is passed on the CLI, logging is enabled for all
modules and level is set to `debug` for all modules.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-10-26 15:41:00 +02:00
8ec8b0aca7 Hotfix: Ignore code snippet here 2018-10-24 23:46:21 +02:00
d87ea7b305 Add StoreId::local_display_string() function
for safely creating a `String` object from `StoreId` which can be shown
to the user.

Mainly introduced because this is useful for error handling (when
putting a `StoreId` into an error kind, the compiler complains that
`StoreId` does not meet the required trait bounds. But `String` does and
we do not process the ID any further anyways).

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-10-13 11:52:16 +02:00
3090a65446 Add detailed comments on how the logger works 2018-10-11 00:15:48 +02:00
cc11162dd4 Fix: In-Memory filesystem backend did not remove on rename
The implementation of the in-memory filesystem for testing imag code did
not actually use `HashMap::remove()` when an entry was moved, but
`HashMap::get().cloned()`, which caused the original entry to exist
_after_ the move.

I'm not sure why this did not fail much earlier, but it was clearly
wrong. This commit adjust the test to check the "filesystem" before
checking the store and fixes the bug.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-10-09 17:37:56 +02:00
6751c34f1b Fix: Remove call to deprecated function
This actually caused tests to fail if there was indeed a file at
/tmp/store/test and the test tried to create a "test" entry in the store.

Do use backend instead to check whether entry actually exists.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-10-09 17:37:35 +02:00
1161562372 Remove handlebars exception in log implementation
The issue was that the handlebars implementation logs as well and if we
use handlebars in the logger implementation that causes recursion which
crashes the program.

With handlebars 1.0.5, there is a feature[0] to disable logging in
handlebars (compiletime) which we use with this patch. The
exception-checking in the log implementation can be removed therefore.

[0]: https://github.com/sunng87/handlebars-rust/pull/236#issuecomment-427014611

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-10-06 12:22:15 +02:00
c162856a21 Update dependencies
regex: 0.2 -> 1
semver: 0.8 -> 0.9
walkdir: 1 -> 2
2018-10-06 12:21:20 +02:00
e34d67d0a1 Update dependency: ansi_term: 0.10 -> 0.11 2018-09-30 15:05:10 +02:00
1c3bc5951f Update dependency: ansi_term: 0.10 -> 0.11 2018-09-30 15:04:51 +02:00
e838f0d0e7 Update dependency: handlebars: 0.29.0 -> 1.0 2018-09-27 15:05:45 +02:00
61738ac9ee Revert "Update dependency: handlebars: 0.29.0 -> 1.0"
This reverts commit a1f0872486995b80216e8a08a2176debdef3752a.

As updating handlebars needs some more involvement, we roll back to the
version we use currently and schedule the update for later.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-09-27 15:05:45 +02:00
abf6a3c384 Update dependency: handlebars: 0.29.0 -> 1.0 2018-09-27 15:05:45 +02:00
3d12998603 Update dependency: toml-query: 0.6 -> 0.7 2018-09-27 15:05:44 +02:00
0a7afee454 Update dependency: error-chain: 0.11 -> 0.12 2018-09-27 15:05:25 +02:00
d4872f6da3 Optimize the Store::entries() interface
The previous iterator was implemented to simply fetch _all_ pathes from
the filesystem, no matter what.

With this implementation, this changes. The iterator now has
functionality to optimize the iteration, if only a subdirectory of the
store is required, for example `$STORE/foo`.

This is done via functionality where the underlying iterator gets
altered.

First of all, the interface was changed to return a `Entries` object,
which itself only covers the libimagstore-internal `PathIterator` type.
This type was changed so that the backend implementation provides an
"PathIterBuilder`, which builds the actual iterator object for the
`PathIterator` type.

The intermediate `StoreIdConstructingIterator` was merged into
`PathIterator` for simplicity.

The `Entries` type got functionality similar to the
`StoreIdIteratorWithStore` type for easier transition to the new API.
This should probably be removed at a later point, though.

As the `walkdir::WalkDir` type is not as nice as it could be, iterators
for two collections in the store could be built like this (untested):

    store
        .entries()?
        .in_collection("foo")
        .chain(store.entries()?.in_collection("bar"))

Functionality to exclude subdirectories is not possible with the current
`walkdir::WalkDir` implementation and has to be done during iteration,
with filtering (as usual).

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-09-27 12:19:18 +02:00
d59dca1a23 Add StoreIdIterator::with_store() 2018-09-27 12:19:18 +02:00
8b508fe4c3 Optimize backend impl to not hold open files
This patch changes the filesystem-backend implementation of libimagstore
to open files on each read/write rather than holding the file handle in
memory at all times.

Whenever a lot of imag store entries are read into memory, the imag
process may ran out of file descriptors. With this patch applied, a
`Store::get()` call on an entry which is not yet in the store cache
would cause the file to be read, but the FD being dropped after that.
Likewise, a `Store::update()` (which is also called if the imag entry is
dropped) would re-open the file on the filesystem and write the contents
from the imag store cache back to the file.

With this patch, opening hundrets or thousands of imag entries should be
no problem anymore, only the available memory should be a limit then.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-08-11 17:50:22 +02:00
aff6ff105d Remove "Header" extension for Value type 2018-07-19 20:58:27 +02:00
f6a7345b4a Simplify: Move header verification from Value extension to Entry type 2018-07-19 20:58:27 +02:00
9ad1c8d6bd Simplify: Move default header generation from Value extension to Entry type 2018-07-19 20:58:27 +02:00
1705ecbbff Fix: Remove (empty) Drop implementation for Store
The StoreEntry drop implementation takes care of unlocking the files.
2018-07-19 20:58:27 +02:00
b774ac0e67 Fix: Deleting an Entry could leave artifacts in cache
This patch fixes a bug we did not even hit (yet). It is: When deleting
an Entry from the store, this could potentially leave artifacts in the
cache.

Szenario: An Entry, which was loaded (via `Store::get()` for example),
gets `Store::delete()`ed twice. The first call would work as expected,
but leave the Entry in the Store cache. The second call would then fail,
as the Entry is already removed on the FS, but still in the cache. This
would fail - which is the right thing to do here - but with the wrong
error (with a FileError rather than a FileNotFound error).

This patch fixes this.

First of all, the appropriate `PathBuf` object is calculated in all
cases, as this object is needed to check whether the file is actually
there (which could be the case if the Entry is in the cache and if it is
not).

If the entry is in the cache and is borrowed: error. If not, remove the
entry from the cache. Afterwards the file is deleted on disk.

If the entry is not in the cache, but the file exists, the file is removed.
If the file does not exist: error.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-07-19 20:58:27 +02:00
9315a23754 Fix: Use backend abstraction for checking whether a path exists 2018-07-19 20:58:27 +02:00
54c5854761 Remove unnecessary call 2018-06-23 21:56:03 +02:00