Commit graph

128 commits

Author SHA1 Message Date
972327e35e Remove Store::reset_backend() 2018-05-01 21:09:51 +02:00
2f0a557068 Fix for passing Arc<_> to store interface instead of Box<_> 2018-05-01 21:08:57 +02:00
42e2f18fb3 Pass around Arc<FileAbstraction> internally
Because the iterators need to be able to check whether the file exists
_in the backend_ (not on disk, but in the backend, because of in-memory
test for example), we need to be able to pass the backend to the
iterator intermediate type.

This patch implements this. It does so by changing the internal backend
member of the store from `Box<FileAbstraction>` to
`Arc<FileAbstraction>`, which gives us the ability to clone the
reference to the backend easily without needing to rely on lifetimes
here, because of the Arc.

Also, less boxes are always good.
2018-05-01 21:08:57 +02:00
d08c27e623 Rewrite StoreIdIterator extensions
This patch reimplements the iterator extensions.

As we iterate (in StoreIdIterator) over Result<StoreId> now anyways, we
don't need the extensions for Result iterators anymore.

This patch rewrites the extensions to be more simple in every way and
generic over the error type in the iterator.
All the errors have to do is implement From<StoreError>, which is what
they do when linking the generated error types with error_chain to the
libimagstore error types.
2018-05-01 17:44:01 +02:00
a2ff298e67 Rewrite Store::entries()
This patch rewrites the Store::entries() function to not be collecting
the iterator.

It therefore introduces a new, internal, iterator type which creates the
StoreId objects from the pathes the PathIterator yields internally.

With this patch, the Store iterator interface changes, as the iterators
now yield `Result<StoreId, StoreError>` instead of `StoreId`.
This is necessary, as the internal conversion errors shouldn't be
hidden.

Of course, the iterator types (like the StoreGetIterator and so on)
should hold a Result<StoreId> internally as well, and also yield
appropritely. This was changed in this commit, too.
2018-05-01 17:44:00 +02:00
f4556f3983 Rewrite backend to not collect on pathes_recursively() 2018-05-01 17:44:00 +02:00
900d9f0984
Merge pull request #1474 from matthiasbeyer/libimagstore/cache-flush
Add functionality to flush the internal cache
2018-04-30 21:07:37 +02:00
6a5bcb2709 Remove unused error kinds 2018-04-30 18:54:11 +02:00
6a40ac6b98 Add information about which storeid is the cause of the error in errors 2018-04-30 18:53:54 +02:00
860e47a256 Add functionality to flush the internal cache
Before we had the problem that when iterating over _a lot_ (like 5k)
entries and also fetching them, at some point the OS would return with
"Too many files open".

That is because the store internally caches a lot.

With this change, the Store gets an API to query how big the cache is,
how much the cache can currently hold and (and that's the main thing in
this patch) to flush the cache to disk.

A function to simply ask the store whether its cache should be flushed
(which would us require to ask the OS how many files we can open...
which would be possible with `libc::getrlimit`) does not yet exist,
though, but could be added easily if desired.
2018-04-30 18:39:59 +02:00
ea80a5a09b Derive Default for InMemoryFileAbstraction, remove constructor 2018-04-25 19:15:07 +02:00
6a34e7a8fd Derive Default for FSFileAbstraction, reduces boilerplate 2018-04-25 19:15:07 +02:00
bf363c4748 Use "if let" if matching on a single pattern 2018-04-25 19:15:07 +02:00
f1142c414d Use ok_or_else() instead of ok_or() 2018-04-25 19:15:07 +02:00
22170a0d55 Do not String::from(String) 2018-04-25 19:13:51 +02:00
6f5590713a Use single quotes here 2018-04-25 19:13:51 +02:00
44521a7c8c Dont use "return" keyword at end of function 2018-04-25 19:13:51 +02:00
dc18e650bb Collapse nested if-else-if 2018-04-25 19:13:51 +02:00
84249e3fb5 Be less verbose when constructing an object
The rust compiler does some fancy things for us: It automatically finds
the right fields if the name of the variable and the file is the same.

Lets use that to reduce boilerplate with this patch.
2018-04-25 19:13:50 +02:00
524c391ee0 Remove unused dependency 2018-04-24 22:16:40 +02:00
563c76c375 Remove "stdio" file abstraction from store implementation 2018-04-24 22:16:40 +02:00
19e0471f5b Remove tests with JsonMapper in backend 2018-04-24 22:16:40 +02:00
bae9188a08 Remove GlobStoreIdIterator helper iterator type 2018-04-24 13:40:42 +02:00
d70092d691 Remove: Store::retrieve_for_module 2018-04-24 13:40:42 +02:00
d2dceb6640 Update version numbers: 0.7.0 -> 0.8.0 2018-04-19 10:05:53 +02:00
4568b1f625
Merge pull request #1380 from matthiasbeyer/minor
Minor patches
2018-04-18 16:46:41 +02:00
98f8366e4c Fix documentation in crate metadata 2018-04-17 15:55:08 +02:00
604f350340 Add logging in store 2018-04-14 16:55:36 +02:00
a769186cd7 Make iterator lifetimes less restricting 2018-04-07 13:55:54 +02:00
e61ccc9561 Fix: Entry::to_str() should return Result<_>
Because serializing might fail.

Also fixes all usages of the API.
2018-03-26 14:24:42 +02:00
9e0d0d9f6c Add Entry::replace_from_buffer() 2018-03-23 15:44:34 +01:00
7e623f39b8 Add message why panic
So we see that when grepping the source for `unimplemented!()`.
2018-03-23 10:05:58 +01:00
41f7bab1c5 Fix: Store::create() should fail if the entry exists 2018-03-22 21:16:11 +01:00
68ace5d147 Fix typo 2018-03-12 13:32:23 +01:00
da408b9e67 This fixes the file parsing (again)
Unfortunately, our latest fix to file parsing did not solve all issues.
So we have to fix it _again_.

The problem was the `std::str::Lines` iterator, which apparently fails
this:

    assert_eq!(1, "".lines().count());

as an empty line seems not to be a line.

Because of that, when reading a file with an empty line at its bottom
got stripped off that line.

This patch removes the use of the `lines()` iterator and uses
`split("\n")` instead. This only works on Unix operating systems, but as
we only target unix operating systems with imag, this is not considered
an issue right now.

This patch also adds extensive tests on multiple levels in the
`libimagstore` implementation:

* On the parsing level, for the function which implements the parsing
* On the filesystem abstraction levels
* On the `Store` levels

to make sure that everything is parsed correctly.
2018-03-10 14:53:15 +01:00
579c00b62a Replace hard coded version strings with compiletime env 2018-03-06 12:45:41 +01:00
ee5a17790f Fix multi-line reading
Reading entries with multiple entries caused all lines to be read as one
line.

This bug is fixed with this patch.
2018-03-02 21:48:17 +01:00
87d6f1221f Add testcase for reading multiple lines 2018-03-02 21:48:13 +01:00
9fb5f453fe Rewrite entry parsing algorithm
Rewrite without regex crate.

The regex approach was broken. If the following _content_ was provided
in the entry:

    foo

    ---

    bar

The regex approach parsed the header until the "---" in the content.
This is, of course, not the way to do that.

Now, the parsing is implemented by hand. Should be faster as well,
though I don't care about this.

This fixes a severe bug.
2018-02-20 21:06:36 +01:00
5f99018807 Simplify store debug print 2018-02-19 11:51:21 +01:00
9e3c966d84
Merge pull request #1273 from matthiasbeyer/update-version-numbers
Update version string: 0.6.0 -> 0.7.0
2018-02-11 17:50:52 +01:00
30d2c28a22 Update version string: 0.6.0 -> 0.7.0 2018-02-11 14:22:24 +01:00
8fcd2a6c8c Prettify Store debug output implementation 2018-02-11 13:36:37 +01:00
d5a9e4930d Update dates in license header
And add missing header in one file
2018-02-07 02:48:53 +01:00
de613c9ebd
Merge pull request #1227 from matthiasbeyer/libimagentryutil/filtered-iterators
libimagentryutil: filtered iterators
2018-02-01 20:22:12 +01:00
06cb8d50fb Make StoreId::is_in_collection() generic over AsRef<str> 2018-02-01 16:46:57 +01:00
b3f546129a Add StoreIdIteratorWithStore::without_store() 2018-02-01 16:46:38 +01:00
8e931cd79f Change calls to into_get_iter() to not take argument 2018-02-01 16:46:38 +01:00
3b01a9eb2f Add API in StoreIdIteratorWithStore to get acting iterators
This patch adds API functions in the StoreIdIteratorWithStore iterator
type to transform it into a iterator which _does_ something (as in the
`libimagstore::iter` API).

It mimics the API which is offered by `libimagstore::iter`.
2018-02-01 15:47:39 +01:00
030e32e44f Provide StoreId Iterator which has a ref to the Store
This change is needed so we can refactor the "get" iterator to not take
an argument (the store) later, which improves the API.
2018-02-01 15:47:39 +01:00