Commit graph

705 commits

Author SHA1 Message Date
c162856a21 Update dependencies
regex: 0.2 -> 1
semver: 0.8 -> 0.9
walkdir: 1 -> 2
2018-10-06 12:21:20 +02:00
989ca3dba8 Update dependencies: regex: 0.2 -> 1, tempfile: 2 -> 3 2018-10-06 12:21:20 +02:00
e34d67d0a1 Update dependency: ansi_term: 0.10 -> 0.11 2018-09-30 15:05:10 +02:00
1c3bc5951f Update dependency: ansi_term: 0.10 -> 0.11 2018-09-30 15:04:51 +02:00
d9e4eaad3c Update kairos to master branch
Because of the new error-chain version which kairos can depend on, for
which kairos is not yet released.
2018-09-27 15:05:45 +02:00
1fa9b86531 Update dependency: lazy_static: 0.2 -> 1 2018-09-27 15:05:45 +02:00
e838f0d0e7 Update dependency: handlebars: 0.29.0 -> 1.0 2018-09-27 15:05:45 +02:00
f97d9e0d96 Temporarily set vobject to custom fork until we have a new release 2018-09-27 15:05:45 +02:00
946d9891e4 Update dependency: vobject: 0.{4,5} -> 0.6 2018-09-27 15:05:45 +02:00
61738ac9ee Revert "Update dependency: handlebars: 0.29.0 -> 1.0"
This reverts commit a1f0872486995b80216e8a08a2176debdef3752a.

As updating handlebars needs some more involvement, we roll back to the
version we use currently and schedule the update for later.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-09-27 15:05:45 +02:00
c65e73acc8 Update dependency: kairos: 0.1 -> 0.2 2018-09-27 15:05:45 +02:00
abf6a3c384 Update dependency: handlebars: 0.29.0 -> 1.0 2018-09-27 15:05:45 +02:00
764a019817 Update dependency: log: 0.3 -> 0.4 2018-09-27 15:05:45 +02:00
3d12998603 Update dependency: toml-query: 0.6 -> 0.7 2018-09-27 15:05:44 +02:00
0a7afee454 Update dependency: error-chain: 0.11 -> 0.12 2018-09-27 15:05:25 +02:00
d4872f6da3 Optimize the Store::entries() interface
The previous iterator was implemented to simply fetch _all_ pathes from
the filesystem, no matter what.

With this implementation, this changes. The iterator now has
functionality to optimize the iteration, if only a subdirectory of the
store is required, for example `$STORE/foo`.

This is done via functionality where the underlying iterator gets
altered.

First of all, the interface was changed to return a `Entries` object,
which itself only covers the libimagstore-internal `PathIterator` type.
This type was changed so that the backend implementation provides an
"PathIterBuilder`, which builds the actual iterator object for the
`PathIterator` type.

The intermediate `StoreIdConstructingIterator` was merged into
`PathIterator` for simplicity.

The `Entries` type got functionality similar to the
`StoreIdIteratorWithStore` type for easier transition to the new API.
This should probably be removed at a later point, though.

As the `walkdir::WalkDir` type is not as nice as it could be, iterators
for two collections in the store could be built like this (untested):

    store
        .entries()?
        .in_collection("foo")
        .chain(store.entries()?.in_collection("bar"))

Functionality to exclude subdirectories is not possible with the current
`walkdir::WalkDir` implementation and has to be done during iteration,
with filtering (as usual).

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-09-27 12:19:18 +02:00
d59dca1a23 Add StoreIdIterator::with_store() 2018-09-27 12:19:18 +02:00
8b508fe4c3 Optimize backend impl to not hold open files
This patch changes the filesystem-backend implementation of libimagstore
to open files on each read/write rather than holding the file handle in
memory at all times.

Whenever a lot of imag store entries are read into memory, the imag
process may ran out of file descriptors. With this patch applied, a
`Store::get()` call on an entry which is not yet in the store cache
would cause the file to be read, but the FD being dropped after that.
Likewise, a `Store::update()` (which is also called if the imag entry is
dropped) would re-open the file on the filesystem and write the contents
from the imag store cache back to the file.

With this patch, opening hundrets or thousands of imag entries should be
no problem anymore, only the available memory should be a limit then.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-08-11 17:50:22 +02:00
acdc1e84bc
Merge pull request #1495 from matthiasbeyer/imag-diary/fix-missing-header
Imag diary/fix missing header
2018-07-27 20:10:59 +02:00
22d63f0946 Add functionality to create entry at a certain time 2018-07-20 01:37:36 +02:00
aff6ff105d Remove "Header" extension for Value type 2018-07-19 20:58:27 +02:00
f6a7345b4a Simplify: Move header verification from Value extension to Entry type 2018-07-19 20:58:27 +02:00
9ad1c8d6bd Simplify: Move default header generation from Value extension to Entry type 2018-07-19 20:58:27 +02:00
1705ecbbff Fix: Remove (empty) Drop implementation for Store
The StoreEntry drop implementation takes care of unlocking the files.
2018-07-19 20:58:27 +02:00
b774ac0e67 Fix: Deleting an Entry could leave artifacts in cache
This patch fixes a bug we did not even hit (yet). It is: When deleting
an Entry from the store, this could potentially leave artifacts in the
cache.

Szenario: An Entry, which was loaded (via `Store::get()` for example),
gets `Store::delete()`ed twice. The first call would work as expected,
but leave the Entry in the Store cache. The second call would then fail,
as the Entry is already removed on the FS, but still in the cache. This
would fail - which is the right thing to do here - but with the wrong
error (with a FileError rather than a FileNotFound error).

This patch fixes this.

First of all, the appropriate `PathBuf` object is calculated in all
cases, as this object is needed to check whether the file is actually
there (which could be the case if the Entry is in the cache and if it is
not).

If the entry is in the cache and is borrowed: error. If not, remove the
entry from the cache. Afterwards the file is deleted on disk.

If the entry is not in the cache, but the file exists, the file is removed.
If the file does not exist: error.

Signed-off-by: Matthias Beyer <mail@beyermatthias.de>
2018-07-19 20:58:27 +02:00
9315a23754 Fix: Use backend abstraction for checking whether a path exists 2018-07-19 20:58:27 +02:00
newpavlov
04b3e7243d replaced rust-crypto with RustCrypto crates 2018-07-18 19:51:05 +03:00
54c5854761 Remove unnecessary call 2018-06-23 21:56:03 +02:00
3bdd5c959b Add documentation to StoreId type 2018-06-23 21:56:03 +02:00
c86e4e7db4 Add documentation for Runtime::init_logger() 2018-06-23 21:56:02 +02:00
8d1022b1ac Fix: Explicitely use Itertools::flatten()
As of rustc 1.26, the `flatten()` method on iterators is preserved by
the rust standard library.
This could cause this code to hard-error some time in the future with
the `flatten()` function actually implemented by the standard library.

Hence we move to use the `Itertools::flatten()` function here
explicitely.
2018-05-11 15:45:42 +02:00
2e41656d1a Fix comment for rustc 1.26 2018-05-11 14:55:23 +02:00
664edc7943 Update version string: 0.8.0 -> 0.9.0 2018-05-09 11:39:33 +02:00
dd3adcd08d
Merge pull request #1482 from matthiasbeyer/libimagrt/cli-validation
libimagrt: cli validation
2018-05-04 13:13:33 +02:00
c65a3d1af9 Add CLI validators in default CLI setup 2018-05-04 11:21:49 +02:00
9bf3f6da1b Update help 2018-05-04 11:20:09 +02:00
23c3a4b863 Update help, clap does the rest here 2018-05-04 11:19:47 +02:00
2e2bce77a0 Add functionality to remove categories 2018-05-04 11:09:20 +02:00
eb20a9d881
Merge pull request #1477 from matthiasbeyer/libimagstore/remove-walk
Remove Store::walk()
2018-05-02 18:39:11 +02:00
258d9f90d3 Remove Store::walk()
This patch removes the Store::walk() interface.

It was cumbersome and unused anyways.
2018-05-02 17:46:45 +02:00
888b31377e Fix: Do not check whether path exists - check whether its a file
This fixes a bug introduced in 195d921218
where we didn't check whether the path is actually a file.
2018-05-02 14:04:59 +02:00
2bf09e7737 Only create directory if it does not exist 2018-05-01 22:29:46 +02:00
84f426297e Add debug output 2018-05-01 22:29:46 +02:00
b27b392f4b Add trace output 2018-05-01 22:29:46 +02:00
972327e35e Remove Store::reset_backend() 2018-05-01 21:09:51 +02:00
2f0a557068 Fix for passing Arc<_> to store interface instead of Box<_> 2018-05-01 21:08:57 +02:00
42e2f18fb3 Pass around Arc<FileAbstraction> internally
Because the iterators need to be able to check whether the file exists
_in the backend_ (not on disk, but in the backend, because of in-memory
test for example), we need to be able to pass the backend to the
iterator intermediate type.

This patch implements this. It does so by changing the internal backend
member of the store from `Box<FileAbstraction>` to
`Arc<FileAbstraction>`, which gives us the ability to clone the
reference to the backend easily without needing to rely on lifetimes
here, because of the Arc.

Also, less boxes are always good.
2018-05-01 21:08:57 +02:00
d08c27e623 Rewrite StoreIdIterator extensions
This patch reimplements the iterator extensions.

As we iterate (in StoreIdIterator) over Result<StoreId> now anyways, we
don't need the extensions for Result iterators anymore.

This patch rewrites the extensions to be more simple in every way and
generic over the error type in the iterator.
All the errors have to do is implement From<StoreError>, which is what
they do when linking the generated error types with error_chain to the
libimagstore error types.
2018-05-01 17:44:01 +02:00
6ee3f4a3f6 Refactor libimagentrycategory to fit new store iterator interface 2018-05-01 17:44:01 +02:00
6f81d02445 Refactor libimagentryannotation to fit new store iterator interface 2018-05-01 17:44:01 +02:00
40688a3c2d Refactor libimagwiki to fit new store iterator interface 2018-05-01 17:44:01 +02:00
715753ed25 Refactor libimagtodo to fit new store iterator interface 2018-05-01 17:44:01 +02:00
72c83ae3f7 Refactor libimagtimetrack to fit new store iterator interface 2018-05-01 17:44:01 +02:00
08114bbf36 Refactor libimagnotes to fit new store iterator interface 2018-05-01 17:44:00 +02:00
e643f36fa3 Refactor libimaghabit to fit new store iterator interface 2018-05-01 17:44:00 +02:00
bf0bef058d Refactor libimagdiary to fit new store iterator interface 2018-05-01 17:44:00 +02:00
8f03b4a71a Refactor libimagcontact to fit new store iterator interface 2018-05-01 17:44:00 +02:00
a2ff298e67 Rewrite Store::entries()
This patch rewrites the Store::entries() function to not be collecting
the iterator.

It therefore introduces a new, internal, iterator type which creates the
StoreId objects from the pathes the PathIterator yields internally.

With this patch, the Store iterator interface changes, as the iterators
now yield `Result<StoreId, StoreError>` instead of `StoreId`.
This is necessary, as the internal conversion errors shouldn't be
hidden.

Of course, the iterator types (like the StoreGetIterator and so on)
should hold a Result<StoreId> internally as well, and also yield
appropritely. This was changed in this commit, too.
2018-05-01 17:44:00 +02:00
f4556f3983 Rewrite backend to not collect on pathes_recursively() 2018-05-01 17:44:00 +02:00
900d9f0984
Merge pull request #1474 from matthiasbeyer/libimagstore/cache-flush
Add functionality to flush the internal cache
2018-04-30 21:07:37 +02:00
41615bcc97
Merge pull request #1473 from matthiasbeyer/libimagstore/more-error-information
Add information about which storeid is the cause of the error in errors
2018-04-30 21:07:02 +02:00
6a5bcb2709 Remove unused error kinds 2018-04-30 18:54:11 +02:00
6a40ac6b98 Add information about which storeid is the cause of the error in errors 2018-04-30 18:53:54 +02:00
860e47a256 Add functionality to flush the internal cache
Before we had the problem that when iterating over _a lot_ (like 5k)
entries and also fetching them, at some point the OS would return with
"Too many files open".

That is because the store internally caches a lot.

With this change, the Store gets an API to query how big the cache is,
how much the cache can currently hold and (and that's the main thing in
this patch) to flush the cache to disk.

A function to simply ask the store whether its cache should be flushed
(which would us require to ask the OS how many files we can open...
which would be possible with `libc::getrlimit`) does not yet exist,
though, but could be added easily if desired.
2018-04-30 18:39:59 +02:00
b3cf96d88b Refactor error handling, use more error chaining 2018-04-30 17:29:27 +02:00
fc92cfc36f Move header filters to FailableFilter 2018-04-29 11:43:32 +02:00
30ac77d626 Add error module 2018-04-29 11:43:32 +02:00
e08ec34893 Remove empty module 2018-04-29 11:43:32 +02:00
6d21d8eb49 Update dependency: filters: 0.2 -> 0.3 2018-04-29 10:46:36 +02:00
0976cab014
Merge pull request #1464 from matthiasbeyer/libimagentrygps/32-bit-gps-values
libimagentrygps: 32 bit gps values
2018-04-29 00:35:21 +02:00
93c3bb5862 Upgrade variables to 64 bit 2018-04-28 23:27:12 +02:00
5870fa0785 Fix: Use 16 bit variables for GPS degree/minute/second 2018-04-28 23:18:02 +02:00
5f3b7b31e7 Add support for storing email properties
We have to move the `Email` type at the bottom of the DeserVcard type
because it contains a table and we must emit tables as last values when
de/serializing.

Maybe this will shoot us in the foot later, but only with TOML I guess.
We'll see. For now, this is good.

For that we need to update a dependency: vobject -> 0.5
2018-04-27 16:57:37 +02:00
82867f3ff6 Fix tests for new interface 2018-04-27 13:24:59 +02:00
f0969db47c Rewrite
This patch rewrites the whole libimagcategory and brings it to a nice
standard (the code before was rather messy).

Now, categories are represented by an entry in the store and all entries
which have this category are linked to that entry.
2018-04-26 12:05:28 +02:00
9fc9e7fe17 Outsource iterator in iter module 2018-04-26 12:05:28 +02:00
ab15d89e64 Rewrite CategoryRegister as CategoryStore 2018-04-26 12:05:28 +02:00
5f305ef7a7 Add module for category handling 2018-04-26 12:05:28 +02:00
c26f1aae2a This module should be named "entry" 2018-04-26 11:20:40 +02:00
f321aa82a1 Make sure category is linked when setting it 2018-04-26 11:20:09 +02:00
0bacfb0a88
Merge pull request #1446 from matthiasbeyer/travis-clippy
Travis: clippy
2018-04-26 10:25:45 +02:00
ea80a5a09b Derive Default for InMemoryFileAbstraction, remove constructor 2018-04-25 19:15:07 +02:00
6a34e7a8fd Derive Default for FSFileAbstraction, reduces boilerplate 2018-04-25 19:15:07 +02:00
bf363c4748 Use "if let" if matching on a single pattern 2018-04-25 19:15:07 +02:00
f1142c414d Use ok_or_else() instead of ok_or() 2018-04-25 19:15:07 +02:00
22170a0d55 Do not String::from(String) 2018-04-25 19:13:51 +02:00
6f5590713a Use single quotes here 2018-04-25 19:13:51 +02:00
44521a7c8c Dont use "return" keyword at end of function 2018-04-25 19:13:51 +02:00
dc18e650bb Collapse nested if-else-if 2018-04-25 19:13:51 +02:00
67675b2ab2 Remove unecessary clone 2018-04-25 19:13:51 +02:00
a25f650ca0 Be more ergonomic here 2018-04-25 19:13:51 +02:00
84249e3fb5 Be less verbose when constructing an object
The rust compiler does some fancy things for us: It automatically finds
the right fields if the name of the variable and the file is the same.

Lets use that to reduce boilerplate with this patch.
2018-04-25 19:13:50 +02:00
154c2e482b Change generate_variants() helper to use base by ref 2018-04-25 19:13:50 +02:00
42c4186dec Use or_else() instead of or() 2018-04-25 19:13:50 +02:00
80c07171aa Make sure we can deserialize
Use default values when field is not there when we deserialize. This
does automatically the right thing.
2018-04-25 13:45:00 +02:00
56c1b78d93 Add getters for DeserVcard helper type 2018-04-25 13:45:00 +02:00
abc142f4b5 Rewrite library
libimagcontact stores all contact information in the entry header now.
2018-04-25 12:54:01 +02:00
3eab3af7b0 Remove all libimagentryref usage 2018-04-25 11:30:42 +02:00
775508a6ed Refactor: Use AsRef<str> instead of String as arg type 2018-04-24 23:12:14 +02:00
488ce9fe91 Clippy fixes 2018-04-24 23:12:14 +02:00