John-Mark Gurney
ef95e9f03f
prep for decay, add support for limit and refactor tests..
1 year ago
John-Mark Gurney
108d1dff3d
fix an issue with large work counts...
My testing machine has 10 cpus, and so didn't trigger the failure
where not all the work was submitted.. We need to pop the completed
work items, and keep doing the for loop while we have futures to
process... this submits and processes all work..
1 year ago
John-Mark Gurney
7212192801
use threads to read/hash multiple datablocks at a time..
as we might have a lot of work to submit, BUT it might fail early,
don't submit too much work early on, just for it to fail, so we
limit how much work we submit..
1 year ago
John-Mark Gurney
5ee796735b
add a class for hashing pieces in order...
This will be used to allow parallel processing of torrent pieces..
each piece of the torrent can be processed in parallel, and this
class will make sure that when processing the hash of a file in
the torrent, it will be hashed in the correct order...
1 year ago
John-Mark Gurney
df16a93fe1
don't call a function excessively...
setdefault works best for simple data types, calling the factory
function every time doesn't make sense...
1 year ago
John-Mark Gurney
f215c73705
a few notes, spell not in correctly..
1 year ago
John-Mark Gurney
000bf03652
basic docs on how to do development..
2 years ago
John-Mark Gurney
21b3122604
add some useful examples for this project to the template..
2 years ago
John-Mark Gurney
e51b7a62ac
cover a couple missed cases for _archive..
hide coverage in test code
2 years ago
John-Mark Gurney
b75a4d82da
add support for archives, such as tar.gz...
2 years ago
John-Mark Gurney
2c9deb7c58
add some more namespaces, and talk about uri for hashes
2 years ago
John-Mark Gurney
c2ac1018e5
fix issue w/ calculating size of last piece...
2 years ago
John-Mark Gurney
47763953aa
use file size from torrent instead of file, when file size mismatch
2 years ago
John-Mark Gurney
a0f3da4c4c
update debugging to add timestamps for making query times easier..
2 years ago
John-Mark Gurney
51c68c27c0
add useful debug info when a file cannot be found..
2 years ago
John-Mark Gurney
b5214e47a4
support getting file hashes at same time as verification...
2 years ago
John-Mark Gurney
33ea645f1d
make sure we iterate through possible returns from rglob
2 years ago
John-Mark Gurney
6c3c694f71
apparently shared cache is discouraged, and doesn't make sense anyways
2 years ago
John-Mark Gurney
f4fd04ec10
if container is already present and complete, skip it..
2 years ago
John-Mark Gurney
aeec98115e
bump cache to 15 entries..
2 years ago
John-Mark Gurney
f7548e6c0c
add comments to tests, make sure modifying count counts as modifying..
2 years ago
John-Mark Gurney
4761f7f43f
allow the tag cache count to be modified...
2 years ago
John-Mark Gurney
17453c9b8c
w/ sqlite3 objects aren't exactly the same, use == instead
2 years ago
John-Mark Gurney
725f32102b
add code to properly dump os.stat_result's..
2 years ago
John-Mark Gurney
1267f53d3a
Merge branch 'main' of ssh://gold.funkthat.com/home/jmg/git.repos/medashare
2 years ago
John-Mark Gurney
92bfafa077
make tests pass on FreeBSD.. implement hostuuid for it...
Hard code stat results so that the tests pass on a different checkout
w/ different mtimes..
2 years ago
John-Mark Gurney
712ceeca74
add script to autogen, it's a hack by creating/upgading a db, but works
Looks like this is the only way:
https://github.com/sqlalchemy/alembic/discussions/1086
2 years ago
John-Mark Gurney
4ddccd6fbd
no cover the test code and a few other places...
2 years ago
John-Mark Gurney
e1499cc30e
covering this doesn't make sense, will never "downgrade" to empty db
2 years ago
John-Mark Gurney
c0b475e1d1
move tag cache to it's own file..
2 years ago
John-Mark Gurney
e7f9cdc846
add tests for the TagCache, fix a bug, docs, drop unneeded sorted..
2 years ago
John-Mark Gurney
ab7d88f6d3
allow browsing dirs as well, make the displayed list work better
2 years ago
John-Mark Gurney
5006cb622f
drop unneed arg to ObjectStore, add type column via migration
add tests for migration, good example for future needs..
2 years ago
John-Mark Gurney
633a957bbc
change the ignore to match the correct extension.
2 years ago
John-Mark Gurney
0a036e092c
switch to edgold from cryptography.
We were only using the ed448 code, and cryptography requires rust
to compile, and contains a LOT of other code, while edgold is much
smaller, and doesn't require a rust compiler, and will work on more
systems.. Keep around an old identity generated from cryptography
to make sure the two libraries are compatible.
2 years ago
John-Mark Gurney
39a0ef6d50
add database migration so that the db can be more easily updated now..
2 years ago
John-Mark Gurney
233b9fe865
needed to make setup.py work..
2 years ago
John-Mark Gurney
8dbbdd2406
fix interactive mode, use environment var to change location
2 years ago
John-Mark Gurney
f8f84632ef
standardize path name fetching...
2 years ago
John-Mark Gurney
dd434d452b
use a decorator to standardize getting the needed structures
2 years ago
John-Mark Gurney
46e88fde76
now that updates are immediately written, name it just quit
2 years ago
John-Mark Gurney
003612d47e
convert the main data store to SQLite3...
one minor issue is that the identity object gets dump'd twice..
2 years ago
John-Mark Gurney
5c3ec4fc78
big update, fix a couple bugs, and add an interactive mode
make sure tags/properties are unique...
support tuples in asn1coder, for the tag cache
make sure that only one of each uuid is stored in hashes, and that
it is the last modified version.
for containers, since we're uniquified them, just replace them..
2 years ago
John-Mark Gurney
dcc46bc962
add comment
2 years ago
John-Mark Gurney
356be8332b
support mappings, that is one host has the same files as another...
this fleshes out hostuuid better..
add a more useful debug print..
2 years ago
John-Mark Gurney
56503fdcdb
add host object, to be used for mappings...
2 years ago
John-Mark Gurney
0d0ae8ead4
make ms:tag unique, drop dups..
2 years ago
John-Mark Gurney
c004e8fdb5
add support for torrents with a single file in them..
2 years ago
John-Mark Gurney
f2db9119e4
print out which container didn't have all it's files added
2 years ago
John-Mark Gurney
e5e22a0361
allow updating the container object...
2 years ago