was: current work: par2+file manager/indexing server (v2.0)

alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

was: current work: par2+file manager/indexing server (v2.0)

Post by alex »

something along those lines.

the focus is mainly on quality, in part on the optimal performance (when it goes to calculations - at least the same fast as quickpar, but it is like multiset quickpar embedded into newsreader and processing data in the real time), not a cheap solution like running par2.exe or its code on probable sets how it is done in many unpackers, so the implementation will take a little time. also a user driven mode will be possible.

since for par2 we need to track downloaded files, i raised it to the interface level as well, so it may eventually become a kind of file manager.
jaapf
Posts: 203
Joined: Thu Sep 11, 2003 3:06 pm
Contact:

Post by jaapf »

I like the sound of that!
Dutch? Visit the Dutch UE/Newspro forum at: http://www.binaries4all.nl
Nederlandse UE handleiding op http://www.binaries4all.nl/ue
English UE tutorial online at http://www.binaries4all.com/ue/
Handy links at: http://jpfx.zapto.org/
Bert
Posts: 147
Joined: Sun Feb 22, 2004 10:50 pm
Location: The Netherlands

Post by Bert »

Im sure this will be worth waiting for . . .
bassie
Posts: 490
Joined: Sat Jul 19, 2003 9:58 am
Contact:

Post by bassie »

such a feature is always welcome.

Thanks in advance.

:D
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

i've resumed work on the data structures, i realized a redesign is needed as the development is on the basic data structure level with some elements of interface it is easier to do now.

par2 contains ID so par2 files or source files can be dealt with even when being destined to different save locations, but i've decided to make the design with the artifical restriction of par2 file set or any kind of multipart file to be contained in a single save directory, otherwise the interface becomes too cumbersome (still several sets can be in the same directory of course).

we've exhausted v1.x version numbers, so those features will appear in v2.0
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

the basic data structure proved to be inadequate (too abstract, most likely would be difficult to expand later), so i'm readding it from scratch, i still can reuse most of the already written code.

as to the server i wrote a squeezing function (not applied yet) which would be needed if another server retention upgrade would take place, with UE it is automatic but as to the server it is a long procedure, i tested it on a small sample, but i wouldn't risk to run it on the current database (if applied it would decrease server database full backup time), i will do it only on a spare computer when available, i'm not even sure how much time it will take (those are mostly disk read/write operations). even without the function the server database is shrinking gradually to the optimal size, but if to increase retention the natural process might take too long.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

the revised data structure is OK, i'm building on it, most of the prior code has been digested.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

now the work is on the interface (unpack task manager pane).

still there is more work on internals, but existing data structures are sufficient for adding the interface and having visual feedback is always helpful.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

the interface code idea and its connection to the internals is now clear, so i can write the code.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

i've added some basic interface.

next maybe i'll add the repair/unpacking code - since the heavy weight on resources functions will be done in a different thread (in the same thread which saves attachments to avoid too much disk thrashing), synchronization issue is quite important, but to finalize the synchronization model i need to have working functionality which i need to synchronize.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

i'm adding par2 code, had also to add some more interface and synchronization code so i can test what i'm doing.

i'm a kind going for high quality here as to revision technique of the third party code.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

with par2 verification code it is mainly clear, now i'm moving to par2 repair code.

i'm maintaining this thread apart from the doubtful informational value also to evaluate my work dynamics on relatively time consuming tasks in retrospective.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

the repair code is now implemented (most of what will be needed), it is already functional and even more important fully manageable.

i'll take a break from par2 code (it appears i was boiling in it almost 2 weeks) and check unpack (unrar etc.) sdk so we have full function set to play with.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

unrar is ok, since every kind of task is now represented i start putting it all together.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

i've realized that to follow the right implementation order i need to add task manager (article/save queues) to unpack integration first - to know when to trigger unpack jobs , so in a sense the scope is still widening.

this is the part which is not possible with separate unpacker programs.

it is now one month since i had started working on the topic very closely, freezing the rest of the project in the meantime.
Post Reply