Yesterday I got myself a professional grade NAS ( thank you) and I noticed that UE behaves strangely ever since.
For example it downloads and saves files in the NAS, checks them and if necessary repairs them against the PAR2 files (although now I don't see the PAR files anymore) and then it just tags the entry as "Queud for unrar" endlessly and moves to the next download and so on.
Now since this behavior started only when I got the NAS i am wondering if there is anything I overlooked or that I should do.
Yesterday I got myself a professional grade NAS ( thank you) and I noticed that UE behaves strangely ever since.
A NAS should not change anything in UE unless you made a new database and got the default settings.
(although now I don't see the PAR files anymore)
UE by default deletes all PAR2 files after a check/repair, that can be changed in MENU->Edit->Properties->Save/Unpack->Unpack settings.
and then it just tags the entry as "Queud for unrar" endlessly and moves to the next download and so on.
Do you have the "Queud for unrar" showing even after all tasks have finished?
If no, then you can change the save/Unpack priority in MENU->Edit->Properties->Tasks->"Relative chance to run"->Save/Unpack setting.
I've seen similar behaviour with my NAS. I can't remember if I tried downloading directly to the NAS with UE (but I probably did at some point - it was a while ago I got it). What happens if you save files to the NAS from the web with a multithreaded downloader, e.g. Firefox and DownThemAll? I find the downloads appear to work fine then at 100% completion they fail. When saving files with one thread, as with the normal FF dialogue, everything saves fine. I think one file with many threads working on it can be problematic on a NAS. Databases with multi user accesses can work with no problems but databases are a different beast altogether.
I would recommend you don't download directly to the NAS itself anyway. Unless your NAS, router and computer all have GigaE then you will easily saturate your bandwidth. This means that any UE downloads, repairs and/or unrars will all suffer if two are in progress since everything is done over the network - downloads, repairs and unrars. Plus, if you were to browse the web in the background you would see a major slowdown there as well.
On mine, I download to my computer, let UE repair/unrar everything there and manually move things over later. Speed and stability-wise it's a safer bet. If not that, change UE's options to unrar to an absolute directory somewhere on the NAS. The thing to remember is that anything to do with networks can be flaky. Are these download locations shares mapped to drive letters or are you using network paths?
Another option that I use is to create TrueCrypt containers on the NAS, mount them and save everything there. At least you will have pseudo 'real' drives to save to and they will have proper drive letters, not to mention they will be encrypted.
if tasks are queued for unrar but no unrar is performed it should mean the unpack queue is paused (then in the context menu in the unpack pane there is the unpause option).
if the storage behaves in a weird way it is difficult to tell what it is unless i can see it myself, queued for unrar would mean the file is ok while being saved, the files are verified during the saving process, if the storage malfunctions (like write operations go through without any error but what is written to the disk is different) UE may think the fies were saved ok since there is no reason they are not, then you can e.g. select those items which fail to unrar and invoke from the context menu "advanced->reload exact files" to synchronize them with the storage content, but if it is the case the question is why to use such unreliable storage in this manner.
alex wrote:...the question is why to use such unreliable storage in this manner.
Remember that a NAS will incorporate its own OS and its own file system. The way Windows interacts with the storage space on the NAS will be different to simply saving straight to a hard drive. The storage is solid but the method of transferring the data may not be perfectly ideal.
This is why I mentioned a TrueCrypt container. It would make available, to all intents and purposes, a Windows volume on which you could perform chkdsk, defrag it, etc. that you couldn't do on a NAS.
I download to a NAS and what I experience is that UE will occasionally corrupt a rar file while writing it out. Since UE verifies the file before writing it out, it doesn't think there's anything wrong with it and then doesn't bother downloading the par2 files. Then when UE tries to unrar, it fails and then just sits there with it in the queue.
What I do now is not use the unpack feature of UE and use PAR Buddy to do all of that for me.
thanks for the clarification, so then it is reduced to what i talked about in my first reply.
first of all UE cannot write a corrupted file in the sense if a write operation fails exception will be thrown, then UE will suspend the save queue and give the save error message box also citing the exact system error message.
it doesn't write without checking for errors, every read or write error will result in an exception.
but what it means then the NAS storage cannot be considered as a reliable storage, since something can be written into it corrupted without raising any error condition.
the solution would be then a check box which would guide UE not to mark file as complete but rather to queue it for verification. what also should be ensured the system doesn't cache those files which may well be the case (then any program won't detect the corrupted write as well). with no caching it will be a drain on resources (local network bandwidth).
but you might still the same problem later if the file is corrupted and repair would also result in corrupted file so in such cases you would need to invoke repair repeatedly until it succeeds, also at best no difference from any other par2 stand alone repair program.
in short such kind of option would be needed if we are talking about unreliable storage, although it cannot be complete solution whatever method you are using.
The corruption may not be the fault of UE, but I've never had another program cause corrupted files when writing to the NAS (granted, I haven't tried that many other programs). It also happens when UE writes to my linux file server using Samba.
The option to queue all files for verification would be awesome as then I wouldn't have to use PAR Buddy anymore. I would be more than happy to test that for you if you decide to implement it.
what antivirus/firewall are you using, maybe we find some correlation here?
what is maybe not typical - UE saves from a single, but separate worker thread - not from the main interface thread.
UE has its own limited write bufferization, so a single write operation is minimum like several tens of kilobytes, it writes sequentionally write after write without any seeks, so very standard behaviour apart from using the separate saving thread (the thread is also engaged for unpack to prevent too many disk operations in the same time and to simplify synchronization).
i could add the check box to verify files after save (by not flagging them as verified when on fly check indicates the data written to the disk are ok) maybe it will fix this issue, so there will be need to "reload" the files manually, if par2 repair will fail because of the same issue you could retry it again, anyway it may fail then in the same way with any other par2 repairer.
can you reproduce it easily? i could put a test version with the option to see if it makes any difference.
alex wrote:what antivirus/firewall are you using, maybe we find some correlation here?
I don't use any active antivirus on the PCs involved here. For firewall, the downloading PC has the WinXP firewall active, but I'm pretty sure I've tried turning it off already. The NAS and linux box don't use an active firewall.
alex wrote:can you reproduce it easily? i could put a test version with the option to see if it makes any difference.
Very easily . It happens pretty much every time I download a set that is 700MB or more.
alex wrote:first of all UE cannot write a corrupted file in the sense if a write operation fails exception will be thrown, then UE will suspend the save queue and give the save error message box also citing the exact system error message.
it doesn't write without checking for errors, every read or write error will result in an exception.
A question that semi-relates to the above.....
I've noticed that when UE chacks the PAR2 files, it will repair the files fine EXCEPT in the case of xxxx.nzb or file.diz type files. If those are missing or need to be renamed, etc., UE doesn't handle them. I would assume that UE would handle all the files in the PAR2 set regardless. Is this a programming issue or something to eventually be added?
jmbailey2000 wrote:I've noticed that when UE chacks the PAR2 files, it will repair the files fine EXCEPT in the case of xxxx.nzb or file.diz type files. If those are missing or need to be renamed, etc., UE doesn't handle them. I would assume that UE would handle all the files in the PAR2 set regardless. Is this a programming issue or something to eventually be added?
If I'm not mistaken, PAR2 has support for including some files that it only checks but can't repair, often small or non critical files, maybe this is what you are seeing?
jmbailey2000 wrote:I've noticed that when UE chacks the PAR2 files, it will repair the files fine EXCEPT in the case of xxxx.nzb or file.diz type files. If those are missing or need to be renamed, etc., UE doesn't handle them. I would assume that UE would handle all the files in the PAR2 set regardless. Is this a programming issue or something to eventually be added?
If I'm not mistaken, PAR2 has support for including some files that it only checks but can't repair, often small or non critical files, maybe this is what you are seeing?
jonib
If I run the PAR2 file outside UE, it corrects the misc. files no problem. Usually a rename or a creation. Strange.
thank you) and I noticed that UE behaves strangely ever since.
. It happens pretty much every time I download a set that is 700MB or more.