Hi all,
I've started to have an issue with UE downloading multiple copies of a file that has the "430 No such file on server error".
For example I used the Integrated Indexing (super search) to find season one of "The Wire" - imported the result and scheduled the download (about 8 GB). When I checked this morning UE had downloaded about 30 copies of EACH file that had some missing lines... The result was a folder with almost 55 GB of files with up to 30 copies of each "broken" file - each with a (2), (3), (etc...) suffix and the same exact (and incorrect) size. (this seems to be a function of the "retry failed tasks" timer which is 20 mins in my case -- 10 hours of downloading = 30 retries?)
I am using Giganews on 443 without the accelerator as my only server and have retries set to "2" in server properties. Also Msg-ID direct set on server tab and "Request Article Method" on Articles tab.
The trigger is definitely the "430 - No such article error" - which might be understandable with imported search results BUT last night I downloaded headers and tagged a complete (i.e. all headers present) HD movie for download and about 8 of the files received this error and were downloaded about 30 times each ( 110 MB files x 30 x 8 = 25 GB of bad files!!! all with the (2), (3), (4) etc etc suffix.
I am suspicious of Giganews, I know Alex has incorporated a filter for special handling...
I have tried various settings re: Request Article Method with no luck.
Any ideas? Another night of 145 GB of downloads when I only tagged 20 GB is a complete pain in the a**.
Any more info needed I am happy to post - I hope this makes sense & I hope (strangely enough) that I am not the only one...?
Thanks for any help!
No such article error and duplicate file auto-rename
Duplicate file names is one thing, but "no such article" or more exactly "430" is not "recoverable" error, thus UE moves it to errors pane right away.
Thus maybe you set "retry failed tasks" in properties->tasks in addition to auto rename for duplicate file names in properties->save/unpack, save.
So the article migrates to the article "errors" pane (red question sign tab pane), then "retry failed tasks" causes it to go back to the articles pane.
Maybe this is the solution:
"No such article" error is a special error which has a dedicated option enabled by default in properties->save/unpack, "Ignore no such article errors". When it is checked - UE saves the incomplete article body for the purpose of par2 recovery.
If you uncheck the option - UE won't save, but then sometimes you may need to save those article bodies manually when there is not enough data for repair - just use "save attachments" instead of "download&save" you can do it also directly from the article "errors" pane context menu.
If there is enough data for repair and subject is intelligible - UE will clean up (also when set in options - deleting bodies of) the relevant entries in the error pane after the successful repair.
You may check those cheap alternatives on the UE web page, the complete posts index came from them, I don't use giganews for search service indexing (too expensive, 3 times more for the same or worse header feed quality). I will also take a look if I can make the program more sofisticated in this regard.
Thus maybe you set "retry failed tasks" in properties->tasks in addition to auto rename for duplicate file names in properties->save/unpack, save.
So the article migrates to the article "errors" pane (red question sign tab pane), then "retry failed tasks" causes it to go back to the articles pane.
Maybe this is the solution:
"No such article" error is a special error which has a dedicated option enabled by default in properties->save/unpack, "Ignore no such article errors". When it is checked - UE saves the incomplete article body for the purpose of par2 recovery.
If you uncheck the option - UE won't save, but then sometimes you may need to save those article bodies manually when there is not enough data for repair - just use "save attachments" instead of "download&save" you can do it also directly from the article "errors" pane context menu.
If there is enough data for repair and subject is intelligible - UE will clean up (also when set in options - deleting bodies of) the relevant entries in the error pane after the successful repair.
You may check those cheap alternatives on the UE web page, the complete posts index came from them, I don't use giganews for search service indexing (too expensive, 3 times more for the same or worse header feed quality). I will also take a look if I can make the program more sofisticated in this regard.
OK, I think I understand now. 
-> 
= 
I did have Duplicate file names set to "Auto-rename" & "Overwrite" for the Save & Unpack option - now I have both set for "overwrite". The "Retry failed tasks" option is set to 60 mins now in props->tasks.
The migrating articles being reprocessed (circular processing?) seems to have been the issue - thank you Alex for the explanation. I have experimented with "Ignore no such article errors" unchecked and the behavior is as you describe - bodies are retained "in line" and a manual save allows completion and PAR repair.
Is it correct to assume that the "Auto-rename" option is a more complex variable than "overwrite"? I can understand the benefits re: poorly labeled duplicate named posts - I have turned this feature off now & will monitor for any negative impact. With Giganews in particular, the circular downloading trap is a far worse fate...
I sympathize with your frustration re: Giganews - what an odd way for an error to report when it is in fact "mostly" available and not at all "no such article"? I will try one of the suggested servers for a change - afterall, how will giganews learn if I keep paying them for a poor service.?
I have added the "Delete headers" option as well as "Log saved attachments" and this helps tidy up and track activity very well.
Last nights downloads worked very well and the desired files are all repaired and complete - no extra files whatsoever.
Thanks for the reply and help Alex and Jonib - most informative and insightful.
Alex, you are a relentless - I have great respect for your drive!
As a footnote, it occurred to me that some form of "export settings log" might be a useful feature? Perhaps a txt based dump of all the UE variables ( except logins & pwords of course).
Again, thanks for the assist - all systems working very well & UE rocks!
-> = I did have Duplicate file names set to "Auto-rename" & "Overwrite" for the Save & Unpack option - now I have both set for "overwrite". The "Retry failed tasks" option is set to 60 mins now in props->tasks.
The migrating articles being reprocessed (circular processing?) seems to have been the issue - thank you Alex for the explanation. I have experimented with "Ignore no such article errors" unchecked and the behavior is as you describe - bodies are retained "in line" and a manual save allows completion and PAR repair.
Is it correct to assume that the "Auto-rename" option is a more complex variable than "overwrite"? I can understand the benefits re: poorly labeled duplicate named posts - I have turned this feature off now & will monitor for any negative impact. With Giganews in particular, the circular downloading trap is a far worse fate...
I sympathize with your frustration re: Giganews - what an odd way for an error to report when it is in fact "mostly" available and not at all "no such article"? I will try one of the suggested servers for a change - afterall, how will giganews learn if I keep paying them for a poor service.?
I have added the "Delete headers" option as well as "Log saved attachments" and this helps tidy up and track activity very well.
Last nights downloads worked very well and the desired files are all repaired and complete - no extra files whatsoever.
Thanks for the reply and help Alex and Jonib - most informative and insightful.
more ...!?!I will also take a look if I can make the program more sofisticated in this regard.
Alex, you are a relentless - I have great respect for your drive! As a footnote, it occurred to me that some form of "export settings log" might be a useful feature? Perhaps a txt based dump of all the UE variables ( except logins & pwords of course).
Again, thanks for the assist - all systems working very well & UE rocks!
You can continue to use Giganews with the addition of a cheap block account server as a backup and then no repair is needed. Giganews must remove all of the articles that are listed in a DMCA takedown notice, they have no choice. Lucky for us downloaders that the law firm which is sending out the notices on behalf of HBO sees fit to list only the first multipart of each file. This is enough to cause GrabIt at least to come to a screeching halt.
all providers are running the same open source diablo software, maybe with few modifications, the cost of storage dropped considerably, as the result the completeness appears to be the same and retention almost the same across a number of different providers. a fills account maybe would be only needed to complement a poorly maintained free ISP news server.
it appears some usenet providers get more take down notices than others. second thought is if there would be a single usenet provider, it is the most vulnerable scenario, usenet needs true server diversity and only users can make it possible.
i cannot tell anything more as to server completeness since this is beyond my control, all what i know for sure - in the next release the technical side will be resolved, namely when the duplicate mode is skip or rename - the option won't be enabled in properties since it wouldn't save again if skip or would fall into multiple rename when rename mode. when overwrite mode - it will work in the best logical way possible which means it won't save the same file again unless there are new parts downloaded.
later: resolved in v2.6
i cannot tell anything more as to server completeness since this is beyond my control, all what i know for sure - in the next release the technical side will be resolved, namely when the duplicate mode is skip or rename - the option won't be enabled in properties since it wouldn't save again if skip or would fall into multiple rename when rename mode. when overwrite mode - it will work in the best logical way possible which means it won't save the same file again unless there are new parts downloaded.
later: resolved in v2.6
Re: No such article error and duplicate file auto-rename
From few months; I was facing speed problem on my PC and wanted to remove the same date for multiple locations. I was searching for software to find duplicate files. Then I found DuplicateFilesDeleter website and come to know that DuplicateFilesDeleter is the best software for finding and deleting duplicate files. I use this software and found it same as they mentioned on their site. It is really effective. You can get unique and authentic data by using this software.