Search Service showing huge amount of incomplete parts

03 Oct 23: One server was relocated, the server currently doesn't have a public IP address since in the meantime it is running on starlink. I wrote some additional tunnel code running separately to handle this. When the server is engaged the connection is 5600+ days uniform (since around 24 July 2008). In the unlikely case I disengaged the server because of some problem the retention is 1200-3800 days depending on newsgroup. If you experience any issue please let me know.

29 Nov 23: Because of encrypted and obfuscated posts flood mass posted by a few nzb indexing websites using usenet servers as private storage for their members to download the posts - the situation with the content is pretty chaotic since their purpose is to post in such a way, so users must use their website exclusively, when the website disappears - the encrypted posts just eat usenet providers' hard disk space uselessly. If you can't find something specific please let me know at alexbirj at gmail dot com what exactly you can't find for me to check how it is possible to retain the posts. Non-obfuscated posts shouldn't be affected at all, let me know if you notice anything missing.

29 Nov 23: The search protocol had to be upgraded to extend 3 byte (16M+) limit on the server side for number of sets per instance, so at least the version 5.8.3 is needed to access it, otherwise should be no any difference.
Post Reply
victor.alberts
Posts: 7
Joined: Mon Apr 04, 2005 7:15 pm

Search Service showing huge amount of incomplete parts

Post by victor.alberts »

Hi Alex,

How are you doing?

Just wondering, is there currently something wrong with Usenet Explorer's Search Service? Over the past days I noticed a relatively huge amount of incomplete parts, as shown by Usenet Explorer's Search Service. However, when I verified the same posting by using a different NZB index service, the posting was complete. Moreover, I was able to download the complete posting (after importing the NZB file in Usenet Explorer from the NZB index service) without any problems whatsoever, as opposed to when I tried to download the same posting based on Usenet Explorer's Search Service results, in which case there were not enough recovery files available.

Example:
"American Idol S11E06 720p DD5.1 448kbps-ALANiS.ts"

Please find attached two screen dumps plus the actual NZB file I used, which will provide you with further details.

Could you please investigate?

Thanks in advance!

P.S. I am a subscriber to Giganews and Newscene.

Kind Regards,
Victor Alberts.
Attachments
American Idol S11E06 720p DD5.1 448kbps-ALANiS.ts.txt.zip
American Idol S11E06 720p DD5.1 448kbps-ALANiS.ts.txt.nzb (zip)
(163.51 KiB) Downloaded 875 times
Different NZB index service - All Complete
Different NZB index service - All Complete
Different NZB index service - All Complete.jpg (136.87 KiB) Viewed 15252 times
Usenet Explorer's Search Service - Incomplete Parts
Usenet Explorer's Search Service - Incomplete Parts
's Search Service - Incomplete Parts.jpg (183.69 KiB) Viewed 15252 times
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Re: Search Service showing huge amount of incomplete parts

Post by alex »

Maybe it is because of removing one feed 5 days ago, it appears astraweb has also problem with their headers like they have constantly missing article bodies at least in older posts.

I'll post updates in this thread, there is capability to redownload older headers, I need to change something in the code first to prevent too many users messages caused by header feed insonsistencies. I hope to finish all that within two days.
victor.alberts
Posts: 7
Joined: Mon Apr 04, 2005 7:15 pm

Re: Search Service showing huge amount of incomplete parts

Post by victor.alberts »

Hi Alex,

Thanks for your reply!

Just curious, could you please explain in a little bit more detail what's actually causing the reported problem? I understand that you removed a feed 5 days ago, but what kind of feed exactly? Do the currently existing feeds miss certain information, or is there just a mismatch between the provided information from the feeds and the header information sitting on some Usenet servers?

Is there maybe available a temporary solution for me as an end user or should I just be patient? ;-)

Thanks in advance!

Kind Regards,
Victor.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Re: Search Service showing huge amount of incomplete parts

Post by alex »

I don't know, maybe astraweb had temporary problem with headers coming out of sync (gaps in article numbering when fetching headers at too frequent intervals) or it is something more permanent which got exposed when I removed the feed, I'm trying to clarify with them.

hdtv group (your cited posts from that group only?) I've redownloaded last 14 days (I removed failing feed 10 days ago, just added few more days for some case)

I'm also redownloading 14 days for all groups it should be over by tomorrow.

I've also added giganews as additional feed.

If you see something more incomplete let me know.
victor.alberts
Posts: 7
Joined: Mon Apr 04, 2005 7:15 pm

Re: Search Service showing huge amount of incomplete parts

Post by victor.alberts »

Hi Alex,

Great, thanks a lot for your time and efforts you spent on investigating and fixing this issue! Sure, I will keep an eye on all incomplete posts (if any). Nice one that you've also added giganews as an extra feed, now things should only get better! :-)

Kind Regards,
Victor.
victor.alberts
Posts: 7
Joined: Mon Apr 04, 2005 7:15 pm

Re: Search Service showing huge amount of incomplete parts

Post by victor.alberts »

It seems that alt.binaries.e-book.technical also suffered from missing parts.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Re: Search Service showing huge amount of incomplete parts

Post by alex »

check, all should be ok now, i finished the update across all groups.

i solved the problem with astraweb, code i wrote earlier to handle low quality feeds clashed with astraweb way which is sometimes not to return headers after returning increased header range.
tomhock
Posts: 30
Joined: Sun Nov 26, 2006 11:49 pm

Re: Search Service showing huge amount of incomplete parts

Post by tomhock »

This answered the problem I incorrectly posted in the "Support" group, which I have deleted. Thanks very much.
victor.alberts
Posts: 7
Joined: Mon Apr 04, 2005 7:15 pm

Re: Search Service showing huge amount of incomplete parts

Post by victor.alberts »

Hi Alex,

Thanks again for your time and efforts, nice to hear that you've solved the problem! :-)

Have a great weekend!

Kind Regards,
Victor.
Post Reply