Usenet Explorer

Here is my servers config:

1- news.astraweb.com
Priority: 1
2-new.demon.nl
Priority: 100
3-newszilla.xs4all.nl
Priority: 50 S <-Strict
MSG ID Direct enabled in all of them.

What I observed:
I wanted to download something from a.b.dvdr. It gets the headers from astraweb or demon (newszilla doesn't have a.b.dvdr), then it tries to download from Demon, and it also tries in newszilla.xs4all.nl, then it fails (of course) and instead of waiting to try again in Demon, it starts downloading in astraweb, what I DO NOT want.
(I also observed in groups that newszilla has, but for any reason it fails to download the msg).

So, to summarize, the program tried to download from the server with lower priority(50) when the highest priority server (100) was busy with max number of tasks (this is ok, we want that), BUT it failed and then it started the task in the server with priority (1) (we DO NOT want that since the server with highest priority HAS the body of the msg).

Is there a way to solve this ?
Thanks !
Marcello

the purpose is to get the body of the message, if all failed it newspro will try to download a message from a server with a lower priority, if you define astraweb as message-id direct it will be always the case when everything else failed.

if you don't want to download bodies from astraweb at all download the latest update in the development section, it allows to instruct newspro not to download bodies from a server.

I understand that, but not "all" failed. Only the server with priority 50 failed !
The server with highest priority (100) didn't have the chance to try that body...

Explaining again: The high server (100 - Demon) has 3 tasks limit...When it reached this limit, it tried the server (50 - Newszilla). But it doesn't mean that Demon doesn't have the body of that msg.
Then the server Newszilla fails.
Instead of requeueing to Demon (100) to see if it would fail getting the body, it goes to astraweb.

So,the problem is: the fact that the task limit was reached, it doesn't mean it would fail to get the body ! That's my issue.

Is it more clear now ? (Or maybe I'm missing something... ?)
Thanks.

newspro doesn't have memory about servers which had all task slots busy in the last time when the download article task had been launched.

say, it tried newszilla, it doesn't have the article.

then it excludes newszilla from further attempts, so if afterwards the demon server has tasks slots available - newspro will use it since it has higher priority, only if newzilla doesn't have task slots - newspro will use a lower priority server, which in your case is astraweb (since newzilla doesn't have the article).

if you will mark demon 'strict' newspro will wait until demon has available task slot (you can also assign newzilla and demon the same priority and mark it strict for both if you want the servers to be of the same preference).

it is just some kind of scheme, logic, if you can think of something better (the only condition is it should be fast to calculate) i could consider it.

You said:

"then it excludes newszilla from further attempts, so if afterwards the demon server has tasks slots available - newspro will use it since it has higher priority"

That's the ideia, but this is exactly what is NOT happening to me.
If Demon had no task slot available, it tries then with Newszilla (and fails) then it goes directly to astraweb instead of afterwards trying Demon.

Maybe I'm doing something wrong or missing something ?

Once the task fails on the server that is set to strict, the program will try to get the article on other lower priority servers. It is working properly. Think of how you want the program to behave between the top 2 servers on your list. You want tasks opened for the top priority server until its max is reached, then additional tasks on the 2nd one until the overall max is reached. Most of your downloads will come from the top server, but some will come from the 2nd, and you'll have better overall speed. Now, set the strict flag on the top server, and you'll not get any tasks from the 2nd server until they fail on the first.
Keep this in your mind, go back to your server setup, and visualize what will happen if you disable the 2nd server . . . . which is essentially the same on a task by task basis for a failure . . . . The program is now free to check the lower priority server. This is the only way it can work.
The only way to do what I think you're trying to do is set your top 2 servers to the same priority and set both of them to strict. That way, the lowest priority server won't be touched until the article has been tried on both of the higher priority servers. It will also allow your maximum number of tasks to be run between the 2 top servers (I expect you didn't set the top server to strict because you tried it and saw it wouldn't open any tasks on server 2 until the articles failed on the top server, this will alleviate that).

marcellos:

i meant the same thing as edfreeman - set both higher priority server to strict with the same priority.

it cannot be in your situation that newspro won't try demon first if there are available task slots, check console. maybe it didn't try since all happened fast so task slots of demon still were all taken, i guess it is a bit difficult to create such an situation since 'no such article' is returned pretty fast (to reproduce such sequence temporary set server connect delays on every server - e.g. 60 seconds and disable keep alive, wait until tasks on demon are near end and then launch the task, so 'no such article' will take at least 60 seconds and within the time you'll have slots on demon).

newpsro doesn't work like you described, after newzilla is excluded because the article is not there (newspro launches task and gets 'no such article') all the story starts over again so all available servers will be tried, since there were no actual task on demon launched - demon retry count doesn't change so no reason why demon won't be used later if possible. newspro has Alzheimer as to everything but servers that irreversibly failed (task launched and the error is not retriable), no task - no error.

ok so :

demon 100 (max 3 tasks)
xs4all 50 Strict (max 3 tasks)
astraweb 1 (max 8 tasks)

now i go downloading 1000 articles
i want to download 6 at a time normally (3*demon (each 50kb/s) and 3*xs4all (each 15kb/s)

sometimes xs4all has problems, (1 day delay in receiving new postings)
so it is bussy downloading :

3 tasks from demon
now it will try to open 3 tasks to xs4all, these 3 tasks fail

now what does this Strict mean with xs4all ?
i though it means 'only drop below priority 50, if it's not on 50 or higher'

but it looks like it means 'only drop below 50 if it's not on this server'

so if xs4all is having a problem it will download :

3 tasks from demon
and 8 tasks from astra

so at the end it will be +/- 270 articles from demon (3 tasks)
and +/- 720 articles from astra ? (8 tasks)

i was thinking about a solution :

you can create a server subset and put the demon and xs4all server inside this subset

now it would be nice to put an Strict to this subset (instead of putting it to the individual servers)

take xs4all out of the equation altogether for your scenario, since it won't have any of the articles. The program will do 3 tasks from demon and a number of tasks from astra to get up to the maximum number of tasks you have set. Remember, you have no strict flag set for demon. The strict flag is per server, not per priority level.
If you set the strict flag for demon, astra won't be touched until an article fails there. Unfortunately, neither will xs4all. In the scenario you have set up, you'll get 3 tasks from demon only. xs4all will only open tasks that fail on demon. Then, when the tasks fail there also, astra will be opened. It is quite likely that astra will not see much action provided demon is fairly complete. But, download speed will be limited to what you can get from 3 demon tasks.
If you set demon and xs4all to the same priority and both strict, you'll get 6 tasks while it tries to download the articles. Of course, xs4all will fail, but all the articles will be tried on demon before it goes to astra. This isn't any better speedwise than the previous paragraph, but that's primarily because you've said none of the articles will be present on xs4all. Under normal circumstances, where most of the articles will be on all 3 servers to varying degrees, you'll have 6 tasks total, and astra won't be touched until articles fail on both demon and xs4all.

All you and Alex said is true, but untill now I didn't really see an option to do as I want:

Demon - Fast / higher priority
Newszilla - Slower / average priority
Astraweb - Expensive / Only download here IF BOTH servers above do NOT have the body.

The initial question is: "Is there a way to solve this ?"

So the answer is:

Yes - How....?
or
No - Newspro won't do it.

Thanks again!

Demon : priority 100, strict
Newszilla: priority 50, strict
Astraweb: priority 25, strict

Have fun...

The answer lies above, my friend. And it depends on how you want Newzilla to behave. If you don't mind downloading some articles from Newzilla, set both Demon and Newzilla to the same priority and set the strict flag for both. Make sure your max tasks will allow enough for both of them (ie - if the max for each server is 3, set the max tasks to 6). The program behavior will be to download from both of them (Demon will get more, since it is faster), and only touch Astra when an article fails on BOTH.
If you do mind downloading articles from Newzilla that are also present on Demon (or, if Demon will fill up your connection bandwidth), set the strict flag on Demon and leave the priorities where they are. The program will download all available articles from Demon, check Newzilla and get any missing ones it can from there, then use Astra only for those parts missing on the first 2.
Either way, with the strict flag set on both Newzilla and Demon, Astra won't be touched until the articles fail on both.

Unfortunately the answer doesn't lie above. Setting strict to Demon will not allow Newszilla to get things if the task slots from demon are full.

It looks like the answer is: Newspro won't do it.

Edfreeman, your workaround is not too bad (setting both with same priority) but now actually I'm using another solution that seems to work a little bit better:

news.demon.nl :
- priority 105 (NO strict) / 2 tasks limit

corp.supernews.com (another name for news.demon.nl)
- priority 100 Strict / 1 task limit / Supress header download
(This way I still keep my 3 tasks limit for Demon)

newszilla
-priority 100 Strict / 3 tasks limit

astraweb
-priority 1 / 5 tasks limit

I guess this works pretty much like I wanted untill someone comes with a better idea (or implements something like DDX said or use the strict "per priority" and not "per server")

Demon : priority 100, strict
Newszilla: priority 100, strict
Astraweb: priority 25

what is the problem with such setup?

even if newzilla is slow you may wait only for very last parts when demon finished and newszilla is finishing.