Filters "composite of composite" buggy?

[Maccara]

Hi,

"Composite of composites" does not seem to work right. (using v2.2.3 right now - haven't tested in earlier versions)

Example:
author1 - author - contains "foo"
author2 - author - contains "bar"
authors - composite - author1|author2

subject1 - subject - contains "xyz"
subject2 - subject - contains "zyx"
subjects - composite - subject1|subject2

kill - composite - authors|subjects

I have made the "kill" filter also as a view filter so I can see the results, and indeed it will only take the rule which is first before the boolean or and ignores the other.

I.e. I get different results depending on if I define authors first and different when I define subjects first - the second part is always ignored.

Of course, I have tested that these filters work individually, and they work if I define them all in a single composite (and that is a workaround for now, but becomes quickly difficult to manage for me - I like structure and this would make it easier to manage) but composite of composites breaks.

Edit: I think I understand what happens here, but of course I can't be certain without having the code myself. There indeed seems to be a bug for nested composites whereas they always will behave as boolean AND operators effectively.

What probably happens is it looks at the first term and then applies the latter term what is left after that, whereas with an OR operator it should really look at the items which were already filtered out with the first term.

I once wrote a similar bug myself.

[alex]

i think it worked initially, but probably i changed something when fixing rare crashes a long time (maybe couple of years) ago, i remember there was something related to filters.

i'll check the code, it will work in the next release, should be something very small.

it doesn't apply as composite, it rather combines everything it into a first level composite filter, so something goes wrong in the process, maybe it misses to add the operator between the composite filters when parsing.

[Maccara]

i think it worked initially, but probably i changed something when fixing rare crashes a long time (maybe couple of years) ago, i remember there was something related to filters.

i'll check the code, it will work in the next release, should be something very small.

Thank you very much!

it doesn't apply as composite, it rather combines everything it into a first level composite filter, so something goes wrong in the process, maybe it misses the operator.

I was actually wondering how it was implemented internally (for example, if it would be very very important to group/arrange operators correctly, like in case of just a "dumb" recursive "flatten" etc) and therefore tested different ways of combining stuff and noticed it almost always breaks with an OR operator.

What you wrote above, tells me much about the inner workings, thanks.

[alex]

it also processes nested expressions, it is like a simple recursive parser, it cannot be wrong in principle, maybe it just forgets to add something in the middle when adding those simple filters together.

i'll reduce it to a simpler case and check it in debug.

[Maccara]

As a side note, would it be possible to implement NEAR operator sometime in the future too?

(preferably in some form of "near/3" (i.e. user can specify the scope also))

Would help simplify some predicates, where a wildcard is too much and a phrase isn't enough...

[alex]

ok i fixed it here, the problem was it also added terminating character when it wasn't supposed to.

so it didn't work from the beginning, just noone used composite of composite in the past.

i'm not sure about extending the wildmat code further, but there are some extensions of wildmats here added:

http://www.netwu.com/ue/help/patterns.htm#bw

[Maccara]

Heh, extensive filtering possibilities was one of the main reasons I bought this, so that was something I noticed quite fast (once I got the learning curve).

Thanks for the superbly well support!

[alex]

i consolidated all changes as v2.3, you can download it now from the website, otherwise the release notes started looking too messy.

just there was some work in parallel on internals (not filters) and it appears now i mostly achieved what i wanted.

[Maccara]

Seems to be working fine now!

Tested with some relatively complex filtering nested 3 levels deep via composites.