Caller ID Superfecta

More playing more information. I have discovered that if I do not use "N" in the rules definitions, and instead only use "X"'s then the rules work properly. I have changed to above rules to the following with very good results:

1902XXXXXXX
1+XXXXXXXXXX
1902+XXXXXXX

This is a very good work around. There is no need really (at least for North America, could be different elsewhere) to check the incoming numbers to the module.

Lorne

I spent (too much) time looking through the code for parsing the CID rules, and it is too far above my meager skills. I have determined that CID rules containing a "+" and an "N" is the combination to blame.

lgaetz - this ticket seems to be emcompassed by a couple of other ones. Should this case be closed - as it is coveredin toher tickets?

tshif wrote:

lgaetz - this ticket seems to be encompassed by a couple of other ones. Should this case be closed - as it is covered in other tickets?

No this is a separate issue, consistant and repeatable at least for me. I spent a bit of time looking but don't see anything obvious. My guess is the problem is somewhere between lines 395-750 in callerid.php

Someone please check Revision 282 and let me know if it fixes this issue. CID matching seems to be working for me now. Before it wouldn't work on more than one rule.

Ok, think I found it this time. Please test revision 283.

I have downloaded and installed r283 version of callerid.php and can confirm that the N+ error seems to be fixed when using the debug feature. I will keep in place for a while to make sure it works properly in the wild and report back.
Good job.

Good to hear.

I've made changes to a few more files as well. It may be a good time to do more exhaustive testing on everything, including the installer and uninstaller.

With regard to other changes you made, is there anything specific I should be watching/testing for? Phones a bit quiet here this morning, but I can confirm that r283 is working great so far.

Probably best to just view my recent comments in the repository

r281 is more of a UI fix -- but before it I couldn't see the description of each source. I'm not sure if this will have issues with any other version of PIAF/FreePBX (I'm on FreePBX 2.8.0.4). It's a work-around for a poor implementation of $(document).ready (JQuery) in FreePBX.
r279 and r280 change the install/uninstall routines. This may be difficult to test out on a production system, requiring you to reconfigure everything to do with superfecta after uninstalling. The upgrade (2.2.3 -> 2.2.4) portion of the installer has not has much testing.

I have some ideas I'd like to start testing out (mainly focused on increasing performance of searching multiple sources), but would prefer to have 2.2.4 stable and released before I start messing around with them.

r281 fixes the popup help bug that tshif has been chasing for a while, he will want to know about that. I can confirm that popup descriptions also work in FreePBX 2.6. I will not be testing r279 or r280 anytime soon. I will put that on my to do list if I setup a non-production machine.

As far as increasing performance goes, you may be thinking along the same lines as I was when I first got started. Tho I am not a programmer, I don't see a good reason why all the lookup sources couldn't be querried simultaneously and then choose the first result based on the priority in your list. Response time then would be shortened considerably if you had a large number of lookup sources.

lgaetz wrote:

I don't see a good reason why all the lookup sources couldn't be querried simultaneously and then choose the first result based on the priority in your list. Response time then would be shortened considerably if you had a large number of lookup sources.

Yes, this is the basic approach I'm planning. It's complicated by PHP not being able to do asynchronous HTTP requests -- however, using raw sockets to create an HTTP request, but not waiting around for the reply, it should be possible to get one "parent" script to initiate multiple "child" requests on the same server. Instead of replying as an http response, the "child" request would write to a file. The "parent" script would then need to poll for the file. Not optimal, but it should work.

The more difficult part will be the UI changes to allow one to group sources for parallel query. I can see the need for some sources to be a last resort -- especially those with a per-query subscription model. Also, some sources should be tried before unleashing the parallel monstrosity, like the superfecta cache.

Still, I'd like to see a formal release of 2.2.4 before going down this path.

jkiel wrote:

The more difficult part will be the UI changes to allow one to group sources for parallel query. I can see the need for some sources to be a last resort -- especially those with a per-query subscription model. Also, some sources should be tried before unleashing the parallel monstrosity, like the superfecta cache.

As long as the changes you introduce respect the individual Caller ID Schemes, then the mechanism is already in place for what you are describing. All sources in each scheme could be querried simultaneously, and if they all fail then proceed on to the next scheme where the "last resort" sources are querried.

jkiel wrote:

Still, I'd like to see a formal release of 2.2.4 before going down this path.

Agreed.

lgaetz wrote:

All sources in each scheme could be querried simultaneously, and if they all fail then proceed on to the next scheme where the "last resort" sources are querried.

That's a good point, and could simplify implementation greatly -- though users would need to be made well aware of this behavior when upgrading from a superfecta that does not do simultaneous queries.

See r284 for yet another fix related to this bug.

Yet another stab at getting pattern matching under control. r285 is a complete rewrite of the match_pattern function. In r285, I attempt to simplify pattern matching by converting the Asterisk dial pattern to regular expressions.

I have dloaded and did some prelim testing of r285. One issue noted: if there is a blank line or lines in the list of CID rules then it thinks there is a rule on that line and you get an error like this "Invalid character '.strtolower().' in pattern - position 1"

I will leave in place an report any other issues here.

Thanks for testing. Looks like I wasn't testing for blank lines. r286 should take care of that.

QS: Pass. Pattern matches now work as expected in all tested cases.

QS: Pass - Alternate Platform PBXIAF 1.4 / Asterisk 1.4

All tests appear to work as expected in multi scheme and single scheme environments.

I have never used the DID field before in a superfecta scheme. In testing r286 or callerid.php and using one of my DID's in the DID field I don't get CID lookups any more. Debug doesn't work either, no matter what number I use to debug, I get "DID matching enabled...DID did not match."

Can you roll back to a previous version and see if it still works?

To clarify. CID lookups to take place properly, just debug will not work with DID field defined. Will try a roll back per your suggestion.

Also, what is the DID and Pattern you are expecting a match on?

Rolled CID back and buggered the superfecta altogether. Removed superfecta and reinstalled 2.2.3 from scratch. Problem still persists, it could be it has always been this way.

DID Number: is set to my 11 digit toll free DID
CID Rules: I have tried with my regular list of rules: 1XXXXXXXXXX, 1+XXXXXXXXXX, 1902+XXXXXXX and I also tried by removing all rules, same behavior

Error message is always "DID did not match". Problem goes away and debug works fine as soon as I remove the number from the DID field. CID lookups still work fine on incoming calls (provided DID matches) but debug doesn't work.

DID is not set on debug, only on a real call. Let me see what it would take to get some type of DID debug working -- at the very least it will require a debug field for DID in addition to CID on scheme configuration page -- that or DID checking is simple skipped on debug.

skipping DID check on debug seems the most sensible way to go to me

Got a working fix to ignore DID on debug. Change callerid.php line # 148 from:

if((!empty($param[$this_scheme]['DID']))

if((!empty($param[$this_scheme]['DID'])) & !$debug)

debug is working, am testing now in the wild

Looking over the DID matching in callerid.php, one thing I'm confused with is:

$DID_return = match_pattern(substr($param[$this_scheme]['DID'],1),$DID);

In particular: substr($param[$this_scheme]['DID'],1)

In this case, let's say the scheme's DID pattern ($param[$this_scheme]['DID']) is 555NXXX, and let's say the DID ($DID) is 5551234. callerid.php looks like it will try matching 55NXXX (note that we've chapped the first character due to substr($param[$this_scheme]['DID'],1)) against 5551234 -- missing the match.

What am I missing here?

Ok -- looking at it further, I see that DID matching is wanting the see the pattern matching underscore, "_".

Why is this? Why require "_" with a DID pattern but not even allow it with CID? For that matter, why require a "_" for pattern matching, when leaving it off completely will not allow matching even without a pattern.

Should we allow a "_" prefix on both DID and CID patterns, and also allow it to be missing on both?

The underscore prefix denotes a pattern to match, numbers without underscore are absolute. A rule that matches all DID's with area code 567 would be expressed as "_1567NXXXXXX" This is standard FreePBX/Asterisk input behavior to distinguish between patterns and absolutes

I understand that. What I don't understand is superfecta's inconsistency between DID and CID patterns, specifically in the use of the underscore.

The inconsistency stems from FreePBX. As I understand it, Asterisk always requires the underscore when doing pattern matching. FreePBX limits extensions to digits only, but asterisk doesn't, it needs a way to distinguish the letter X from a pattern X which is the underscore. When FreePBX presents a field intended for a pattern, it will internally prepend an underscore before writing the asterisk instructions. When FreePBX presents a field intended for a number, we are able to fool it (sometimes) with a pattern by putting in the underscore manually. The current superfecta behavior is in line with normal FreePBX operation, at least as I understand things. That said, there is no harm in assuming that everything placed in a CID/DID field is a pattern because "_666" is the same as "666"

See r298. I made the "_" optional for both CID and DID. This "shouldn't" cause any problems in a normal configuration, but as I note in the commit:

Both DID and CID will now try absolute and pattern match, and take whatever works. The only issue this may cause is if you have a DID named with a valid pattern that could also match another DID. Say, you named one DID NXX, and then had other DIDs named 211, 332, etc. A DID pattern match of NXX would then match the DID NXX, 211 and 332 -- but would anyone name their DID "NXX"?

jkiel wrote:

The only issue this may cause is if you have a DID named with a valid pattern that could also match another DID.

If I understand what you are saying, as long as the schemes are stepped thru in the defined priority sequence, I don't think it matters if the DID pattern of multiple schemes matches the incoming DID. The first scheme with matching DID pattern will be called, and if unsuccessful then the next scheme with matching DID pattern will be called. I will test r298 on my system now.

r298 is now up and running for me (FreePBX 2.6.0.3 and Superfecta 2.2.3.2) and works well both for debug and incoming calls.

Looks like info pop-ups are handled very differently pre FreePBX 2.8.x. Let me know if r299 works for you.

Ignore that last post -- wrong thread, sorry.

jkiel wrote:

the "_" optional for both CID and DID

It seems like FreePBX is slowly deprecating use of the underscore for this purpose, perhaps superfecta should go this route too. I think the popup help for the DID field should remove mention of the underscore and just allow the pattern to be input directly, and of course maintin the present functionality with the underscore for backwards compatibility

Also we might consider renaming the DID field from "DID Number" to "DID Rules" to better describe functionality

See that DID field is renamed to "DID Rules" in 2.2.3.4 but it instantly struck me that is should read "DID Rule" with no 's'

Yeah - I agree. I wish it had struck me before I rolled up 2.2.3.4 :)
I fixed it. r302

Holding this case for final additional (negative or positive) feedback from testers before accepting for build 2.2.4.

QS: Pass. No further negative test results, and no further test results expected

I doubt we will have any better testing from outside our group than has already been done - so I'm closing this case and accepting it for 2.2.4 build.