Caller ID Superfecta

It’s the time of year when the desire to give presents and make merry starts to fill us with anticipation and delight. Santa will be along soon – and all us good little girls and boys will get their hearts desires on that special day. The “Developer Elves” who work on the CallerID Superfecta module have been hard at work to bring you this KringleFecta edition of the CallerID Superfecta.

Happy Holidays From The CallerID Superfecta production team!

Since this is possibly the last major release of the Superfecta 2.2.x series (we are preparing the groundwork for version 3.0) – it seems a good time to briefly recap Superfecta development for 2010. As they guy lucky enough to be the Superfecta project administrator, I’ll also answer three interesting questions that tend to surface fairly regularly. Those questions are
Where did the Superfecta Module come from? What’s new in this release? And, What does the future hold?

Where did/does the Superfecta Module come from?
The shortest and best answer is, The Superfecta Module comes from people just like you, who contribute to the CallerID Superfecta, and other open source software projects. People who have the desire to produce something useful, and hope that others can enjoy using it for free.

For example, when I originally created the CallerID Superfecta Module for FreePBX, there were two main motivators.

First, thanks to Mr. Mundy, there were a few Nerd Vittles scripts floating around that were just the coolest little functions for your pbx that you ever did see! I really liked them – but I hated installing them. I thought some folks might even be stumped and unable to use them because of manual nature of their installation.
One of the best was Ward Mundy’s Caller ID Trifecta, which Ward tells us was itself based on a another author’s previous work in Open Source Software. Ward released the Trifecta into the wild. As if in testimony to Open Source Software – his creation went on to be maintained and updated by the members of this forum. Renamed by Mr. Mundy during one of it’s previous upgrades to “Caller ID Superfecta”, it had been extended to include more sources of information (up to 5 at one time) than ever before.

Second – I really wanted to be able to give something back to a community that had helped me so much when I was stumped – with so much professionalism and friendliness. Without PBXIAF and this Forum, I would have given up on VOIP as an idle plaything – impossible to rely upon. Instead, I’m a town crier extolling the virtues of VOIP – especially if you start with PBXIAF.

There was just a bit of a problem with me contributing. I’m a LAMP neophyte, and far less experienced than those offering their help around this place already... So, I looked around for a project that would be of minimum scope that would let me try and gain a little experience doing something productive that I could contribute back to the community.

Eventually, I hit on the idea of trying to figure out how FreePBX modules can be created, converting a few of my favorite existing scripts into modules that anyone could install in seconds with FreePBX, and trying to share that experience with anyone that wanted to hear it. (The first one was the Hotel Style Wakeup module.)

During the development of the modules it became apparent that as an Open Software Projects, they would need some suitable public home. After flirting with many public options like Googlecode, and others, it became obvious that a dedicated site with all the trimmings of ticketing, documentation, and SVN under the same roof was the best choice; A site dedicated to the hosting of any and all software projects that might be associated with the use of our PBXIAF’s, and other distributions. That led to the creation of the Development Support site – a place where it was hoped that other folks who wanted to contribute would feel comfortable pitching in. It was especially important that first time contributors would feel welcome, and be brave enough to indulge their desires to contribute to open source software.

Trust me when I tell you I never imagined what a great experience my involvement would be. If you have ever thought about contributing to an Open Software project – and were reluctant because you weren’t sure you had anything to offer – do yourself and the community a favor and jump in, the water’s fine! ?

Right now there are more than 100 registered contributors at the development support site – participating at various levels as their lives and interest dictate. What a GREAT response considering registering is akin to saying you’re about to start giving away free labor!

The skill levels of the participants are just as varied. Folks who are professional developers pitch in along-side people like me (I am almost completely php impaired), and every skill level in –between, and there’s no blood in the water! There’s truly room for everyone serious about wanting to pitch in.

The people I have been privileged to become acquainted with and work alongside on various projects have been generous and giving in the extreme. They get absolutely nothing out of their participation except our appreciation, and their pride in work well done. Their time and talents are donated freely to help make and upgrade many of the add-ons that we get for free and use everyday. To each one of them – I repeat my thanks for every moment they spent contributing to the projects they work on.

Open Software Should Be Open
It’s a rare and beautiful thing to see folks freely contributing to a coding project. But have a look around here at the PBXIAF forums and you see it; A lot of it. It may not be unique – buts it’s rare enough, and worthy of all our support. For that reason, a Development Support Site and SVN have been provided to host the Superfecta (and several other Open Source PBX projects, such as End Point Manager) development effort and provide a place for the distributed community development effort to thrive. (http://projects.colsolgrp.net) This new release of The Caller ID Superfecta is the latest exciting piece of software to be released through the Dev site. All work done on the module has been accomplished by members of the community, on a completely volunteer basis. All the code in the development site is 100% Open Source, nothing compiled, hidden, or obfuscated – because that wouldn’t be “Open Software”, would it?

Happy Birthday! The Superfecta Module turns 2
When I was preparing to write this introduction, I looked back to find the very first announcement I made when version 1.0 of the module was released. As it turns out, it wasn't too hard to find - as it was released in the PBX in a flash forums. http://pbxinaflash.com/forum/showthread.php?t=3747 , March 13, 2009, a mere 21 months ago.

If you count the time I spent pulling out my hair, and trying to understand how to build a module in the first place, we can say that the caller ID Superfecta module for FreePBX has been under development for a total of a little less than two years. Of course, the basis for the caller ID Superfecta module was Ward Mundy's then community maintained script, the caller ID Superfecta.

In technological terms, 21 months can be a long time, but I have to confess that the advances in the module during that time have been nothing short of amazing. For example, we have gone from 5 data sources to 49!

Evolution – Brief and Abridged
Versions 1.x.x of the Superfecta module were almost a direct adaptation of the then-current version of the Superfecta script being maintained in the PBXIAF forums.

In version 2.0.0, the underlying architecture of Superfecta was changed to prepare for better support of caller ID schemes used outside of the United States, as well as other changes designed to enhance the configuration options available within Superfecta.

In releasing version 2.2.0 (8/17/09) , the Superfecta development team addressed and closed every trouble ticket which had been filed against all version 2.0 releases. We also deployed some of the most exciting changes to the module yet: (1) The concept of CID Schemes – which allows the use of different sources for different inbound routes based on pattern matching on both CID and DID, and (2) Online Source Updating, which provides a completely interface driven way of updating data source scripts direct from the development live update repository.

In version 2.2.1 (10/16/09) , the Superfecta development team addressed and closed every trouble ticket which had been filed against all version 2.0 releases, and added the basic core code required to support international CID for the very first time. All the data sources were made smarter, improving their speed. Version 2.2.1 released with 25 data sources, and was the first version to support the exciting online “live-update” feature for the data source scripts.

In version 2.2.2 (10/29/09) , every ticket filed against 2.2.1 was addressed. Bugs in worldwide CID support, character set problems, and Scheme DID/CID matching rules were fixed.

In version 2.2.3 (11/23/09), bugs in post processing, extended characters in the login, and refinements to live update were addressed. The companion windows program Winunciator, that speaks CID information was released as a companion product and data source. Version 2.2.3 was released with 32 data sources.

So – what could we possibly add to that? I’m so glad you asked!

I am delighted to announce the release and immediate availability of Caller ID Superfecta: THE MODULE Version 2.2.4

What’s New in This Release of v2.2.4?
This release, 2.2.4, is considered a major maintenance release and upgrade for the version 2.2.x series of the Superfecta module. But for the folks that like to create new data source scripts, its also a mini feature release too.

More than 25 bug reports and feature requests were addressed. So many new data sources have been added in the last week – I have literally lost track of the number. Every time I turn around, jkiel or lgaetz has added another one! Such wonderful problems for a project admin to have – don’t worry guys – I can take it – hehe – Let the data sources roll! ?

In general, the developers have:
• Cleaned up various minor code errors to eliminate background noise in the error log file.
• Upgraded the flexibility of the CID / DID Rules for Schemes.
• Added Exception Rules for the Superfecta Cache
• Improved Pop Up mouse-over help.
• Added and upgraded a bunch of Data-sources
• Improved Installation / Upgrade / Uninstall functions

Bug Fixes and Enhancements
Beyond the 25 bugs and features, our Guru at Large jkiel included a few Stocking Stuffers in the form of new functions in the Superfecta API. These new functions help Superfecta get data from sources with more esoteric access requirements than simple unadorned http (such as cookies). He’s also released the first data sources taking advantage of some of the new functions available to data source authors. In fact, in this release John will have contributed at least 5 brand new data sources, and participated in the creation or upgrade of many, many more.

And even though producing documentation is nobody’s expression of Joie de vivre, as the Pièce de résistance for his work on v2.2.4, John’s in the process of helping our volunteer documenters get the new functions listed and explained clearly in the Superfecta Wiki (See! Some developers will participate in documentation.) It’s not done yet – but were trying! ? Remember, when it comes to documenting, there’s always room for one more. Muhahahaa

In general terms, while preparing this release, upgrades were made in 10 existing data sources. Those new versions began appearing in the live update function of the Superfecta module on December 17th, , 2010 .

Our first 5 data sources for Australia make their debut in v2.2.4, along with sources for Argentina, Sweden, and 3 new or upgraded sources for Canada.

Previous versions of the CallerID Superfecta module were….. Noisy in the logs. All the developers rolled up their sleeves for the effort of squashing this nuisance and when they were finished – as far as the error_log file is concerned, the Superfecta module is now zero-emission when no actual errors are present.

Previous versions of the Superfecta Module could, when un installed under the right circumstances, leave your inbound routes inaccessible by outside callers. That problem has been corrected.

CID Schemes
Some of the most exciting enhancements in Superfecta are in the CID / DID Scheme rules area. The scheme rules allow you to use pattern matching in ways very similar to FreePBX. Using schemes you can:

• Compensate for inbound CID’s of varying length – for example in multiple provider situations.
• Determine by CID / DID rules which scheme(s) may be applied to inbound calls.
• A lot more -

Here are some rules to remember when dealing with Schemes:
1. Superfecta treats each Scheme as a separate entity, and processes them in the order you have them listed.
2. If a scheme has rules, and they fail for an inbound call, that scheme is not executed.
3. Only the first scheme providing a CNAM match will be executed.
4. Post Processing will only occur for data sources listed within the successful scheme.
5. You should generally always have a catch all scheme at the bottom with no rules to handle unexpected situations.

Support for CID and DID Pattern Matching Rules
As we have for inbound call routing and trunk selection in FreePBX, CallerID Superfecta provides configuration options to specify both the DID, and CID rules to be used with each scheme. This function has been totally rewritten in Superfecta v2.2.4 to better support all the variations of the traditional pattern matching capabilities. If you had less than perfect success with rules before –revisit them again in release 2.2.4- you’ll find their flexibility can work wonders to extend the functionality of the Superfecta Module in special needs cases.

Thanks entirely to lgaetz, documentation to help you get the most out of CID / DID Rules, and Schemes is available here:

Don’t miss out on the amazing flexibility made available through CID /DID rules – lgaetz has done a great job providing plenty of info for you to get started meeting the needs of those special circumstances.

Data Source Dependencies
Remember, data sources can have dependencies upon certain versions of the Superfecta Module. If you’re running the latest version of the module, you shouldn’t have anything to worry about, but if you are adding data sources to prior versions, remember what you have read here. For example, with version 2.2.4 we have released some new data sources that require this latest version of the module to work. For those of you using a previous version of the Superfecta – pay attention to the pop-up text for each data source and note any requirements it may have for specific versions of the module. Or better yet, just upgrade to 2.2.4. All your old data sources are compatible with 2.2.4. You know you really want to upgrade, right? ?

Data Source Enum Renamed
The data source Enum has been renamed to EnumQuery. When you upgrade to 2.2.4, you may end up with both. Support for source-Enum.php has been discontinued – so delete that data source and use source-EnumQuery.php instead from now on – it’s the same source.

Data Report-Back
Some data sources have a “Report Back” option which, when you enable it, causes a successful CNAM resolution to be Reported Back to its data source. This is primarily used when the data source is one that is maintained by user contribution. If a Report-Back enabled data source is part of the successful scheme, the CNAM collected will be reported back, or contributed to the data provider.

Be conscientious with the report back option. If you enable it, take care to assure that you don’t have a generic source, such as Telco-Data, in the scheme. Doing so could cause you to send generic information such as SAN DIEGO, CA to the data source – which would be considered data pollution – since nobody will want to receive that data from this source in the future.

Superfecta Cache
Speaking of data pollution – the Superfecta Cache has been upgraded to support CID exclusion rules. If you use the Superfecta cache, you can now specify via CID pattern matching things that you do NOT want to go into the cache. This can be extremely useful – here’s a typical situation.

Lets say you use the data source Trunk_provided as first priority because most of your calls come in over analog lines which provides decent and accurate CNAM data.

However, there are circumstances where the trunk provided CNAM is different for each person calling you from within an organization, but the CID number is the same for all. In this case it is inappropriate to cache that CNAM data – but you want to use the cache for other sources in the same scheme.

The solution is quick and easy with the new Superfecta Cache. Simply created a CID rule that either pattern matches, or lists explicitly (use both together if need be) the numbers you do not want cached and bingo! Situation solved. (We can all thank lgaetz for asking for, and jkiel for providing this extra cool, extra functional little upgrade to the Superfecta Cache data source.

Check out the trouble/feature tickets at the dev site directly if you’re curious about any specific details – its all available site for anyone to review.

Who did all this?
As is always the case, this release would not have been possible without the significant contributions of many people – from those who report the bugs, to those that fix them, and those that help test the results. To everyone who stepped up during this last year to help with a new data sources, bug chasing, or coding – please accept my thanks, along with the thanks of everyone who has benefited from your efforts.

For the release of version 2.2.4, PBXIAF forum member and dev site participant Lgetz (Lorne) has stepped up to the plate in a big way. He has created, or facilitated the creation and addition of several new data sources, as well as working with bug reports and other developers on resolving several important trouble tickets, and feature requests. His testing efforts have been tireless – and the module is better for it. Lorne also as helped several new contributors find their way to, and start contributing at the development support site. Thanks Lorne for all your efforts! The project has benefited greatly from your participation.

Also contributing very significantly to this release has been PBXIAF forum member and dev site participant jkiel (John). John stepped into several of our most severe bug tickets and corrected them for us – as well as making ongoing significant contributions in planning for future versions of the module. John’s work in this release made the user-wish for new data sources for Australia come true, and introduced the first group of extended API functions for data-sources that have more extensive requirements than simple http. In fact, he’s contributed more than 5 data sources already. Thanks for all you have already done, John!

What do we owe for all this?
Developing good software is hard work and takes a lot of time. Doing that for free, when you already have a full-time job, as well as a "real life", can only be the result of having suffered repeated head injuries as a child.

Of course I'm only kidding!

In fact, doing all that for free can only be the result of having the most generous of spirits, and a genuine desire to contribute to the creation of open source software.

I’m sure you will agree Lorne, John, and all our contributors deserve a lot more than only our thanks and appreciation for these great upgrades. I hope Karma rewards them richly!

But, until Karma gets around to it, please join me in a round of applause and sincere thanks for forum members Lgetz, jkiel, nixi, zorka, adamgoldberg, mageminds, and Patrick_elx, and the anyone I managed to forget mentioning for giving us all a really nifty little surprise Christmas present: CallerID Superfecta: THE MODULE. V2.2.4. (The KingleFecta edition)

Caller ID Superfecta: THE MODULE Version 2.2.4 May Be Downloaded from the Development Support Site at This Location.

-tshif