Property List Support for Asterisk & CallWeaver
The key to better GUI integration
Background
Asterisk uses the outdated Windows INI format to store its configuration. This format was the default configuration file format for Microsoft Windows before 1995, when it was superseded by a better system of storing configuration data. Even when it was first introduced with Microsoft Windows it was inferior to other formats in use at that time. The format has serious disadvantages and its continued use by Asterisk represents an impediment to better GUI integration for Asterisk both on MacOS X and other platforms.
MacOS X inherited its property list based system for persistent storage of configuration data from its predecessor, NeXT. It is the most advanced configuration system available today and it is used systemwide for configuration data of both the operating system and applications. The flexibility and ease with which MacOS X developers can create rich GUI applications is to a large extent the result of this system. Yet, with CF-Lite, Apple has made the system available as free open source software for use on other Unix based and Unix like platforms, including BSD, Linux and Cygwin, a Unix compatibility layer for Windows.
Further reading on MacOS X property lists and persistent configuration data storage:
Using Property Lists
Sunrise Telephone Systems Ltd. started development on property list support for Asterisk in order to make better GUI integration possible for Asterisk both on MacOS X and other platforms. The aim is to entirely replace the INI based configuration system with a property list based system.
The immediate benefit will be that the configuration files will be fewer, clearer and easier to understand and edit. For example, the Windows INI format does not support nesting of data and therefore several files are needed to group settings which could otherwise have been stored in a single file. Further, the property list format is cleaner and it is common practise to make identifiers more intuitive so that their purpose will be obvious from their name even to a novice.
All static configuration settings that belong to the asterisk core will be stored as a property list in a single master configuration file. Property lists can be stored in three different formats, binary, XML or plaintext. The following is an example of a master configuration file in plaintext plist format ...
/*
* Default Asterisk/CallWeaver master configuration file - Version 0.90
*
* This master file consolidates all static configuration settings
* of the Asterisk/CallWeaver core which were previously located in
* asterisk.conf, logger.conf, manager.conf and modules.conf,
* further the configuration settings of the optional resource
* module res_config.so previously stored in extconfig.conf.
*
* This file is in ASCII plain text property list format which
* replaces the previously used Windows INI file format as default.
*
* Syntax:
*
* plist = key-value-pair { key-value-pair }
* key-value-pair = identifier "=" ( value | value-container ) ";"
* identifier = simple-identifier | quoted-identifier
* simple-identifier = letter | digit { letter | digit }
* quoted-identifier = '"' printable-char { printable-char } '"'
* value = boolean | number | string
* value-container = dictionary | value-list
* dictionary = "{" plist "}" ";"
* value-list = "(" value { "," value } ")"
* boolean = "yes" | "no"
* number = digit { digit } [ "." digit { digit } ]
* string = '"' printable-char | escaped-char '"'
*/
/* settings for debugging */
Debugging = {
ExtendedDebugMode = no;
TimestampFormatString = "%F %T";
};
Dialplan = {
// use Asterisk 1.0 dialplan engine
Engine = "asterisk_v1.0";
Filename = "extensions.conf";
MethodOfConfiguration = ini;
};
/* settings previously located in asterisk.conf */
Directories = {
// change these when using CallWeaver
AGIDir = "/var/lib/asterisk/agi-bin";
BinDir = "/usr/sbin";
EtcDir = "/etc/asterisk";
KeyDir = "/var/lib/asterisk/keys";
LogDir = "/var/log/asterisk";
ModDir = "/usr/lib/asterisk/modules";
RunDir = "/var/run";
SpoolDir = "/var/spool/asterisk";
VarLibDir = "/var/lib/asterisk";
};
Filenames = {
// change these when using CallWeaver
Database = "astdb";
Executable = "asterisk";
PIDFile = "asterisk.pid";
Socket = "asterisk.ctl";
};
/* settings previously located in logger.conf */
Logfiles = {
// logging to the console
console = {
Enabled = yes;
LogLevels = (notice, warning, error);
};
debug = {
Enabled = no;
LogLevels = (debug);
};
// logging to the default log file
messages = {
Enabled = yes;
LogLevels = (notice, warning, error);
};
// logging to syslog daemon
syslog.local0 = {
Enabled = no;
LogLevels = (notice, warning, error);
};
// example of log file with all log levels
verbose = {
Enabled = no;
LogLevels = (notice, warning, error, debug, verbose);
};
};
/* settings previously located in manager.conf */
MgrAPI = {
Enabled = no;
IPaddress = "0.0.0.0"; // listen on all available interfaces
PortNumber = 5038; // port to listen on
UserAccounts = {
fred = {
Enabled = no;
HostsDenied = "0.0.0.0/0.0.0.0";
HostsPermitted = "1.2.3.4/255.255.255.0";
Password = 12345;
PermissionsToRead = (system, call, log, verbose, command, agent, user);
PermissionsToWrite = (system, call, log, verbose, command, agent, user);
};
};
};
/* settings previously located in modules.conf and extconfig.conf */
Modules = {
// the file name of a module is used as key for its entry
// LoadOnLaunch determines whether or not to load the module
// MethodOfConfiguration can be any of: none, plist, ini, odbc, pgsql
app_conference.so = {
LoadOnLaunch = yes;
MethodOfConfiguration = none; // no config file
};
chan_applemodem.so = {
LoadOnLaunch = yes;
MethodOfConfiguration = plist; // property list format
PluginOfModule = chan_modem.so; // dependency
};
chan_modem.so = {
LoadOnLaunch = yes;
MethodOfConfiguration = ini; // Windows INI format
};
res_shadowconf.so = {
LoadOnLaunch = yes;
MethodOfConfiguration = plist; // property list format
};
res_zeroconf.so = {
LoadOnLaunch = yes;
MethodOfConfiguration = plist; // property list format
};
};
/* settings previously located in asterisk.conf */
Options = {
AppendHostToLogFiles = no;
AutoLoadModules = yes;
AutoRotateQueueLog = no;
ConsoleVerbosityLevel = 3;
DumpCoreOnCrash = no;
InitCryptoKeysOnLaunch = no;
PseudoRealtimeMode = no;
RecSoundsAsTempFiles = yes;
};
The Asterisk Dialplan
Unlike the configuration data in the master plist, the data stored in the dialplan is considered to be of dynamic nature and therefore, it will be stored separately. The dialplan section in the master plist controls which dialplan Asterisk will be using, which dialplan engine to use and which format the dialplan is represented in. The default settings will mimic the exact same behaviour of Asterisk as it is at present, both dialplan language and format will initially remain the same ...
/* Configuration to exhibit the exact same dialplan behaviour as Asterisk 1.0 */
Dialplan = {
// use Asterisk 1.0 dialplan engine
Engine = "asterisk_v1.0";
Filename = "extensions.conf";
MethodOfConfiguration = ini;
};
Nevertheless, we have been experimenting with ways how to represent the existing dialplan syntax in a property list and add aliases for patterns to be matched. The results look promising so far, but more work is needed to finalise it. The following is an experimental snippet of dialplan data represented in plaintext property list format ...
// E X P E R I M E N T A L M E T A D I A L P L A N
// Define named keypad patterns
// This is for most folks and uses
patterns = {
// 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, *, # match themselves
// X matches any single digit from 0 to 9
// Y matches any single digit from 1 to 9
// N matches any single digit from 2 to 9
// % matches any single digit, * or #
// leading + matches + and international prefix
// trailing ~ matches any number of digits
// [0-8] matches any single digit from 0 to 8
// 0[11,22,33]9 matches any of 0119, 0229, 0339
// spaces are allowed but ignored, for readability
+ = "011";
International = "+YX~";
NANPA = "1 NXX NXX XXXX";
4-Digit-PIN = "%%%%";
2-and-3-Digit-Service-Code = ( "*XX", "*XXX" );
US-tollfree = "1 8[00,66,77,88] NXX XXXX";
JP-tollfree = ( "0120 XXX XXXX", "0077 XXX XXXX" );
};
// Define named regular expressions
// This is for experts and witchcraft
regex = {
// any regular expression
matchall = ".*";
SIPURI = "^[a-zA-Z0-9._%-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,4}$"
};
// Define defaults
defaults = {
// use asterisk 1.0 dialplan engine
engine = "asterisk_v1.0";
// store scripts inline
storage = inline;
// cache external scripts
cache = yes;
};
// Define a context "incoming"
incoming = {
include = (mainmenu, disa, default);
// match calls to our US toll-free number
18005551234 = {
script = {
1 = "NoOp(incoming call on ${EXTEN})";
2 = "Dial(SIP/2000,60,r)";
3 = "Hangup"
};
// engine omitted - default will be used
// storage omitted - default will be used
};
// match calls to SIP URIs using named regex "SIPURI"
SIPURI = {
script = "sipuri-in.pl";
engine = "mod_perl.so";
storage = plaintext;
};
};
// Define a context "outgoing"
outgoing = {
// match named pattern "US-Tollfree"
US-Tollfree = {
script = "ustf-out.io";
engine = "mod_io.so";
storage = plaintext;
};
// match named pattern "NANPA"
NANPA = {
script = "nanpa-out.io";
engine = "mod_io.so";
storage = plaintext;
};
// match named pattern International
International = {
script = "intl-out.js";
engine = "mod_js.so";
storage = odbc;
};
};
// Define a context "foobar"
// This shows how to pass every match to a single script
foobar = {
// match named regex "matchall"
matchall = {
script = "foobar.pl";
engine = "mod_perl.so";
storage = plaintext;
};
};
Asterisk Plug-in Modules
Each plug-in module will continue to have its own configuration file, but we will convert them all into property list format as well and this will become the default. However, the format for module configuration data will remain to be configurable on a per module basis through using the res_config module which so far supports odbc, mysql and pgsql. We will add plist and ini format plug-ins. The dialplan section in the master plist controls which modules will be loaded and what configuration file format to use ...
Modules = {
// Dialplan applications
app_conference.so = {
LoadOnLaunch = yes;
MethodOfConfiguration = none; // no config file
};
// etc
// Channel drivers
chan_iax.so = {
LoadOnLaunch = yes;
MethodOfConfiguration = plist; // property list format
};
// etc
// Resource modules
res_zeroconf.so = {
LoadOnLaunch = yes;
MethodOfConfiguration = plist; // property list format
};
// etc
};
Removal of the Asterisk INI configuration file reader
Asterisk's INI file reader is a very naive and badly written data file reader. Not only does it violate every priniciple of software design, but it also represents a serious security threat because it does not perform any checks on the data it reads into memory.
Since the 1950's the commonly accepted practise for reading information stored as plaintext has been to read characters one by one from the source file and pass them to a scanner to perform lexical analysis during which malformed text and illegal characters are either rejected or filtered out, then tokenize the input stream and pass the token stream to a parser for syntax analysis. The Asterisk INI file reader does none of this and therefore the integrity of the configuration data cannot be guaranteed. Worse still, there is not even a formal definition of what constitutes legal configuration data because the Asterisk INI file reader is not based on any grammar. It will take any input without any checks whatsoever.
For this reason alone the Asterisk INI file reader ought to be removed or replaced by a proper parser that is based on a formal grammar. The good news is that with the replacement of INI configuration files by property lists, the INI file reader will no longer be needed inside the Asterisk core and it will be removed altogether from future MacOS X Asterisk releases. The property list parser is part of Apple's Core Foundation and CF-Lite frameworks so it will not need to reside within the core.
Sunrise Telephone Systems has developed a standalone conversion utility to automatically translate Asterisk INI files to plaintext property lists. Unlike Asterisk's configuration file reader, this utility is based on a formal grammar and it implements a proper scanner for lexical analysis and a proper parser for syntax analysis to be performed on INI files prior to conversion to property lists, thereby ensuring data integrity.
In addition to the conversion utility there will be a resource plug-in based on the Sunrise Asterisk INI file scanner and parser to allow Asterisk to read INI configuration files as an optional format in conjunction with res_config. It is worth noting that the open sourced scanner and parser code provided by Sunrise is modular and could be used to replace the existing INI file reader in the Asterisk core to fix the data integrity problem in Asterisk.org releases. The code is released under an MIT license.
Configuration compatibility with legacy systems
There will be several measures to ensure configuration compatibility between old and new.
The compatibility of module configuration data is achieved through the use of res_config plug-ins for INI and plist format. These plug-ins can be loaded on any Asterisk system and will work on any platform.
In addition, and in order to allow exchange of configuration data of the Asterisk core, a standalone conversion utility is provided to convert between INI and plist format.
Further still, another plug-in module, res_shadowconf, will be developed to shadow the active configuration data of Asterisk and loaded modules into a set of INI formatted shadow configuration files located in /etc/asterisk/shadow. The shadow files will be for reference only, modifying them will not have any effect on the configuration, but they can be used to show other Asterisk users who do not yet use plists when asking for assistance.
Cross-platform compatibility
The ideal situation would be for Asterisk on other platforms to adopt the OSX property list system which is vastly superior to the way in which other platforms store configuration data. Apple's free open source CF-Lite framework makes this possible.
However, without strong popular demand in the Asterisk community, it will be rather difficult to get the necessary changes accepted into Asterisk.org releases, regardless of the merits. We have therefore approached the people at CallWeaver.org and suggested to adopt the property list based configuration system into their upcoming release.
CallWeaver.org is a vendor-independent project based on a forked version of Asterisk 1.2. The Project agreed to adopt property list support if we provide the necessary modifications.
GUI integration, that's what it is all about
The main driver behind the new configuration system is better GUI integration. Once Asterisk supports property lists, all of its configuration data can be examined and modified with Apple's Property List Editor ...
All available GUI tools from Sunrise Telephone Systems will then be upgraded to take advantage of Asterisk's property list support. The following screenshots are some examples on how Asterisk Launcher will extended in the future to provide better control ...
Source Code
The following source files have been released for review thus far ...
Modifications and additions to the Asterisk/OpenPBX.org core
- Drop-in replacement for function ast_readconfig() in asterisk.c
- Preferences.h -- Core Foundation Preferences API wrapper for Asterisk
- Preferences.c -- Core Foundation Preferences API wrapper for Asterisk
ast_plist_support.zip -- all the above files in a single zip archive
Asterisk INI file to property list conversion utility -- aini2plist v1.10
- Makefile -- Build script for use with the make utility
- globaldefs.h -- Global aliases and macros
- ASCII.h -- ASCII macros
- UTF8.h -- UTF8 macros
- hash.h -- Hash function
- hash.c -- Hash function
- scanner.h -- Asterisk INI file scanner
- scanner.c -- Asterisk INI file scanner
- parser.h -- Asterisk INI file parser
- parser.c -- Asterisk INI file parser
- intermediate.h -- Asterisk INI file IR tree
- intermediate.c -- Asterisk INI file IR tree
- hashcodes.h -- Keyword hash codes
- coredict.h -- Core system keyword dictionary
- plugindict.h -- Plugin modules keyword dictionary
- dialplandict.h -- Dialplan keyword dictionary
- keywords.h -- Asterisk INI file keyword dictionary
- keywords.c -- Asterisk INI file keyword dictionary
- plistgen.h -- Property list code generator
- plistgen.c -- Property list code generator
- converter.h -- Asterisk INI file conversion engine
- converter.c -- Asterisk INI file conversion engine
- symdump.h -- Asterisk INI file symbol stream output
- symdump.c -- Asterisk INI file symbol stream output
- pathnames.h -- Library functions for pathname handling
- pathnames.c -- Library functions for pathname handling
- main.c -- Asterisk INI file to property list conversion utility
aini2plist.zip -- all the above files in a single zip archive
aini2plist-xcodeproj.zip -- all the above files as an Xcode project file
Developers welcome
Now, all we have to do is finish the work. Developers interested to join this effort please contact us by email. The code for Asterisk and CallWeaver (both core and plug-ins) is being written in C and we use the Core Foundation framework on OSX, CF-Lite on other platforms. The configuration file conversion utility is being written in ANSI C and has no dependencies on any third party libraries.
Join the Macintosh Telephony Mailing List
List membership is free. Click here for details.
Legal Disclaimer
Asterisk is a trademark of Digium Inc. Apple, AppleScript, Bonjour, Cocoa, Macintalk, Macintosh and MacOS X are trademarks of Apple Inc. The Asterisk software is released under the GNU General Public License (GPL). The Asterisk for Macintosh Distribution is maintained and released by Sunrise Telephone Systems in accordance with the GPL whilst Apple's and Digium's trademarks are mentioned for reference only in order to inform about the origin and purpose of the software. The mentioning of those marks does not imply any kind of endorsement neither by Apple Inc nor by Digium Inc. A4M is a trademark of Sunrise Telephone Systems. A4M specific icons shown on this page are the property of Sunrise Telephone Systems, their use for other purposes than use or redistribution of the unmodified A4M software distribution or parts thereof requires written permission.
