High Availability Backup and Restore

In our never ending quest to make FreePBX the best phone system that doesn’t require money to buy (and even better than most that do cost money…), allow us to introduce you to the latest features in the Backup & Restore module. [url=/news/2010-05-09/preview-the-all-new-directory]Last week[/url] we alluded to a critical server error, but left you guessing as to how we address that. This is a big step in that direction.

Along with the kind assistance provided by some customers of [url=/support-and-professional-services]FreePBX Professional Support[/url] (in the form of time donated for a feature they needed for their business), the boys over at Schmoozecom (disclaimer: including yours truly) have once again surpassed all expectations. As of FreePBX 2.8, (currently in beta – more on that later) the Backup module includes built in support for High Availability clustering!

[img]/files/images/HA-backup.png[/img]

The following is an interview I had with Me (aka myself), elaborating on the new features:

[b]I: So, what is this new HA (High Availability) feature, and why are we so excited with it?[/b] [b]Me:[/b] To ensure that their critical infrastructure maintains a high level of reliability, many business seek out HA solutions of one type of another. To that end, we have introduced an automated capability for a designated server (herein: backup server) to periodically backup a production server (herein: primary server) and to restore said backup on the backup server. This ensures that the backup server always has a fresh copy of the primary server’s settings, and is ready to take over should the primary server fail.

[b]I: Billions of bilious blistering barnacles – that’s not true HA! True HA means the backups in real time and instant hand off between the two servers?![/b] [b]Me:[/b] Actually, according to ( wait for it – you’ll never guess!) Wikipedia, HA is [i]“is a system design … to ensures a certain degree of operational continuity…“.[/i] While there may be some benefit to having both the backups and hand-offs in real-time, such solutions can to be overly complex and way above the requirements of many organizations. Additionally, such solution tend to require a unique and complex setup with additional components not usually installed on FreePBX based distributions. Being OS agnostic, FreePBX is best not left to deal with the configuration and maintenance of such solutions. Instead, FreePBX takes a simplistic approach, removing much obscurity and confusion from the picture. Keep in mind, however, that FreePBX’s HA implementation can be used as part of a greater HA solution, perhaps including real-time fail-over support.

[b]I: So, how does FreePBX backup a server?[/b] [b]Me:[/b] The backup server can perform backups at any requested interval – ranging from every minute (not recommended for most scenarios) to multiple times per hour/day/week/month, etc. It then restores the backup to itself, wiping away any previous configurations, and replacing them with the latest setting from the production server.

Optionally, the backup server can be configured NOT to restore the setting locally, and act as a storage area for backing up MANY primary servers. In this configuration, the backup server keeps copies of backups handy, ready to be restored locally in the event of a failure to any of the primary servers. Please note that in this scenario, the backup server will continue to function as a backup server, even in the capacity of a surrogate to the failed production server.

[b]I: How does production move to the backup server?[/b] [b]Me:[/b] As mentioned previously, on its own – it doesn’t. The sysadmin is required to manually change the ip address of the server so that all peers can find it. This also allows the sysadmin more control of the switching over process, more notification to error’s and an incentive to ensure that the primary server doesn’t fail in the first place!

Additionally, there are services that can do the fail over automatically, but those are beyond the scope of this discussion. Apparently, these can be enabled in certain routers as well.

[b]I: Great! This is the perfect way to backup my CDR’s![/b] [b]Me:[/b] Actually, if you require real-time backups of cdr’s, your probably better off with a master-slave setup for your database (for the cdrs that is).

[b]I: After a backup is restored, what happens to my trunks – will they automatically try to register to my provider and ‘steal’ the incoming calls from the primary server?[/b] [b]Me:[/b] No. FreePBX requires that you click the orange bar at the top of the screen for any changed settings to go in to affect. This can be done manually or, if you’re using other elements as part of you HA solution, programmaticly by calling [code]/var/lib/asterisk/bin/module_admin reload[/code]. If you have traditional PSTN trunks (not VoIP), additional measures will have to be taken to connect those lines to your PBX and assure they are up and running.

[b]I: Bougainvillea! How do I backup the backup server?![/b] [b]Me:[/b] Exposed now in the GUI are many different options that can be run after a backup is complete. For example, copy the backup to an ftp server, ssh the backup to another host, or even have it emailed to you.

[b]I: Doesn’t the ability to pull a backup off a primary server pose a security risk?[/b] [b]Me:[/b] We have gone to great lengths to ensure that your phone system remains safe and secure throughout this process. The backups are all executed over ssh and encrypted with the public key.

[b]I: Public keys rock! (Uh, What are public keys?)[/b] [b]Me:[/b] Public/private keys are a method to encrypt data sent over insecure connections – easily considered one of the securest method in the world. See here for more info.

[b]I: Right, I knew that! But just to make sure, can you ‘remind’ me how to set up ssh keys?[/b] [b]Me:[/b] That’s probably something better left for the wiki/forums, but here is a very, very, very quick primer (designed for Cent OS systems, assuming asterisk is running as Linux user asterisk):
[code] sudo -u asterisk ssh-keygen
hit [enter] hit [enter] hit [enter] [/code] Then copy the public key to the primary server:
[code]sudo -u asterisk ssh-copy-id -i /var/lib/asterisk/.ssh/id_rsa.pub root@[/code] Your output should look something like this:
[code] [root@localhost /]# sudo -u asterisk ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/var/lib/asterisk/.ssh/id_rsa):
Created directory ‘/var/lib/asterisk/.ssh’.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /var/lib/asterisk/.ssh/id_rsa.
Your public key has been saved in /var/lib/asterisk/.ssh/id_rsa.pub.
The key fingerprint is:
ae:2d:fd:b7:19:d3:e8:34:8a:a9:7d:76:1c:71:c4:a8 asterisk@localhost.localdomain

[root@localhost /]# sudo -u asterisk ssh-copy-id -i /var/lib/asterisk/.ssh/id_rsa.pub root@myserver.example.com

The authenticity of host ‘myserver.example.com (21.158.66.3)’ can’t be established.
RSA key fingerprint is 8e:ae:6a:49:bb:1b:1b:91:3f:02:4f:65:ab:e7:5e:6b.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘myserver.example.com,21.158.66.3’ (RSA) to the list of known hosts.
root@myserver.example.com’s password:
Now try logging into the machine, with “ssh ‘root@myserver.example.com'”, and check in:

.ssh/authorized_keys

to make sure we haven’t added extra keys that you weren’t expecting.
[root@localhost /]#

[/code] [b]I: Duh! Now, what do I need to do in the backup module?[/b] [b]Me:[/b] Under Remote Backup Options, put in the hostname or IP address of the primary server, the user name (probably root), and the private ssh key (if you followed the steps above, that should be: [code]/var/lib/asterisk/.ssh/id_rsa[/code]). To restore the backup immediately, check “Restore to this server”.
[url=/files/images/ha-backup-opts.png][img=235×75]/files/images/ha-backup-opts.png[/img][/url] [b]I: Great, I’m all set. One last thing – why don’t I see these options in my backup module?[/b] [b]Me:[/b] These options were included starting with FreePBX 2.8, currently in beta. Feel free to start beta testing 2.8! Have a look [url=/news/2010-05-24/freepbx-2-8-beta2]here[/url] for more info.

[b]I: Thank you so much for sharing your thoughts with me. Is there anything else you would like to add?[/b] [b]Me:[/b] Yes. As of [url=/news/2010-05-09/preview-the-all-new-directory]last week[/url] there were some issues threatening the continuity of Custom Contexts as of FreePBX 2.8. Congratulation and a tip of the hat to all those that step up to [url=/bounties/custom-context]contribute and help resolve[/url] the issue!

[b]Moshe Brevda[/b]

FreePBX 2.8 beta2

Version 2.8 has really shaped up with quite a few new features from when we first launched [url=/trac/milestone/2.8?by=type]the 2.8 Milestone[/url]. I’m really excited to see many of the changes and nuggets of features that we have squeezed into this release since kicking it off just 3 months ago! We are almost at the home stretch so roll up your sleeves and help us get there!

We started out highlighting the [url=/news/2010-05-09/preview-the-all-new-directory]All New Directory[/url] which has panned out quite nicely. We still need to get some better recordings made up to polish off the default recordings but we are very much looking for your feedback on what’s there so far. Also, don’t miss the tie-in with the 2.8 IVR which let’s you enable direct dialing to Directory members in addition to or in place of the dial-by-name functionality.

I showed some [url=/news/2010-03-20/sneak-preview-of-2-8]Sneak Previews[/url] of some of the changes including several visual improvements such as the double cascading destination selections and the new Outbound Routes and Trunks Dial Rules GUI boxes. However, as excitement built up for this release so did the additional contributions, bug fixes and various other great ideas that have made this another blockbuster release! As of this blog, there are [url=/trac/query?status=closed&milestone=2.8]223[/url] closed tickets against the milestone so far, of which about 1/3 of those are new Feature Requests, big and small!

For those of you who followed the efforts of Kenn10 and the [url=/bounties/custom-context]Custom Context Efforts[/url], it looks like that was a success so I will be coordinating with Kenn10 to get that effort under way as well!

Since there are several changes in some of the critical core components of the dialplan, I’m approaching this beta more cautiously then usual. We could really use another wave of testing and new testers before I add the 2.8 Version Upgrade module to the 2.7 repository. Your help in both flushing out any issues or giving us other feedback about things you like (or dislike) would be very welcome! (For instance – check out the Accordion effect on the Left Module navigation bar – do we keep that, or put it back the way it was, or other suggestions?)

With this beta2 release we are more-or-less freezing new features on 2.8. There is a small handful of changes that we plan on rolling in and did not want to hold up beta2, but beyond these, we will be pushing most new feature requests out until we get 2.8 finalized.

There are two options to install or upgrade to 2.8 and help us test. If you want to stick with the GUI only, you can manually download the 2.8 Upgrade Module onto your existing FreePBX install and then upgrade though the Module Admin GUI.

You can get the Upgrade module here:

[url=http://mirror.freepbx.org/modules/release/2.7/versionupgrade-2.7.0.2.tgz]2.8 Version Upgrade Module[/url]

which is not yet available through the online repository until we get a little more testing done.

Alternatively, you can [url=http://mirror.freepbx.org/freepbx-2.8.0beta2.tar.gz]download the 2.8.0beta2[/url] tarball and use the traditional install_amp installation/upgrade option, instructions can be found on [url=/support/documentation/installation/upgrading-your-system]the Upgrade Instructions[/url] page.

Which ever route you take, please give us feedback either here, though the ticket system or in the forums to help us drive this release to a final version!

[b]Philippe[/b] – On behalf of the FreePBX Team

Preview the all-new Directory

Do you use the Dial-by-Name option of the current FreePBX Directory? If no, read on as you might find there is goodness to come. If yes, have you ever wanted to add a user with no voicemail? have your admin called instead of you? wanted to add a Ring Group or Queue to it, call an external cell phone, or more?

The current Dial-by-Name directory achieves some basic goals but has always left a lot to be desired. In a lot of use cases it was plagued with issues that made it less than desirable and in my opinion often not viable.

If all your users had voice mail boxes, and all your users were on the same system, and none of your users were reachable via Queues, Ring Groups, or Custom Extensions, and you didn’t need to exclude any voice mail user from the list, and you’re happy with the default prompts, and [your own specific needs here] – then it was great to have around. For the rest of us, read on…

The FreePBX development team, in conjunction with Schmooze Com are proud to announce the all-new Directory Module. We have aimed to fix just about every thing that was wrong with the old directory – and then some. Some of the features that we are really excited about include :
[list] [*]Add/Remove just about anything anyone/thing that you can spell
[*]Include users from other systems (by setting a custom user)
[*]Multiple directories
[*]Multiple entries (for different spelling/tags)
[*]Include all users or just some
[/list] There are other features as well, but we wouldn’t want to spoil the surprise! Nevertheless, here is a screen shot of the new Directory (spoiler alert 😉 ):

[url=http://www.freepbx.org/files/images/download_0.png][img=320×240]http://www.freepbx.org/files/images/download_0.preview.png[/img][/url]

If your running the 2.8 beta (in a beta environment of course), you can install this Directory module from the Module Admin. If you aren’t actively testing the 2.8 beta, what are you waiting for? Install it now and start checking out all the 2.8 goodness, including [url=/news/2010-03-20/sneak-preview-of-2-8]new tooltips, reworked outbound routing,[/url] and a really cool new feature that I will leave to you to find (hint). Oh, and don’t forget to look back here next week for the answer.

Of course, should your extensive test show any abnormalities in the Directory module feel free to file a [url=http://www.freepbx.org/trac/newticket]bug[/url].

One more thing. Some of you have been running the Custom Context module for a while. Save for some quirks, the module has been pretty stable. Until today. With the advent of FreePBX 2.8, Custom Context is pretty much End of Life.

But … it doesn’t have to be. A [url=/node/10354]group of concerned community members[/url] have set out to bring the Custom Context module up to date so that you can continue with your context quirkiness. [b]They need your help[/b]. Please take a moment [url=/news/2010-04-26/v3-2-8-and-custom-contexts]to read the details[/url] and help out in every way you can. Because Open Source is really for You.

p.s. Just a quick heads up: Some of you may have known me by my forum handle lazytt. As of today, I’ll be mbrevda in both the forums and on irc.

[b]Moshe Brevda[/b] – on behalf of the FreePBX Team!