| Author |
|
ajdiaz
Junior
Joined: December 10 2007
Location: United States
Posts: 76
|
| Posted: January 07 2009 at 9:11am | IP Logged
|
|
|
We have a few questions with regards to network infrastructure.
When we scale our media engine over 90 lines we start to get instances of dropping packets and loss of audio quality. We have a 40MegaBit circuit to work with - which should have plenty of bandwidth to go over 200 lines or more - so my thought is that it might be the firewall. We're currently using a Cisco ASA 5505.
1) Is there a particular firewall you guys have had success with for large multi line installations? We were thinking about maybe using the Cisco ASA 5510 - but would like to know if you have any recommendations.
2) Under which architecture and network scenarios have you successfully tested 512 simultaneous calls going out?
3) Also it would be appreciated if you could provide insight as to which other factors would affect dropping packets and loss of audio quality as more and more lines are added; and any other recommendations as to which network configuration is ideal for optimal performance with 200, 400, and 512 calls/lines going out at the same time.
4) Are there any application design considerations that we should also consider that would affect the quality of playing a wave file by the Media Engine or our VOIP application as more and more lines are added?
Thanks.
-AJ
|
| Back to Top |
|
| |
support
Administrator
Joined: January 26 2005
Location: United States
Posts: 1666
|
| Posted: January 07 2009 at 6:22pm | IP Logged
|
|
|
Hi Aj,
Thanks for waiting for us to reply. Its been busy here today.
We normally don’t get involved with specifying the network equipment that a customer should deploy for their particular needs. That does not mean we can’t or won’t help you. We will have to review the same Cisco specs that you have access to. Normally when we perform call testing, we do it without other network components between the call test nodes. For example, we may perform concurrent call testing between 2 dual quad core Xeon servers running our call test software based on the media engine.
Looking briefly at the specks for the Cisco ASA 5505, that firewall should be more than adequate for 200 concurrent calls. Likewise the Cisco ASA 5510 would give additional packet headroom. Unless we are missing a critical spec on these firewalls, either one “should” work and not cause data loss for 200 concurrent calls.
The statement above assumes 200 concurrent 20Ms G729 calls are being used and does not consider the additional fractional traffic used by SIP call setup and teardown.
The 40Mbit pipe you are using is sufficient for 200 concurrent 20Ms G729 calls.
I have additional question for you but unfortunately they will have to wait until tomorrow morning. In the mean time:
1)
I need to understand how you are conducting your current load testing and why you suspect the Cisco firewall.
2)
Also need to know if you have call load tested from one of your media engine based servers directly to a terminating VOIP server (or VOIP test box) in the same network without having any other network elements in between the initiating and terminating nodes.
3)
We may have test software here that we can use to help you to isolate and automate the testing.
Thanks,
Randal
|
| Back to Top |
|
| |
support
Administrator
Joined: January 26 2005
Location: United States
Posts: 1666
|
| Posted: January 09 2009 at 2:10pm | IP Logged
|
|
|
Hi Aj,
I was thinking more about this…
1)
Are you absolutely sure we are experiencing RTP packet loss and that this is the cause of degraded voice quality?
2)
Do you have further test info that has determined that the cause of the packet loss is absolutely the firewall? You would have to network sniff both sides of the firewall and compare RTP traffic for all calls. I think this is too hard and that we should qualify your system before looking at the firewall.
3)
How many IVR calls can you scale to without loss of audio quality if you test the IVR server directly with other SIP/RTP test software or a SIP/RTP test box? By the way – VOIP test boxes are very expensive.
We use a simple “call tester” app here to help with load testing and call scaling. Maybe this is something we can use to see if your media engine based IVR server is actually transmitting RTP packets in a timely manner as you scale up call volumes. If we find that your IVR app is not keeping up with needed RTP transmits, then we are looking at a possible Tx IVR sample block data bottle neck in your code or the media engine or some other possible cause of the Tx bottle neck.
SIP/RTP Call Tester Application:
4)
On your server, open task manager and see what the network utilization is reporting on the “Networking” tab. If it’s railed at 100% for 90 concurrent calls, maybe your NIC hardware or drivers are causing a problem.
Better yet, during call load testing, take a look at Window’s Performance console (Control Panel->Administrative Tools->Performance). You can use Performance monitor to look at NIC activity quantities. Of particular importance for Tx IVR apps are the values for “Packets Outbound Discarded” and “Packets Outbound Errors”. If either of these values is reporting too many occurrences, Tx RTP audio will suffer greatly.
5)
Is the problem really packet loss or “non real time” playout of RTP packets from your media engine based IVR server?
If this is the case, then it is possible that your app will need to buffer more IVR sample block data per phone line using the Tx IVR output channels of the phone lines. Once the playback sample block are passed to the phone line’s Tx IVR channels, they are completely in memory, processed and sent out the network vie the media engine’s RTP transmitters.
Finally:
I could go further but we need to start to isolate the Tx call quality problem. What I would do is test your IVR server using the media engine against our “call tester” app. The call tester app could run on one or more other machines and would be used to terminate calls coming from your IVR server.
For test purposes, we would want your IVR server to transmit 600Hz or 700Hz pilot tones on all active phone lines instead of the normal IVR prompts. Tones are used because it is much easier to detect bad audio versus real speech. At any one of the terminating call testers, we can monitor ant terminated line and hear the pilot tones. As you ramp up the number of concurrent calls, we should hit the ceiling where call audio starts to “fall apart”. This we can detect and then determine what the limiting factor is. Using this private network test scenario, we should be able to ramp up to a number of concurrent calls that are only limited by:
1)
Your host processor running your media engine based IVR server app.
2)
Your available network bandwidth.
3)
The throughput of how your app passes Tx IVR sample block data off to the media engine phone lines. Possible bottleneck there?
Thank Aj,
Randal
|
| Back to Top |
|
| |
ajdiaz
Junior
Joined: December 10 2007
Location: United States
Posts: 76
|
| Posted: January 14 2009 at 2:32pm | IP Logged
|
|
|
Hi Randal,
Thanks for your reply. Let me take the time to find out the answers to all your questions. In the meanwhile, we will also try your suggestions (like checking the NIC card).
We would most certainly like to coordinate a test using your “call tester” app. This might provide the insight that we need.
Thanks again.
-AJ
|
| Back to Top |
|
| |
support
Administrator
Joined: January 26 2005
Location: United States
Posts: 1666
|
| Posted: January 14 2009 at 4:23pm | IP Logged
|
|
|
Hi Aj,
Yes, performing a stand-alone test using your IVR server and our call tester app (on a hot host machine) in a 100Mbit private network scenario would be good to see what type of raw call quality you are getting from your server to our call tester.
If we can do 128 or more concurrent calls to the call tester app using monitored pilot tones, then we at least know that your IVR server is pumping Tx RTP packets as required. Then we could look at other possible failure points.
I’ll be here…
Randal
|
| Back to Top |
|
| |
Active Topics 
Member List 
Search 
Help 
Register 
Login
LanScape Support Forum -> LanScape VOIP Media Engine™ - Technical Support
Topic: Network Infrastructure and Firewall Recommendations
