| Author |
|
support
Administrator
Joined: January 26 2005
Location: United States
Posts: 1666
|
| Posted: March 05 2008 at 3:29pm | IP Logged
|
|
|
This thread has been started by LanScape support for a customer who is experiencing RTP call audio problems.
Problem Summary:
The customer has an existing legacy Microsoft Windows hosted “Call dialer/IVR” based application. They have integrated the functionality of a 64 line VOIP Media Engine into the application for the purpose of supporting SIP/RTP VOIP calls. The media engine is wrapped and abstracted with appropriate C++ application code. This application software is located at individual customer facilities.
Customers have the ability to configure their installed software to pass outgoing calls to a “centrally located call processing center” that has Asterisk boxes deployed to terminate further to appropriate PSTN provider(s).
The customer currently has the legacy application and the media engine working together in a single process to make outgoing VOIP calls. The outgoing calls are sent to an Asterisk box and then continue on from the Asterisk box to a PSTN termination provider.
When the VOIP application makes an outgoing call, it eventually plays voice IVR prompts to the far end of the call. When listening to the far end audio (from a PSTN land line or cell phone) the customer is experiencing “choppy” and/or “broken” audio. Call setup and teardown however appear to function as expected.
Initial items to verify:
1)
The application must not call the Windows API procedure timeBeginPeriod() to alter the multimedia timer resolution of the VOIP process. For the media engine to function properly, multimedia timer resolution must be set to 1Ms. This API procedure is called internally by the media engine upon startup where the timer resolution is set to the required 1Ms. Media eninge execution characteristics are guaranteed if timer accuracy is not 1Ms.
2)
When the application plays back Tx IVR prompt data, we need to make absolutely sure that the Tx IVR channel of the phone lines are not going data starve. If this occurs, it will produce choppy transmission of RTP audio.
The application uses media engine supplied event handles to signal when the media engine needs additional Tx IVR transmit sample blocks (20Ms block size). The application logic should perform something similar to the following pseudo code:
Code:
// Check the number of Tx IVR buffers available (should be 6 4 by default)
TotalBuffers = GetNumTxIvrBuffer();
// Initially stuff say 8 of these buffers.
Stuff8buffers();
// loop until the call ends or until we
// are done playing Tx IVR prompt data.
While(!done)
{
// wait for media engine Tx IVR event. This event should
// be set every 20Ms +/-1Ms as the media engine consumes
// the buffered sample blocks.
WaitForSingleObject(...);
// check how many buffers are available for use.
NumIvrTxBuffers = GetNumIvrTxBuffers();
// compute how many sample blocks we need to
// write to the Tx IVR channel to ensure we always
// have 8 buffers in queue.
NumBuffersToQueue = 8 - (TotalBuffers - NumIvrTxBuffers)
While(NumBuffersToQueue > 0)
{
// queue a sample block
//
NumBuffersToQueue--;
}
}
|
|
|
3)
Non realtime test: Verify that RTP media for calls is getting transmitted properly from the media engine. Use the StartPhoneLineRecording() procedure to record all received and transmited phone line RTP data to a file. We simply want to know that no transmitted audio is missing.
4)
Non realtime test: Verify that Asterisk is receiving all RTP media for the call. Do this by allowing Asterisk to record RTP media to disk or whatever method is available.
5)
Realtime test: Test to make sure that RTP media for non PSTN terminated Asterisk SIP phone extensions have proper received RTP audio. Use single or dual line soft phone example apps that come with the media engine.
If we create a test setup of say 6 local SIP soft phone that are registered to Asterisk, and the IVR server makes 6 concurrent calls to the soft phone extensions, all audio prompts should be received by and play clearly on the local soft phones. If broken audio is experienced, then the IVR server app should be configured (if possible) to call the soft phones directly bypassing Asterisk. The IVR server and the soft phones should not be registered with Asterisk for this secondary test. All audio should be received by the test soft phones “unbroken”.
We must verify the media along the call path to see where it is breaking down.
Support
|
| Back to Top |
|
| |
DMcManus
Intermediate
Joined: October 18 2007
Location: United States
Posts: 11
|
| Posted: March 06 2008 at 4:06pm | IP Logged
|
|
|
A quick update on our progress. I was finally able to properly manage my buffer counts and it makes a world of difference. The audio quality has gone up considerably at the PSTN phone end and we are not encountering a lot of the errors we had earlier. We have only done 6 calls simultaneously so far but, as we scare up more people who are willing to be bothered, we will raise that number.
We are, by the way, updated to the 5.12.8.1.
Dennis
|
| Back to Top |
|
| |
DMcManus
Intermediate
Joined: October 18 2007
Location: United States
Posts: 11
|
| Posted: March 06 2008 at 4:10pm | IP Logged
|
|
|
Sorry about this being an additional post.
One concern I have is that a lot of the processing of the audio in my part of the application is done in threads that run at elevated priority. I don't know what priority the LanScape threads run at but, if I am way above that, I could cause LanScape to starve for CPU. Any comments?
|
| Back to Top |
|
| |
support
Administrator
Joined: January 26 2005
Location: United States
Posts: 1666
|
| Posted: March 07 2008 at 8:14am | IP Logged
|
|
|
Hi Dennis,
Thanks for the additional info. We will post info regarding your threading question shortly...
Don't worry at all about multiple posts. Post whatever info you think will help us. It's cool :)
Support
|
| Back to Top |
|
| |
support
Administrator
Joined: January 26 2005
Location: United States
Posts: 1666
|
| Posted: March 07 2008 at 11:13am | IP Logged
|
|
|
Hi Dennis,
You >>>
One concern I have is that a lot of the processing of the audio in my part of the application is done in threads that run at elevated priority. I don't know what priority the LanScape threads run at but, if I am way above that, I could cause LanScape to starve for CPU. Any comments?
<<< Support
Good question. All internal media engine threads that have anything to do with media flow run at THREAD_PRIORITY_TIME_CRITICAL regardless of the process priority. So you can elevate your application “media handling threads” to the same THREAD_PRIORITY_TIME_CRITICAL priority and Windows will make sure all media related threads get executed “round robin” when they are ready to run. This is the best approach. All internal media related threads want to run every 20Ms so make sure your elevated media threads In your app do not run constantly during any single 20Ms block time. In other words, so media processing gets handled as smoothly as possible, all media related threads should execute their logic in 20Ms or less.
If you leave your media handling threads at a lower priority than “time critical”, make sure they are coded such that they are still able to keep up with media demands as system loading increases. This may mean stuffing more than a single Tx IVR phone line buffer each time the app thread executes.
A note about process priority:
The media engine likes to handle the process priority itself. If an application calls the Windows API proc SetPriorityClass(), to lower the process priority when calls are active, it will have an impact on call quality (media flow) depending on overall system load.
Normally an app should call the SetInCallProcessPriority() media engine API procedure to allow the media engine to control the VOIP application process priority as needed. If your app does not call this proc, then the process priority will always be “normal” even when calls are active. If your app calls this proc specifying either PriorityHigh or PriorityRealtime, the media engine will elevate the VOIP app’s process priority automatically whenever one or more calls are active. When all calls end, then process priority will resort back to “normal”.
Support
|
| Back to Top |
|
| |
DMcManus
Intermediate
Joined: October 18 2007
Location: United States
Posts: 11
|
| Posted: March 11 2008 at 2:27pm | IP Logged
|
|
|
That clarifies things as far as your thread priority. I also have my media handling threads running at that priority (2 per phone line). The lower level line sequencing (1 thread per phone line, not directly in the audio handling) runs at THREAD_PRIORITY_HIGHEST (2 notches above normal).
As was explained on the phone with your people, we have adapted this from an existing application that drives USB hardware interfaces to the phone lines. In that application, we can feed up to 48 lines simultaneously with no audio problems.
One difference here is that we are using your rate converter (11kHz to 8kHz) to drive your Touch Tone decoder on one of the threads that runs at THREAD_PRIORITY_TIME_CRITICAL. Our USB implementation had hardware Touch Tone decoders so we didn't have that overhead. How much overhead would the conversion and detection insert?
Dennis
|
| Back to Top |
|
| |
support
Administrator
Joined: January 26 2005
Location: United States
Posts: 1666
|
| Posted: March 11 2008 at 4:56pm | IP Logged
|
|
|
Hi Dennis,
You >>>
How much overhead would the conversion and detection insert?
<<< Support
Last time we checked it was very fast. Converting from 11k to 8k PCM is quick as is the implementation for DTMF decoding. The actual numbers we do not have right at the moment and would be different for your host machine anyway.
A simple test that can be executed by you is to call the procedure shown below. Note that it has been simplified for brevity sake.
Code:
void TimePcmConversionAndDtmfDetect(void)
{
int cnt;
DWORD StartTimeMs;
DWORD EndTimeMs;
DWORD DiffTimeMs;
double TimePerConversionAndDetectMs;
short Pcm11kBuffer[221]; // 11050Hz pcm source buffer - 20Ms.
short Pcm8kBuffer[160]; // 8000Hz pcm source buffer - 20Ms.
// create a media engine DTMF decoder. create the decoder
// in immediate mode. See the docs for more info.
CreateDtmfDecoder(...);
// create a format/rate converter to go from 11k to 8k pcm.
CreateFormatRateConverter(...);
// fill source PCM buffer with random sample data.
for(cnt=0;cnt<221;cnt++)
{
Pcm11kBuffer[cnt] = (short)rand();
}
//////////////////////////////////////////////////////
//
// time how long it takes to perform conversion and
// DTMF detection.
//
//////////////////////////////////////////////////////
// snap the start time +/-1Ms.
StartTimeMs = timeGetTime();
// perform dummy PCM conversions and DTMF detections.
for(cnt=0;cnt<10000;cnt++)
{
}
// snap the end time +/-1Ms.
EndTimeMs = timeGetTime();
// get time diff.
DiffTimeMs = EndTimeMs - StartTimeMs;
// compute time per PCM conversion and DTMF detection.
// the value computed for 'TimePerConversionAndDetectMs' should
// be small - i.e way less that 20Ms.
//
TimePerConversionAndDetectMs = (double)DiffTimeMs/(double)10000;
DestroyFormatRateConverter(...);
DestroyDtmfDecoder(...);
}
|
|
|
Support
|
| Back to Top |
|
| |
support
Administrator
Joined: January 26 2005
Location: United States
Posts: 1666
|
| Posted: March 12 2008 at 2:38pm | IP Logged
|
|
|
Dennis,
Additional info for you: We were working on an In-band DTMF issue for another customer and decided to perform at least ½ of the above test we described above.
The DTMF decoder blocks in the media engine execute very quickly.
We fed a test DTMF decoder about 20 seconds of real PCM sample block data from disk. The DTMF decoder was set up to accept 160 samples of 16 bit 8k PCM DTMF sampled data per iteration.
DTMF execution time is around 8.3 microseconds on a host PC having the following specs:
AMD Athlon 64 CPU 3200+
Windows XP Pro, SP2 (32 bit OS)
2 gig ram
Support
|
| Back to Top |
|
| |
support
Administrator
Joined: January 26 2005
Location: United States
Posts: 1666
|
| Posted: March 12 2008 at 3:20pm | IP Logged
|
|
|
Dennis,
Forgot to mention one other item. Be sure to use the DTMF decoder blocks in “immediate mode”. That way the DTMF decoders you create will not create unneeded internal DTMF decoder threads. This will save memory and other thread resources.
See the “ImmediateEvaluation” parameter of the CreateDtmfDecoder() API procedure.
Support
|
| Back to Top |
|
| |
support
Administrator
Joined: January 26 2005
Location: United States
Posts: 1666
|
| Posted: March 19 2008 at 5:21pm | IP Logged
|
|
|
Hi Dennis,
Just curious what the latest news is.
Thanks,
Support
|
| Back to Top |
|
| |
DMcManus
Intermediate
Joined: October 18 2007
Location: United States
Posts: 11
|
| Posted: June 10 2008 at 11:33am | IP Logged
|
|
|
Hi Support -
I updated my system to version 5.12.8.3 back on 5/20 and recompiled to make sure nothing had changed. I did not, however, try to actually use it until today (6/10). I am getting the following error:
First-chance exception at 0x7c90eb74 in ModuleComboPackConsoleHost.exe: 0xC0000008: An invalid handle was specified.
when making the call to StartSipTelephony().
I switched back to the previous version I have (5.12.8.1) and that does not return the error. Any ideas?
Thanks in advance.
Dennis
|
| Back to Top |
|
| |
support
Administrator
Joined: January 26 2005
Location: United States
Posts: 1666
|
| Posted: June 10 2008 at 11:49am | IP Logged
|
|
|
Hi Dennis,
That does not ring a bell. When the app crashes, see if you can get your debugger or Dr Watson to create a mini dump with full heap. It will be a large file but if you can upload this to your support FTP account, we should be able to see what occurred.
Support
|
| Back to Top |
|
| |
DMcManus
Intermediate
Joined: October 18 2007
Location: United States
Posts: 11
|
| Posted: June 10 2008 at 12:48pm | IP Logged
|
|
|
Hi again...
It appears that I was giving the function an invalid local IP address. My supposedly "Fixed" IP address on the local router somehow changed. When I fixed that, the problem went away. Hope that helps some.
Dennis
|
| Back to Top |
|
| |
DMcManus
Intermediate
Joined: October 18 2007
Location: United States
Posts: 11
|
| Posted: June 10 2008 at 1:06pm | IP Logged
|
|
|
I am now getting the same error sometimes but not always when I shut down the system and call StopSipTelephony(). I don't appear to have any other problems prior to that while using the system.
Thanks by the way for the really prompt reply to my first message today!!
Dennis
|
| Back to Top |
|
| |
support
Administrator
Joined: January 26 2005
Location: United States
Posts: 1666
|
| Posted: June 10 2008 at 3:02pm | IP Logged
|
|
|
Dennis,
If the IP address you told the media engine to use was not available on your host PC, then the media engine should have returned to your app the SipSocketBindError value from the StartSipTelephony() API proc to indicate that it could not bind to the configured Sip UDP socket. It should not have “blown up”. Hmmm….
The other thing to note is: If the media engine "blows up" somehow, generates an exception, access violation or some other error, the faulting address that should be reported by Windows will be in the range of 0x63000000 to 0x63XXXXXX where XXXXXX denotes the high address for the media engine DLL.
The first chance exception address of 0x7c90eb74 is not in media engine code.
If your app code uses try/catch blocks for exception handling, make sure you do not use the catch(…) exception handler construct. That is not good because it can make debugging and tracking down the real cause of the exception painful.
Support
|
| Back to Top |
|
| |
Active Topics 
Member List 
Search 
Help 
Register 
Login
LanScape Support Forum -> LanScape VOIP Media Engine™ - Technical Support
Topic: Audio issue when terminating to PSTN via Asterisk
