Build a mobile video chat frame with the Nex Gen Media Server (NGMS) API

Introduction

Nex Gen Media Server is best known as a multi-purpose media streaming server to deliver live and stored video to a variety of devices. The same media server can be integrated into a mobile application to facilitate real-time video communication. Here, we will use the NGMS API to facilitate building a video chat client for Android devices using the Android Software Developer Kit (SDK) and Google Android Native Developer Kit (NDK).

We use NGMS to provide all RTP services for network transport, video frame compression and decompression. NGMS runs as a built-in compiled shared object used by our Android app. We will use a Java Native Interface (JNI) bridge build to invoke native methods from the Java application code.

Pre-requisistes

This article assumes that you have a basic understanding and at least an intermediate level of experience creating Android apps. I will not go into the details of setup and Android project in an iDE like Eclipse. If you’ve created Android apps in the past, or at least reviewed some sample tutorials, you should be able to use this tutorial to create a real-time video streaming client.

If you haven’t already, download the Google Android Software Developer Kit (SDK). In this example, I used Revision 15 on a Linux 32bit machine. The Java application layer is used to create the actual Android app. The app code will interface with a JNI layer to invoke the NGMS API routines.

You will also need to download Google Android Native Developer Kit (NDK). In this example, I used Revision 6b on a Linux 32bit machine. NDK is used to create the original interface layer that acts as the glue between the original code and the Java application code. You also need to have a basic understanding of C programming languages.

First, create a skeleton Android application using an IDE such as Eclipse. In this example, the application is called ngmsclient. The Java package name is called com.example.ngmsclient and will target Android OS 2.3 or later. The Root project directory should contain a directory called “jni” which will contain the original sources used to interface with the Java application code. The example below assumes that you have a skeleton Android app that works and shows you how to integrate an ExampleChat class into your Android project.

The native layer

The NGMS core embedded library is written in C and packed as a shared object file. Since Android OS is based on Linux, the NGMS core library initially runs within your Android application space. You need to download the NGMS core library for Android OS from the ngmsvid.com site. The essential components are the libngms.so library files, libxcode.so and the ngmslib.h header file. The ngms bundled.so files are bundled with your ngmsclient.apk to provide video chat services.

JNI layer

We are building a shared library called ngmsglue.so that will act as the glue between the ngms API and our Java application code. The structure of the Java Native Interface (JNI) directory in your ngmsclient project directory should look like this.


jni/Android.mk
jni/ngmsglue.c
jni/ngms/include/ngmslib.h
jni/ngms/lib/libngms.so
jni/ngms/lib/libxcode.so

ngmsglue.c will contain the code for checking NGMS output and input functions. The code maintains two separate NGMS API contexts, one for power output parameters and one for power input parameters. You should note that for each function, the naming convention must match the Java package and the class name from which you invoke the original code.

ngmsStartReceiver is called to initialize the NGMS capture input API to listen on port 5004 for a network video stream encapsulated over MPEG-2 TS. Each successive video frame is automatically removed and decoded to RGB565 pixel format. The complete frame can then be read by calling ngmsReceiveFrame.

ngmsStartSender is called to initialize the NGMS stream output API to transmit encoded video frames to an external host over port 5004. ngmsTransmitFrame is called to encode and transmit a single raw frame in NV21 pixel format.


/*
* ngmsglue.c
*
* JNI layer to access the NGMS streaming and capture API
*
*/

#include
#include
#include
#include
#include "ngmslib.h"

/**
* Holds the configuration for the NGMS RTP receiver and decoder
*/
static NGMSLIB_STREAM_PARAMS_T ngmsReceiver;

/**
* Holds the configuration for the NGMS RTP sender and encoder
*/
static NGMSLIB_STREAM_PARAMS_T ngmsSender;

/**
* Starts the NGMS capture service
*/
jint Java_com_example_ngmsclient_ExampleChat_ngmsStartReceiver (JNIEnv* _env, jobject _thiz)

char xcoderConfig[512];
char sdpConfig[512];
int i;

/*
* Before doing anything call ngmslib_open to open the NGMS service
*/
i = ngmslib_open(&ngmsReceiver);
if (i!= NGMS_RC_OK)

__android_log_print(ANDROID_LOG_ERROR, "ngms", "ngmslib_open failed with error %d", i);
return -1;

/*
* Construct an xcoder configuration key value string
*/
sprintf(xcoderConfig, "vc=rgb565,vx=320,vy=240");

/*
* Construct an SDP configuration for the RTP receiver
* Here we setup NGMS to read data over a MPEG-2 Transport Stream.
*/
sprintf(sdpConfig, "sdp://"
"m=video 5004 RTP/AVP 33n"
"a=rtpmap:33 MP2T/90000n"
"a=fmtp:33n");

/*
* Customize the capture configuration parameters
*/
ngmsReceiver.inputs[0] = sdpConfig;
ngmsReceiver.strxcode = xcoderConfig;
ngmsReceiver.islive = 1;
ngmsReceiver.noaud = 1;

/*
* Start the RTP receiver
*/
i = ngmslib_stream(&ngmsReceiver);

if (i!= NGMS_RC_OK)

__android_log_print(ANDROID_LOG_ERROR, "ngms", "ngmslib_stream failed with error %d", i);
ngmslib_close(&ngmsReceiver);
return -1;

return 0;

/**
* Stops the NGMS RTP capture service
*/
jint Java_com_example_ngmsclient_ExampleChat_ngmsStopReceiver (JNIEnv* _env, jobject _thiz)

/*
* Stop the RTP receiver
*/
ngmslib_close(&ngmsReceiver);

return 0;

/**
* NGMS Callback operation which blocks until a complete video frame has been received and decoded
*/
jint Java_com_example_ngmsclient_ExampleChat_ngmsReceiveFrame (JNIEnv* _env, jobject _thiz, jbyteArray frameBytes)

jstring str;

jboolean copy;
jbyte* buffer = (*_env)->GetByteArrayElements(_env, frameBytes, ©);
jsize max = (*_env)->GetArrayLength(_env, frameBytes);
int frameSize;

frameSize = ngmslib_readVidFrame(&ngmsReceiver, buffer, max, NULL);

(*_env)->ReleaseByteArrayElements(_env, frameBytes, buffer, JNI_ABORT);

return frameSize;

/**
* Starts the NGMS streaming service
*/
jint Java_com_example_ngmsclient_ExampleChat_ngmsStartSender (JNIEnv* _env, jobject _thiz, jstring remoteAddress)

char xcoderConfig[512];
char filter[512];
char destination[512];
const char *p;
int i;

/*
* Before doing anything call ngmslib_open to open the NGMS service
*/
i = ngmslib_open(&ngmsSender);

if (i!= NGMS_RC_OK)

__android_log_print(ANDROID_LOG_ERROR, "ngms", "ngmslib_open failed with error %d", i);
return -1;

/*
* Get the destination host string
*/
p = (*_env)->GetStringUTFChars(_env, remoteAddress, 0);

sprintf(destination, "rtp://%s:5004", p);

(*_env)->ReleaseStringUTFChars(_env, remoteAddress, p);

/*
* Customize the stream configuration parameters
*/
ngmsSender.output = destination;
sprintf(xcoderConfig, "vc=h264,vp=66,vb=250,vx=320,vy=240,vgmax=2000,vgmin=1500,vfr=15,vcfrout=-1,vth=1,vl=1,vsc=1,vf=2");
ngmsSender.strxcode = xcoderConfig;
sprintf(filter, "type=yuv420sp, ngmsSender.strfilters[0] = filter;
ngmsSender.inputs[0]= "/dev/dummyvideo";
ngmsSender.noaud = 1;

/*
* Start the RTP sender
*/
i = ngmslib_stream(&ngmsSender);

if (i!= NGMS_RC_OK)

__android_log_print(ANDROID_LOG_ERROR, "ngms", "ngmslib_stream failed with error %d", i);
ngmslib_close(&ngmsSender);
return -1;

return 0;

/**
* Stops the NGMS streaming service
*/
jint Java_com_example_ngmsclient_ExampleChat_ngmsStopSender (JNIEnv* _env, jobject _thiz)

/*
* Stop the RTP sender
*/
ngmslib_close(&ngmsSender);

return 0;

/**
* Encodes and transmits a single video frame using local RTP transport
*/
jint Java_com_example_ngmsclient_ExampleChat_ngmsTransmitFrame (JNIEnv* _env, jobject _thiz, jbyteArray frameBytes)

jboolean copy;
jbyte* buffer = (*_env)->GetByteArrayElements(_env, frameBytes, ©);
jsize size = (*_env)->GetArrayLength(_env, frameBytes);
int i;

i = ngmslib_onVidFrame(&ngmsSender, buffer, size);

(*_env)->ReleaseByteArrayElements(_env, frameBytes, buffer, JNI_ABORT);

return i;

We create makefile Android.mk, which is used by Android NDK to build our ngmsglue library.


#Example Android.mk Makefile

LOCAL_PATH:= $(call my-dir)

include $(CLEAR_VARS)

LOCAL_MODULE:= ngmsglue
LOCAL_SRC_FILES:= ngmsglue.c
LOCAL_C_INCLUDES:= ngms/include
LOCAL_LDFLAGS:= -Lngms/lib
LOCAL_LDLIBS:= -llog -lm -lngms -lxcode

include $(BUILD_SHARED_LIBRARY)

To build the shared directory ngmsglue.so, simply do the following.


cd [YOUR NGMSCLIENT PROJECT DIR]/jni
$ANDROID_NDK_HOME/ndk-build

This should produce the output file [YOUR NGMSCLIENT PROJECT DIR]/libs/armeabi/ibngmsglue.so

You may need to manually copy the ngms / lib / libngms.so and ngms / lib / libxcode.so libraries to the libraries / armeabi output directory because the contents of this directory are automatically wrapped in the project.apk application.

Java application layer

The Java application layer consists of application-level logic for the Android app we are building. In order to use the original code from the previous section, we need to create a simple application using Android SDK. This example demonstrates the use of the ExampleChat class that can be used by your own application to provide real-time interactive video streaming. It does not show all the steps needed to create a complete Android streaming video application. An intermediate-level Android developer should be able to complete the task by building on and including the ExampleChat class in their full application.

The ExampleChat class shown below is used to control the NGMS power transmitter and NGMS power receiver. The startSender method can be called to start streaming video from the built-in camera using the camera preview mechanism. Each video frame broadcast by the camera is delivered to the NGMS API, where it will be encoded and transmitted to the remote by means of built-in RTP encapsulation. The method star Recipient can be called to start receiving video from an external instance of the same app. startReceiver starts a thread that executes a blocking poll for the next available received and decoded video frame. Together, these two methods can be used to establish a two-way video stream between two external instances of the same app across a network.


/*
* ExampleChat.java
*
*/

package com.example.ngmsclient;

import android.hardware.Camera;
import android.hardware.Camera.Parameters;
import android.hardware.Camera.PreviewCallback;
import android.graphics.ImageFormat;

public class ExampleChat
{
private boolean myReceiverRunning;
private Camera myCamera;
private PreviewCallback myCameraCallback;

/**
* Change this value to the destination where the video will be streamed
*/
private static final DESTINATION = "10.0.0.1";

/**
* method to initialize and start sending video from the local camera
*/
public void startSender () throws Exception

myCamera = Camera.open();

Parameters parameters = myCamera.getParameters();
parameters.setPreviewFormat(ImageFormat.NV21);
parameters.setPreviewFrameRate(15);
parameters.setPreviewSize(320, 240);

myCamera.setParameters(parameters);

/**
* Call the NGMS native method to perform streaming initialization
*/
ngmsStartSender(DESTINATION);

myCameraCallback = new PreviewCallback()

public void onPreviewFrame(byte[] frameBytes, Camera camera)

/**
* Call the NGMS native routine to encode and transmit the video frame
* to the remote destination using RTP transport.
*/
ngmsTransmitFrame(frameBytes);

myCamera.startPreview();
myCamera.setPreviewCallback(myCameraCallback);

/**
* method to stop sending video
*/
public void stopSender () throws Exception

ngmsStopSender();

/**
* method to start receiving remote video
*/
public void startReceiver () throws Exception

/**
* Call the NGMS native method to perform input initialization
*/
ngmsStartReceiver();

/**
* Start the thread which will poll for received video frames
*/
new Thread(myVideoFrameReceiver).start();

/**
* method to stop receiving remote video
*/
public void stopReceiver () throws Exception

myReceiverRunning = false;
ngmsStopReceiver();

/**
* Thread used to check for decoded video frames
* received by the NGMS native layer
*/
private Runnable myVideoFrameReceiver = new Runnable()

public void run()

/**
* Allocate a buffer to hold a video frame 320 x 240 pixels
* in RGB565 format, taking 16 bits (2 bytes) per pixel.
*/
byte[] frameBytes = new byte[ 320 * 240 * 2 ];
int size;
myReceiverRunning = true;

while(myReceiverRunning)

/**
* Block until a frame has been received and decoded by the native layer
*/
size = ngmsReceiveFrame(frameBytes);
if(size > 0)

/**
* The frameBytes array represents a video frame in RGB565 format.
* These bytes can be converted into a Bitmap object for display
* on the screen.
*/



/**
* Static initialization to load native libraries into address space
*/
static

System.loadLibrary("xcode");
System.loadLibrary("ngms");
System.loadLibrary("ngmsglue");

/**
* Define native method prototypes
*/
public native int ngmsStartReceiver ();
public native int ngmsStopReceiver ();
public native int ngmsReceiveFrame (byte[] frameBytes);

public native int ngmsStartSender (String remoteAddress);
public native int ngmsStopSender ();
public native int ngmsTransmitFrame (byte[] frameBytes);

}

After you integrate the ExampleChat class into your own Android project, make sure that all three of the original libraries are in the project / libs directory so that they will be included in your.apk output.


libs/armeabi/libngms.so
libs/armeabi/libxcode.so
libs/armeabi/libngmsglue.so

This example produces video output to the destination IP address 10.0.0.1. To enable two clients to stream video bidirectional, you must change this IP address to match your remote client.

conclusion

The above example shows how to stream live video from the camera object in real time and how to receive an external live stream and extract each decoded video frame for viewing. The same process stream can be used to add audio streaming to create a complete video chat application. â € ¨

Go to. To view the Nex Gen Media Server (NGMS) Integrated API Guide http://ngmsvid.com/develop.php