Monday, July 4, 2011

Embedded Tomcat in Maven enable SSI support

As in one of my projects I needed to run a Tomcat server for later deployment of the webapp, I decided to use an embedded variant of the tomcat server in maven.

This works really fine as you can just use a maven plugin:

<build>
<finalName>project</finalName>
<outputDirectory>src/main/webapp/WEB-INF/classes</outputDirectory>
<!-- To use the plugin goals in your POM or parent POM -->
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>tomcat-maven-plugin</artifactId>
<configuration>
<mode>both</mode>
<systemProperties>
<org.apache.catalina.level>FINEST</org.apache.catalina.level>
</systemProperties>
</configuration>
</plugin>
</plugins>
</build>


Important here is to add the extra configuration with mode set to both, i.e. allowing to deploy a given context.xml.

This context.xml is needed to tell tomcat that this webapp needs to be executed in provileged mode. (thus privileged="true" )

Otherwise we would get ane xception like this:
Servlet of class org.apache.catalina.ssi.SSIServlet is privileged and cannot be loaded by this web application

So lets create a context.xml under the META-INF (if not existant create it) in the webapp directory:

context.xml
<?xml version="1.0" encoding="UTF-8"?>
<Context antiResourceLocking="false" privileged="true" useHttpOnly="true" />


Lets now configure our web.xml in order to detect .shtml files and process the SSI inside of them.

web.xml
<?xml version="1.0" encoding="UTF-8"?>
<web-app version="2.5" xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd">

<display-name>app</display-name>

<!-- SSI -->

<servlet>
<servlet-name>ssi</servlet-name>
<servlet-class>org.apache.catalina.ssi.SSIServlet</servlet-class>
<init-param>
<param-name>buffered</param-name>
<param-value>0</param-value>
</init-param>
<init-param>
<param-name>debug</param-name>
<param-value>1</param-value>
</init-param>
<init-param>
<param-name>expires</param-name>
<param-value>1</param-value>
</init-param>
<init-param>
<param-name>isVirtualWebappRelative</param-name>
<param-value>1</param-value>
</init-param>
<load-on-startup>4</load-on-startup>
</servlet>

<servlet-mapping>
<servlet-name>ssi</servlet-name>
<url-pattern>*.shtml</url-pattern>
</servlet-mapping>

<mime-mapping>
<extension>shtml</extension>
<mime-type>text/html</mime-type>
</mime-mapping>

</web-app>


As you can see we used the SSI Servlet instead of the SSI Filter, which also could be used. Just look into the documentation of tomcat server.

Important: We need to set the mime type differently than the official documentation says. Otherwise FireFox (tested in 4/5) won't detect the document and instead trys to download it.

Thats all of it!

JPA joinTransaction Problem

When using JPA as your ORM in Java Web applications, you likely use transactions. As transaction-type you specify "RESOURCE_LOCAL" in your persistence.xml.

Suppose we have these two methods:
 public Object create(Object o){
em.getTransaction().begin();
em.persist(o);
em.getTransaction().commit();
return o;
}
and

public void delete(Object entity){
em.getTransaction().begin();
em.remove(entity);
em.getTransaction().commit();
}




The problem which occured when calling multiple times a new transaction was:

"joinTransaction has been called on a resource-local EntityManager"

That means, somehow a joined Transaction should be used, although we didn't configure JTA.
The source of this problem is, that we would call multiple times new transactions, which won't be executed after each other, but in parallel.

The solution was to set all methods in which a transaction is used to synchronized.
This leads to exact processing of the transactions in correct order.

 public synchronized Object create(Object o){
em.getTransaction().begin();
em.persist(o);
em.getTransaction().commit();
return o;
}
and

public synchronized void delete(Object entity){
em.getTransaction().begin();
em.remove(entity);
em.getTransaction().commit();
}

Monday, June 6, 2011

NAT Holepunch solution in Java

As I needed a solution for sending and receiving data in my videochat application over the internet I came across the problem with firewalls and NAT routers.

While searching the internet for existing solutions I bounced over the so called "Nat Holepunch".
See the article on wikipedia .This actually made that much sense to me, I started developing my own proof of concept.

Details

At first I will describe the Server, as it is the basic component. This is the so called "Relay Server".

RelayServer.java

package de.boehme.app.natholepunch;

import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.util.HashSet;
import java.util.Set;

import de.boehme.app.natholepunch.dto.NATDevice;

public class RelayServer {

private Set<NATDevice> inquiredComputers = new HashSet<NATDevice>();

public static void main(String[] args) {
RelayServer rs = new RelayServer();
try {
rs.startServer();
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}

private void startServer() throws IOException, InterruptedException {
DatagramSocket serverSocket = new DatagramSocket(12345);
byte[] receiveData = new byte[50];

while (true) {
DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length);
System.out.println("Listening...");
serverSocket.receive(receivePacket);
String data = new String(receivePacket.getData());

//NAT Device 1
InetAddress incomingIPAddress = receivePacket.getAddress();
int port = receivePacket.getPort();
NATDevice startComp = new NATDevice();
startComp.setPortOfNAT(port);
startComp.setPublicIpOfNAT(incomingIPAddress);

//NAT Device 2
String accordingIP = data.substring(0, data.indexOf(0));
// String accordingPort = data.substring(data.indexOf(':')+1, data.indexOf(0));
NATDevice destinationComp = new NATDevice();
// destinationComp.setPortOfNAT(Integer.valueOf(accordingPort));
destinationComp.setPublicIpOfNAT(InetAddress.getByName(accordingIP));

System.out.println("Checking: "+startComp.getPublicIpOfNAT()+":"+startComp.getPortOfNAT()+" TO: "+destinationComp.getPublicIpOfNAT());
//check here if a matching entry is already present
NATDevice matchComp = checkMatchingNATDevice(destinationComp);
if(matchComp != null){
System.out.println("Already present.. now sending packets to both of them...");
//send to the socket from previously saved first NAT Device infos from the second NAT Device
sendPacket(serverSocket, matchComp, startComp);
Thread.sleep(1000);
sendPacket(serverSocket, startComp, matchComp);

inquiredComputers.remove(startComp);
inquiredComputers.remove(matchComp);
} else{
System.out.println("Adding "+startComp.getPublicIpOfNAT()+":"+startComp.getPortOfNAT()+" AND "+destinationComp.getPublicIpOfNAT()+" for later matching");
inquiredComputers.add(startComp);
}
}

// serverSocket.close();
}

private synchronized NATDevice checkMatchingNATDevice(NATDevice comp){
for(NATDevice c : inquiredComputers){
if(c.getPublicIpOfNAT().equals(comp.getPublicIpOfNAT())){
return c;
}
}
return null;
}

private synchronized void sendPacket(DatagramSocket socket, NATDevice natDeviceHome, NATDevice natDeviceRemote) throws IOException{

byte[] sendData = new byte[50];
InetAddress homeIPAddress = natDeviceHome.getPublicIpOfNAT();
int homePort = natDeviceHome.getPortOfNAT();
//now the remote destination
InetAddress destIPAddress = natDeviceRemote.getPublicIpOfNAT();
int destPort = natDeviceRemote.getPortOfNAT();
String data = destIPAddress.getHostAddress()+":"+destPort+"-"+homePort;
sendData = data.getBytes();
DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, homeIPAddress, homePort);
socket.send(sendPacket);
}

}


Explanation of the code:
The server should be run on a machine that can accept connections on port 12345. This is because the server listens on this specific port number.
Note: As this is only for development purposes the port number was hardcoded, i.e. when in production systems you would create it dynamically.

Going onwards to the most important method: startServer();
In the while loop we can see that on every connection attempt we differentiate between NAT Device 1 and NAT Device 2. How does this work? From the received packet we get a port number and an IP Adress. Both are needed to create a new object "NatDevice". In the data part of the packet we can extract the information we need to get the according second NAT Device.

The combination of NAT Device 1 to NAT Device 2 will be then added to an internal HashSet. That means when now NAT Device 2 connects to the server and the according NAT Device 1 is already present both NAT Devices get informed with the connection credentials of their connection peer.

How does the Client Side look like?

ClientBehindNAT.java

package de.boehme.app.natholepunch;

import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.SocketException;
import java.net.SocketTimeoutException;
import java.net.UnknownHostException;

public class ClientBehindNAT {

private String relayServerIP;
private String destinationIP;
private InetAddress clientIP;

private int socketTimeout;

public static void main(String[] args) {
ClientBehindNAT c = new ClientBehindNAT();
boolean result = c.parseInput(args);

if(result == true){
try {
c.createHole(true);
} catch (SocketTimeoutException e) {
System.out.println("Timeout occured... Try again?");
} catch (IOException e) {
e.printStackTrace();
}
} else{
System.err.println("Usage: <Relay Server IP> (e.g. 192.168.40.1) <Destination IP> (e.g. 192.168.40.128) <Timeout> (in milliseconds)");
}

}

/**
* Main method used for outer access
* @param standAlone
* @return the free NAT port of the Firewall
* @throws IOException
*/
public synchronized int createConnectionToPeer(String relayServerIP, String destinationIP, int socketTimeout) throws SocketTimeoutException, IOException{
this.relayServerIP = relayServerIP;
this.destinationIP = destinationIP;
this.socketTimeout = socketTimeout;
return createHole(false);
}

private boolean parseInput(String[] args){
if(args.length < 3){
return false;
} else if(args[0].equals("") || args[1].equals("") || args[2].equals("")){
return false;
}else{
relayServerIP = args[0];
destinationIP = args[1];
socketTimeout = Integer.valueOf(args[2]);
return true;
}
}

private int createHole(boolean standAlone) throws SocketTimeoutException, IOException{
// InetAddress serverIPAddress = InetAddress.getByName("192.168.40.1");
clientIP = InetAddress.getLocalHost();
byte[] addrInByte = createInternetAddressFromString(relayServerIP);//{(byte) 192, (byte) 168, 40, 1};
InetAddress serverIPAddress = InetAddress.getByAddress(addrInByte);
int port = 12345;
String data = destinationIP;
System.out.println("Try to send: "+data+" TO: "+serverIPAddress+" ON PORT: "+port);
//first send UDP packet to the relay server
DatagramSocket clientSocket = new DatagramSocket();
clientSocket.setSoTimeout(socketTimeout);
sendPacket(clientSocket, serverIPAddress, port, data);
//now wait for the answer of the server
String peer = receivePacket(clientSocket);
clientSocket.close();
//need to make sure all had the chance to receive packets with infos
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
return initPeerConnection(peer, standAlone);
}

private byte[] createInternetAddressFromString(String address){
byte[] inetAddressBytes = new byte[4];
int index = address.indexOf('.');
int i=0;
while((index = address.indexOf('.')) != -1){
int part = Integer.valueOf(address.substring(0, index));
inetAddressBytes[i] = (byte) part;
address = address.substring(index+1);
i++;
}
inetAddressBytes[i] = (byte) Integer.valueOf(address).intValue();
return inetAddressBytes;
}

private synchronized void sendPacket(DatagramSocket socket, InetAddress destIPAddress, int port, String data) throws IOException{
byte[] sendData = new byte[data.length()];
sendData = data.getBytes();
DatagramPacket sendPacket = new DatagramPacket(sendData,
sendData.length, destIPAddress, port);
socket.send(sendPacket);
}

private synchronized String receivePacket(DatagramSocket socket) throws SocketTimeoutException, IOException{
byte[] receiveData = new byte[50];
DatagramPacket receivePacket = new DatagramPacket(receiveData,
receiveData.length);
socket.receive(receivePacket);

String answerFromServer = new String(receivePacket.getData());
answerFromServer = answerFromServer.substring(0, answerFromServer.indexOf(0));
System.out.println(clientIP+" GOT FROM SERVER:" + answerFromServer);
return answerFromServer;
}

private synchronized int initPeerConnection(String peer, boolean standAlone) throws IOException{
String accordingIP = peer.substring(0, peer.indexOf(':'));
String accordingPort = peer.substring(peer.indexOf(':')+1, peer.indexOf('-'));
InetAddress destIPAddress = InetAddress.getByName(accordingIP);
int remotePort = Integer.valueOf(accordingPort);
int natPort = Integer.valueOf(peer.substring(peer.indexOf('-')+1));
//this only applies in stand-alone mode
if(standAlone){
Thread receiver = new Thread(new PeerReceiveThread(natPort));
receiver.start();
Thread sender = new Thread(new PeerSendThread(destIPAddress, remotePort));
sender.start();
}

return natPort;
}

private class PeerReceiveThread implements Runnable{

int natPort;

public PeerReceiveThread(int port){
this.natPort = port;
}

public void run() {
try {
DatagramSocket serverSocket = new DatagramSocket(natPort);
byte[] receiveData = new byte[50];
DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length);
System.out.println(clientIP+" listening on port: "+natPort);
while(true){
serverSocket.receive(receivePacket);
String data = new String(receivePacket.getData());
System.out.println(clientIP+" FROM "+destinationIP+" WITH "+data);
}
} catch (SocketException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}

private class PeerSendThread implements Runnable{

InetAddress destIPAddress;
int port;

public PeerSendThread(InetAddress destIPAddress, int port) throws UnknownHostException{
this.destIPAddress = destIPAddress;
this.port = port;
}

public void run() {
try {
DatagramSocket clientSocket = new DatagramSocket();
while(true){
sendPacket(clientSocket, destIPAddress, port, "Hello from "+clientIP);
Thread.sleep(2000);
}
} catch (SocketException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}

}

The basic principle works like this: (code taken from own videochat application)
ClientBehindNAT c = new ClientBehindNAT();
dataPort = c.createConnectionToPeer(relayServerIP, destAddress, 15000);
controlPort = c.createConnectionToPeer(relayServerIP, destAddress, 15000);

At first we create a new ClientBehindNAT object and then calling createConnectionToPeer() method. This leads to a connection attempt to the listening RelayServer. As we have defined a timeout of 15 seconds the Client won't wait for too long, in case the other peer didn't get connected to the server or any other unforeseen problem occured.

The most important part is the createHole() method. Inside of this method we define the set up to the relay server by sending an initial packet to it. After that we wait until a packet comes back or the timeout occurs.

When everything went well, we extract the valid port number from the data part of the received message and return it back to the method caller.

Download Source Code:
complete Maven Project
natholepunch.zip

Sunday, May 29, 2011

JMF Video Chat Explained - Local Webcam Access



These series should give you insight into the architecture and implementation of my video chat.
This time I will dive into the Local testing mode of the application.

Overview

  • Learn how to access a webcam locally
Lets start


This image shows you the GUI of the local menu. When clicking on "Start capture" a local video of your webcam should show up. If not you might be asked to query first for existing devices.
Another issue was that the driver has some problems in accessing your webcam. Following error may occur: javax.media.NoDataSourceException: Error instantiating class: com.sun.media.protocol.vfw.DataSource : java.io.IOException: Could not connect to capture device

In my opinion, this is a bug in the driver dll, because when clicking another time on "Start capture" the webcam will get connected properly and everything works as usual.

What is now the magic behind that GUI?

Accessing the webcam

When clicking on "Start capture" a method called "startCapture" will be called.

LocalGUI.java

private void startCapture(){

iFrame = new JInternalFrame();
iFrame.setVisible(true);
iFrame.setSize(320, 250);
iFrame.setLocation(10, 40);
iFrame.setBackground(Color.BLACK);
iFrame.setBorder(BorderFactory.createEmptyBorder());
//disable moving of internal frame
BasicInternalFrameUI ui = (BasicInternalFrameUI)iFrame.getUI();
ui.setNorthPane(null);

try {
// activate the camera
String str2 = "vfw:Microsoft WDM Image Capture (Win32):0";
MediaLocator ml = new MediaLocator(str2);
//bei onboard camera gehts erst beim 3. Mal !!! (liegt am Treiber)
//vorher
//Error instantiating class: com.sun.media.protocol.vfw.DataSource : java.io.IOException: Could not connect to capture device
DataSource original = Manager.createDataSource(ml);
iFrame.getContentPane().add(cv.connectAndCapture(original));
} catch (NoDataSourceException e) {
handleStartError(e);return;
} catch (NoPlayerException e) {
handleStartError(e);return;
} catch (CannotRealizeException e) {
handleStartError(e);return;
} catch (IOException e) {
handleStartError(e);return;
} catch (NoTrackAvailableException e) {
handleStartError(e);return;
}
main.add(iFrame);

globalProps.put("localCaptureActive", true);
globalProps.put("localCaptureMainComponents", main.getComponents());
}

How does it work? Basically we create an JInternalFrame for displaying the video later. The important steps are done between the try and catch block. At first we state that we want the VFW - video for windows driver and create a new MediaLocator. From this medialcoator we can create a new DataSource. The last step is to connect to the webcam and capture live images and add these to the JInternalFrame. Let's have a look at the CaptureVideo class that will take care of the rest.

CaptureVideo.java
     public synchronized JPanel connectAndCapture(DataSource dataSource) throws NoDataSourceException, IOException, NoTrackAvailableException, NoPlayerException, CannotRealizeException{

JPanel video = new JPanel();
video.setLayout(new BorderLayout());

// DataSource clone1 = Manager.createCloneableDataSource(dataSource);

Processor processor = Manager.createProcessor(dataSource);
boolean result = waitForState(processor, Processor.Configured);
if (result == false) {
writeToConsole("Cant configure", ConsoleLogType.WARNING, log);
}
TrackControl[] tracks = processor.getTrackControls();
// Do we have at least one track?
if (tracks == null || tracks.length < 1)
throw new NoTrackAvailableException("Did not find any track to play");
Format format = tracks[0].getFormat();
Dimension size = ((VideoFormat) format).getSize();
float frameRate = ((VideoFormat) format).getFrameRate();
writeToConsole(frameRate + " " + size, ConsoleLogType.INFO, log);

VideoFormat jpegFormat = new VideoFormat(VideoFormat.JPEG_RTP,
new Dimension(320, 240), Format.NOT_SPECIFIED,
Format.byteArray, frameRate);
//todo change resolution from received video AND local camera
// tracks[0].setFormat(jpegFormat);
//works only from device not rtp stream
// FormatControl fmtc = ((CaptureDevice)dataSource).getFormatControls()[0];
// fmtc.setFormat(jpegFormat);

System.err.println("Video received as:");
System.err.println(" " + jpegFormat);
// ContentDescriptor cd = new ContentDescriptor(
// ContentDescriptor.RAW_RTP);
// processor.setContentDescriptor(cd);

result = waitForState(processor, Controller.Realized);
if (result == false) {
writeToConsole("Cant realize", ConsoleLogType.WARNING, log);
}

DataSource dataOutput = processor.getDataOutput();
dataOutput.start();

player = Manager.createRealizedPlayer(dataSource);
processor.start();
player.start();

Component comp;
if ((comp = player.getVisualComponent()) != null) {
video.add(comp);
}
return video;
}


What does now happen in this method? As the result will be a JPanel which can be easily attached to GUI components we have to create one. The next step is to create a Processor to have control over the datasource. When the processor is realized, we can create a Player that contains the visualcomponent which can be added to the Panel.

Note: It is important to follow a certain order when starting the processor and the player, otherwise you won't get it to work together.

That's basically everything to get a webcam working.