Sunday, May 29, 2011

JMF Video Chat Explained - Local Webcam Access



These series should give you insight into the architecture and implementation of my video chat.
This time I will dive into the Local testing mode of the application.

Overview

  • Learn how to access a webcam locally
Lets start


This image shows you the GUI of the local menu. When clicking on "Start capture" a local video of your webcam should show up. If not you might be asked to query first for existing devices.
Another issue was that the driver has some problems in accessing your webcam. Following error may occur: javax.media.NoDataSourceException: Error instantiating class: com.sun.media.protocol.vfw.DataSource : java.io.IOException: Could not connect to capture device

In my opinion, this is a bug in the driver dll, because when clicking another time on "Start capture" the webcam will get connected properly and everything works as usual.

What is now the magic behind that GUI?

Accessing the webcam

When clicking on "Start capture" a method called "startCapture" will be called.

LocalGUI.java

private void startCapture(){

iFrame = new JInternalFrame();
iFrame.setVisible(true);
iFrame.setSize(320, 250);
iFrame.setLocation(10, 40);
iFrame.setBackground(Color.BLACK);
iFrame.setBorder(BorderFactory.createEmptyBorder());
//disable moving of internal frame
BasicInternalFrameUI ui = (BasicInternalFrameUI)iFrame.getUI();
ui.setNorthPane(null);

try {
// activate the camera
String str2 = "vfw:Microsoft WDM Image Capture (Win32):0";
MediaLocator ml = new MediaLocator(str2);
//bei onboard camera gehts erst beim 3. Mal !!! (liegt am Treiber)
//vorher
//Error instantiating class: com.sun.media.protocol.vfw.DataSource : java.io.IOException: Could not connect to capture device
DataSource original = Manager.createDataSource(ml);
iFrame.getContentPane().add(cv.connectAndCapture(original));
} catch (NoDataSourceException e) {
handleStartError(e);return;
} catch (NoPlayerException e) {
handleStartError(e);return;
} catch (CannotRealizeException e) {
handleStartError(e);return;
} catch (IOException e) {
handleStartError(e);return;
} catch (NoTrackAvailableException e) {
handleStartError(e);return;
}
main.add(iFrame);

globalProps.put("localCaptureActive", true);
globalProps.put("localCaptureMainComponents", main.getComponents());
}

How does it work? Basically we create an JInternalFrame for displaying the video later. The important steps are done between the try and catch block. At first we state that we want the VFW - video for windows driver and create a new MediaLocator. From this medialcoator we can create a new DataSource. The last step is to connect to the webcam and capture live images and add these to the JInternalFrame. Let's have a look at the CaptureVideo class that will take care of the rest.

CaptureVideo.java
     public synchronized JPanel connectAndCapture(DataSource dataSource) throws NoDataSourceException, IOException, NoTrackAvailableException, NoPlayerException, CannotRealizeException{

JPanel video = new JPanel();
video.setLayout(new BorderLayout());

// DataSource clone1 = Manager.createCloneableDataSource(dataSource);

Processor processor = Manager.createProcessor(dataSource);
boolean result = waitForState(processor, Processor.Configured);
if (result == false) {
writeToConsole("Cant configure", ConsoleLogType.WARNING, log);
}
TrackControl[] tracks = processor.getTrackControls();
// Do we have at least one track?
if (tracks == null || tracks.length < 1)
throw new NoTrackAvailableException("Did not find any track to play");
Format format = tracks[0].getFormat();
Dimension size = ((VideoFormat) format).getSize();
float frameRate = ((VideoFormat) format).getFrameRate();
writeToConsole(frameRate + " " + size, ConsoleLogType.INFO, log);

VideoFormat jpegFormat = new VideoFormat(VideoFormat.JPEG_RTP,
new Dimension(320, 240), Format.NOT_SPECIFIED,
Format.byteArray, frameRate);
//todo change resolution from received video AND local camera
// tracks[0].setFormat(jpegFormat);
//works only from device not rtp stream
// FormatControl fmtc = ((CaptureDevice)dataSource).getFormatControls()[0];
// fmtc.setFormat(jpegFormat);

System.err.println("Video received as:");
System.err.println(" " + jpegFormat);
// ContentDescriptor cd = new ContentDescriptor(
// ContentDescriptor.RAW_RTP);
// processor.setContentDescriptor(cd);

result = waitForState(processor, Controller.Realized);
if (result == false) {
writeToConsole("Cant realize", ConsoleLogType.WARNING, log);
}

DataSource dataOutput = processor.getDataOutput();
dataOutput.start();

player = Manager.createRealizedPlayer(dataSource);
processor.start();
player.start();

Component comp;
if ((comp = player.getVisualComponent()) != null) {
video.add(comp);
}
return video;
}


What does now happen in this method? As the result will be a JPanel which can be easily attached to GUI components we have to create one. The next step is to create a Processor to have control over the datasource. When the processor is realized, we can create a Player that contains the visualcomponent which can be added to the Panel.

Note: It is important to follow a certain order when starting the processor and the player, otherwise you won't get it to work together.

That's basically everything to get a webcam working.

5 comments:

  1. Hi,

    Your videoChat application is great, a stand alone JMF application without installing JMF locally is what I really need for my app, but I'm having a bit of a problem.

    When my code tries to Realize the processor it throws a message :

    "Unable to handle format: YUV Video Format: Size = java.awt.Dimension[width=640,height=480] MaxDataLength = 614400 DataType = class [B yuvType = 32 StrideY = 1280 StrideUV = 1280 OffsetY = 0 OffsetU = 1 OffsetV = 3"

    then

    "Failed to realize: com.sun.media.ProcessEngine@fd1810"


    Can you help me?

    btw, I use the jmfcustomizer and check all options and used the output jar to run my code.

    Thanks.

    ReplyDelete
  2. How do I get a TV via an antenna polaris conexan video capture?, When I select this device shows nada.Por please help, this is my mail yjulianhc@gmail.com. thank you very much

    ReplyDelete
  3. I am having the following error, when I select to chooser a video device:

    "Unable to handle format: YUV Video Format: Size = java.awt.Dimension[width=640,height=480] MaxDataLength = 614400 DataType = class [B yuvType = 32 StrideY = 1280 StrideUV = 1280 OffsetY = 0 OffsetU = 1 OffsetV = 3"

    ReplyDelete