Secure WebSockets with Jetty

Websockets is a protocol that runs on top of TCP and allows server to push data to the client, unlike with HTTP. Let's see how to use WebSockets with TLS using Eclipse Jetty.

Add following dependencies to the project POM

<dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-server</artifactId>
    <version>9.0.3.v20130506</version>
</dependency>
<dependency>
    <groupId>org.eclipse.jetty.websocket</groupId>
    <artifactId>websocket-server</artifactId>
    <version>9.0.3.v20130506</version>
</dependency>
<dependency>
    <groupId>org.eclipse.jetty.websocket</groupId>
    <artifactId>websocket-client</artifactId>
    <version>9.0.3.v20130506</version>
</dependency>

Create a websocket by annotating with @WebSocket

package org.amila.sample.websocket.server;

import org.eclipse.jetty.websocket.api.RemoteEndpoint;
import org.eclipse.jetty.websocket.api.Session;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketClose;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketConnect;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketMessage;
import org.eclipse.jetty.websocket.api.annotations.WebSocket;

import java.io.IOException;

@WebSocket
public class MyWebSocket {
    private RemoteEndpoint remote;

    @OnWebSocketConnect
    public void onConnect(Session session) {
        System.out.println("WebSocket Opened");
        this.remote = session.getRemote();
    }

    @OnWebSocketMessage
    public void onMessage(String message) {
        System.out.println("Message from Client: " + message);
        try {
            remote.sendString("Hi Client");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    @OnWebSocketClose
    public void onClose(int statusCode, String reason) {
        System.out.println("WebSocket Closed. Code:" + statusCode);
    }
}

This is the jetty server configured with TLS. Pass a SslConnectionFactory when creating the connector to enable secure communication. For this sample, I've generated a keystore and truststore using java keytool and placed them in src/resources.
Call addWebSocket() with your annotated WebSocket pojo to add WebSockets to the server.

package org.amila.sample.websocket.server;

import org.eclipse.jetty.http.HttpVersion;
import org.eclipse.jetty.server.*;
import org.eclipse.jetty.server.handler.ContextHandler;
import org.eclipse.jetty.server.handler.HandlerCollection;
import org.eclipse.jetty.util.resource.FileResource;
import org.eclipse.jetty.util.resource.Resource;
import org.eclipse.jetty.util.ssl.SslContextFactory;
import org.eclipse.jetty.websocket.server.WebSocketHandler;
import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory;

import java.util.ArrayList;
import java.util.List;

public class WebSocketServer {
    private Server server;
    private String host;
    private int port;
    private Resource keyStoreResource;
    private String keyStorePassword;
    private String keyManagerPassword;
    private List<Handler> webSocketHandlerList = new ArrayList<>();

    public static void main(String[] args) throws Exception {
        WebSocketServer webSocketServer = new WebSocketServer();
        webSocketServer.setHost("localhost");
        webSocketServer.setPort(8443);
        webSocketServer.setKeyStoreResource(new FileResource(WebSocketServer.class.getResource("/keystore.jks")));
        webSocketServer.setKeyStorePassword("password");
        webSocketServer.setKeyManagerPassword("password");
        webSocketServer.addWebSocket(MyWebSocket.class, "/");
        webSocketServer.initialize();
        webSocketServer.start();
    }

    public void initialize() {
        server = new Server();
        // connector configuration
        SslContextFactory sslContextFactory = new SslContextFactory();
        sslContextFactory.setKeyStoreResource(keyStoreResource);
        sslContextFactory.setKeyStorePassword(keyStorePassword);
        sslContextFactory.setKeyManagerPassword(keyManagerPassword);
        SslConnectionFactory sslConnectionFactory = new SslConnectionFactory(sslContextFactory, HttpVersion.HTTP_1_1.asString());
        HttpConnectionFactory httpConnectionFactory = new HttpConnectionFactory(new HttpConfiguration());
        ServerConnector sslConnector = new ServerConnector(server, sslConnectionFactory, httpConnectionFactory);
        sslConnector.setHost(host);
        sslConnector.setPort(port);
        server.addConnector(sslConnector);
        // handler configuration
        HandlerCollection handlerCollection = new HandlerCollection();
        handlerCollection.setHandlers(webSocketHandlerList.toArray(new Handler[0]));
        server.setHandler(handlerCollection);
    }

    public void addWebSocket(final Class<?> webSocket, String pathSpec) {
        WebSocketHandler wsHandler = new WebSocketHandler() {
            @Override
            public void configure(WebSocketServletFactory webSocketServletFactory) {
                webSocketServletFactory.register(webSocket);
            }
        };
        ContextHandler wsContextHandler = new ContextHandler();
        wsContextHandler.setHandler(wsHandler);
        wsContextHandler.setContextPath(pathSpec);  // this context path doesn't work ftm
        webSocketHandlerList.add(wsHandler);
    }

    public void start() throws Exception {
        server.start();
        server.join();
    }
    public void stop() throws Exception {
        server.stop();
        server.join();
    }

    public void setHost(String host) {
        this.host = host;
    }
    public void setPort(int port) {
        this.port = port;
    }
    public void setKeyStoreResource(Resource keyStoreResource) {
        this.keyStoreResource = keyStoreResource;
    }
    public void setKeyStorePassword(String keyStorePassword) {
        this.keyStorePassword = keyStorePassword;
    }
    public void setKeyManagerPassword(String keyManagerPassword) {
        this.keyManagerPassword = keyManagerPassword;
    }

}

And finally the client code. WebSocket is included as an inner class. Pass a SslContextFactory when creating the client and sure "wss" as the protocol prefix of the URL.

package org.amila.sample.websocket.client;

import org.eclipse.jetty.util.resource.Resource;
import org.eclipse.jetty.util.ssl.SslContextFactory;
import org.eclipse.jetty.websocket.api.Session;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketClose;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketConnect;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketMessage;
import org.eclipse.jetty.websocket.api.annotations.WebSocket;
import org.eclipse.jetty.websocket.client.ClientUpgradeRequest;
import org.eclipse.jetty.websocket.client.WebSocketClient;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;

public class JettyWebSocketClient {

    public static void main(String[] args) throws IOException, URISyntaxException {
        new JettyWebSocketClient().run(new URI("wss://localhost:8443/"));
    }
    
    public void run(URI destinationUri) throws IOException {

        SslContextFactory sslContextFactory = new SslContextFactory();
        Resource keyStoreResource = Resource.newResource(this.getClass().getResource("/truststore.jks"));
        sslContextFactory.setKeyStoreResource(keyStoreResource);
        sslContextFactory.setKeyStorePassword("password");
        sslContextFactory.setKeyManagerPassword("password");
        WebSocketClient client = new WebSocketClient(sslContextFactory);
        MyWebSocket socket = new MyWebSocket();
        try {
            client.start();
            ClientUpgradeRequest request = new ClientUpgradeRequest();
            System.out.println("Connecting to : " + destinationUri);
            client.connect(socket, destinationUri, request);
            socket.awaitClose(5, TimeUnit.SECONDS);
        } catch (Throwable t) {
            t.printStackTrace();
        } finally {
            try {
                client.stop();
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }

    @WebSocket
    public class MyWebSocket {
        private final CountDownLatch closeLatch = new CountDownLatch(1);

        @OnWebSocketConnect
        public void onConnect(Session session) {
            System.out.println("WebSocket Opened in client side");
            try {
                System.out.println("Sending message: Hi server");
                session.getRemote().sendString("Hi Server");
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

        @OnWebSocketMessage
        public void onMessage(String message) {
            System.out.println("Message from Server: " + message);
        }

        @OnWebSocketClose
        public void onClose(int statusCode, String reason) {
            System.out.println("WebSocket Closed. Code:" + statusCode);
        }

        public boolean awaitClose(int duration, TimeUnit unit) throws InterruptedException {
            return this.closeLatch.await(duration, unit);
        }
    }

}

Creating OSGi Bundles with Maven Bundle Plugin

OSGi is a framework specification to create java applications based on a complete and dynamic component model. That means your application can be entirely consisted of cohesive and loosely-coupled modules. These modules are called "bundles" in OSGi jargon.

There are several implementations including Apache Felix and Eclipse Equinox. These implementations use existing capabilities of Java Archive (jar) files to implement OSGi bundles. If you look inside a general jar file (which is basically a zip file with .jar extension), there will be a file called MANIFEST.MF inside a directory named META-INF. If the jar was built by maven it will contain some meta data like: 
Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven
Built-By: amilas
Build-Jdk: 1.6.0_35

There are manifest headers specific to OSGi so that runtime can recognize the jar file as a bundle ( and more). Bnd is a tool used to insert those headers into jar files.

While there's built-in support for OSGI in famous IDEs such as Eclipse and IntelliJ IDEA, if your project is maven-based, it's convenient to use maven for creating bundles too.
Maven Bundle Plugin can be used for this purpose. We can specify OSGi parameters in the module POM and the bundle plugin will use Bnd to automatically insert them and create an OSGi bundle.

This is a introductory post to show how to use Maven with IntelliJ IDEA to create OSGi bundles.

Goto File-> New Project... and select "Create  project from scratch"
At the next screen, enter give a project name and a location as shown. Make sure to select the type as Maven Module.

Finally enter a group id as shown, and click finish.

This will create a new project and a module from maven archtype.

Now let's create two sub modules for our project. these will be the resulting OSGi bundles.

Goto File-> Add Module... again "Create  project from scratch" and name the module as "module-a". In the final screen "Add module to" and "parent" fields should be automatically filled as shown.

Similarly add another module named module-b

 Create two packages "api" and "impl" inside org.amila.sample.osgi.a in module-a, and similarly in module-b so the project pane will show the project structure as follows.

You can use Bundle activator to perform some tasks when the OSGi framework starts or stops a certain bundle. We can implement org.osgi.framework.BundleActivator interface to utilize this function.
Create a new class Adapter in your impl package in module A. Use the following code:

package org.amila.sample.osgi.a.impl;

import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;

public class Activator implements BundleActivator {

    /**
     * Implements BundleActivator.start().
     * @param bundleContext - the framework context for the bundle.
     **/
    @Override
    public void start(BundleContext bundleContext) throws Exception {
        System.out.println("Module A is starting");
    }

    /**
     * Implements BundleActivator.stop().
     * @param bundleContext - the framework context for the bundle.
     **/
    @Override
    public void stop(BundleContext bundleContext) throws Exception {
        System.out.println("Module A is shutting down");
    }
}

Add the felix dependency to the module POM.


Set the packaging to "bundle" and add the maven bundle plugin.

After these changes, pom.xml of module-a should look like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>OsgiDemo</artifactId>
        <groupId>org.amila.sample.osgi</groupId>
        <version>1.0</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
    <artifactId>module-a</artifactId>
    <packaging>bundle</packaging>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.felix</groupId>
                <artifactId>maven-bundle-plugin</artifactId>
                <version>2.3.7</version>
                <extensions>true</extensions>
                <configuration>
                    <instructions>
                        <Bundle-SymbolicName>${project.groupId}.${project.artifactId}</Bundle-SymbolicName>
                        <Bundle-Name>${project.artifactId}</Bundle-Name>
                        <Bundle-Version>1.0.0</Bundle-Version>
                        <Bundle-Activator>org.amila.sample.osgi.a.impl.Activator</Bundle-Activator>
                        <Private-Package>org.amila.sample.osgi.a.impl</Private-Package>
                   </instructions>
                </configuration>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>org.apache.felix</groupId>
            <artifactId>org.osgi.core</artifactId>
            <version>1.4.0</version>
        </dependency>
    </dependencies>

</project>

Similarly add Activator and bundle configuration for module B.

 Now, you can build the project using maven. Resulting jar files will be OSGi bundles. If you open MANIFEST.MF file of a bundle now it will contain bundle headers as we configured in the module POM.

You don't have to specify packages to import. If your module uses a package from another module, its import header will be automatically added to the manifest.mf file.

Obtaining a List of IP Addresses

During our final year project I wanted to get active ip addresses of a node. This is easily doable using java.net.NetworkInterface
The method below returns a list of IP addresses of a machine except those belonging to loopback or inactive network interfaces. The addresses can be further classified as IPv4 or IPv6 using instance check with java.net.Inet4Address and java.net.Inet6Address

    public static List<InetAddress> getIPAddress() throws SocketException {

        List<InetAddress> ipAddresses = new ArrayList<InetAddress>();
        Enumeration e;
        e = NetworkInterface.getNetworkInterfaces();
        while (e.hasMoreElements()) {
            NetworkInterface ni = (NetworkInterface) e.nextElement();
            if (ni.isLoopback() || !ni.isUp()) continue;

            for (Enumeration e2 = ni.getInetAddresses(); e2.hasMoreElements(); ) {
                InetAddress ip = (InetAddress) e2.nextElement();
                ipAddresses.add(ip);
            }
        }
        return ipAddresses;
    }

Motion Controlled 3D Flight Simulator

I came across this today while going through my PC's hard drive. This is an exhibit we made back in 2010 for exhibition "Exmo" held in University of Moratuwa. It was also exhibited in Techno, Infotel and Deyata Kirula Exhibitions. This kind of things may be quite common these days. However I'm happy that this was an original idea at that time.


Since we had a limited amount of time, we had to reuse some of the segments. 3d world is mostly based on Riemer's tutorials. A 2D array is used to create the buildings of the city. Heightmap technique is used to model the terrain. It looked quite good with mixed textures, flare and water effects.

The program processes webcam feed frame by frame. We used blob counter technique along with several filters to locate an area of predefined color, with the help of AForge. The rectangle covering that area is considered and its center point is calculated. The displacement of this point relative to center of the captured screen is used to maneuver the aircraft in the game. 

Source Code

Written in C#. XNA is used as the gaming framework. AForge is used as the image processing library. I'm sharing the source hoping someone may find it useful, though its quality and readability is not very high.
Install XNA 3.1 and AForge 2.1. You should use Visual Studio 2008.


Get the source code here:
https://github.com/amilamanoj/FlightSim

Dependencies

You need Windows and following installed for this to work

• .NET Framework 3.5
• XNA 3.1
• AForge 2.1

Installers 

I've made an installer with above dependencies. XNA and AForge are embedded in the installer. .NET Framework isn't embedded due to its size. It will be automatically downloaded and installed. You can get the installer here. (~50MB)


If you have these components already installed or prefer to install them manually, use this installer (~17MB)

Playing

Aircraft can be controlled using mouse, keyboard or movement of an object (or your hands with gloves).
Default controller is mouse. You can switch between controllers using Tab during gameplay. Press Q to quit during gameplay.
You can select the webcam and configure the color in options screen. Default color is red. For example, you can use a red colored plastic ball.
If you haven't played a flight simulator before, you should try playing with mouse or keyboard before trying with motion.

Compiler Construction and JavaCC

Scanning and Parsing are two important phases of compilation process.While it is possible to code a scanner and a parser all by hand, it becomes tedios, specially when the grammar of the language gets more complex. There are Lexer and Parser generators so that we can generate them from a specified grammer.

JavaCC is one such compiler generator. Here are key points about JavaCC.

• It can generate both scanner and parser.
• TopDown parsing: generate the parse tree in a top-down manner.
• It generates Recursive Descent parsers. The parser will execute a set of recursive methods to process the input (not based on parse tables)
• Predictive (non-backtracking). It looks ahead a number of symbols to decide which grammar rule to use.
• Supports LL(K) grammars. That means there must not be left recursion, and the grammar should be left refactored.
• Detects left-recursion, possible ambiguities and warns during compiler generation.
  
• Can use different look-ahead values in different local sections
• Can use syntactic look-ahead. Hence can be considered as a LL(*) parser generator.
• Gammar can be provided in EBNF notation. Removing left recursion is easy and the grammar is more readable.
• Provides debugging support: tokenizing / parsing / looking ahead.
• Supports panic-mode error recovery.
• Supports Unicode.
• Lots of good documentation / resources to follow.
• Good tooling support, including an Eclipse plugin, IntelliJ IDEA plugin and a Maven plugin.

Having unicode support means you can build your compiler using any (natural) language you prefer! You'll only have to change the (programming) language keywords for the new (natural) language. You can also change the (context-free) grammar a bit if you want to make your (programming) language closer to the new (natural) language.

Here is my attempt to create a compiler in my own language, සිංහල (Sinhala). This uses a simplified version of c grammer.
https://github.com/amilamanoj/CMinus/

Recording Music from a Musical Instrument

You can record high quality audio by direct cable connection between your computer and musical instrument (such as musical keyboard, electric guitar).

You need a two way audio cable. (with 3.5mm TRS connectors). Just connect it to audio out of the instrument and line-in/mic of the computer.
Most of the instruments have a 6.35mm socket for audio output. In that case you also need a 6.35mm male to 3.5mm female jack stereo audio adapter. You can get these from a local electronics store or a site like amazon, ebay.

Moving on to software, you don't need expensive professional sound editing software. Audacity is a free and open source cross-platform audio recorder/editor and perfectly fits our purpose.
 
Next comes the tricky part. During recording, you hear what you play through the computer. If there's a delay between what you play and what you hear, it's very difficult to play. Let's look at solutions for this.

Windows

When it comes to Windows, drivers pass sound through the system kernel, resulting considerable amount of latency.
This won't be a big issue on sound cards that support ASIO. ASIO is a sound card driver  protocol that minimizes the latency by bypassing operating system layers and connecting directly to the soundcard. However, not many soundcards support ASIO (specially integrated audio chipsets).

ASIO4ALL may help you in this case. It is an independent universal emulated ASIO driver and brings ASIO support to virtually all consumer-grade soundcards and integrated audio chipsets.
The problem here is that, you need a recording program that supports ASIO. Though Audacity can support ASIO, they don't distribute it with ASIO support since it's proprietary and has an incompatible license. You maybe able to find an unofficial version with ASIO support to download, or if you are a software geek, you can compile Audacity yourself with ASIO support.

Linux

Most modern distributions (Ubuntu, Fedora, Mint, OpenSUSE)  use PluseAudio with ALSA. Usually it results in lower latencies. If you are lucky, you may be able to record without any tweaking.
Within audacity, just goto Edit -> Preferences -> Recording and select Software Playthorugh.

If you experience some delay, you can route the input at PulseAudio level by engaging loopback module. Goto terminal and execute:
pactl unload-module module-loopback
In my experience, this can also have some latency, and if your computer (laptop) has a built in microphone, you might get a loud noise.

What worked best for me is using the pacat command. What we do here is capturing the raw audio steam from the specific device we want and playing it back from the output device on the fly, specifying minimum latency. You can quickly get this working by following these instructions. Make sure you have selected "pulse" as both playback and recording device in Audacity Preferences -> Devices, otherwise you won't be able to record while pacat is active.

After you have recorded, you can easily get the mp3 by using Audacity's Export command.

Here is a recording I made. This is a part of the song "Never Forget" in Halo3. Played with Yamaha PSR-E323. Recorded with Audacity in Linux with the last technique above.
Never Forget (cover) by Amila Manoj

AS2 Protocol

AS2 is a protocol that describes how to exchange structured business data securely using the HTTP transfer protocol.

Structured business data exchanged using AS2 protocol can be,

• Electronic Data Interchange (EDI) in either the
• UN Electronic Data Interchange for Administration, Commerce, and Transport (UN/EDIFACT) format or
• The American National Standards Committee (ANSI) X12 format
• XML or any other structured data formats.

Major application of AS2 is to exchange data in EDI formats.

Security is achieved using digital certificates and encryption. Exchanged messages can be signed using to provide security requirements such as authenticity and preventing non-repudiation. They can be also encrypted to provide confidentiality and integrity. Those are optional requirement according to AS2 specification. An AS2 message can have its content in plain text and without a digital signature.

S/MIME is a format and protocol for adding cryptographic signature and/or encryption services to Internet MIME messages. In AS2, files are encoded as attachments in an S/MIME message. This is what we call an AS2 message. Those messages are sent using the HTTP or HTTPS, usually as POST.

Content Types and AS2

 

There can be several content types for AS2 messages.

• When there is no encryption, no signature
• application/EDIxxxx or application/xml
• When there is no encryption, but signature is present
• multipart/signed – message contains two MIME parts
• application/EDIxxxx or application/xml
• application/pkcs7-signature
• When there is Encryption, but no signature is present
• application/pkcs7-mime
• application/EDIxxxx or application/xml (in the decrypted message)
• When there is both encryption and signature
• application/pkcs7-mime
• multipart/signed(encrypted) – decrypted message contains 2 parts
• application/EDIxxxx or /xml)
• application/pkcs7-signature)

Usually AS2 clients are called "Trading partners". When sending a message, they can request an acknowledgement message called MDN (Message Disposition Notification).

There are several options of requesting a MDN. They are Synchronous MDN, Asynchronous MDN and No MDN.

If there are problems receiving or interpreting the original AS2 message, a "failed" MDN may be sent back. Both "failed" MDN and not receiving MDN (when it is requested) are considered as failures according to specs.

Mendelson (GPL) and OpenAS2 (BSD) are open source implementations of AS2 for java, These can be used to send and receive AS2 messages. Specifically, Mendelson has a user-friendly GUI.