Tag Archives: Java

CoheSiVe

I have just added a new project to my github repo: CoheSiVe

From the silly capitalization you can already guess that it’s a CSV library. It differs from other libraries I’ve seen in that it doesn’t attempt to read the whole file in one go, but is uses an event-driven architecture so that your application can decide what to do with each row as it is parsed.

Docs are here: http://www.dataforte.net/software/cohesive/index.html

Share

Cassandra Connection Pool

For a project of mine I’ve had need for a robust, decent connection pool for Cassandra. Since I use the thrift APIs directly without other layers, I decided to use Tomcat 7’s excellent jdbc-pool as a base. The result is my cassandra-connection-pool which at the moment may be fetched from my github repository at http://github.com/tristantarrant/cassandra-connection-pool
It supports most of the features of its parent implementation: connection validation on creation, release and idle, abandoned connection detection, etc.
Let me know if you use it.

Share

Catwalk Model Processor

I have just released the code for Catwalk a Java Annotation Processor for automatically generating derived domain model classes.

Supposing you have a JPA Entity which you want to pass to a servlet stripped of certain private / internal properties, adding a few annotations to the getters you want to expose allows Catwalk to generate a new class with only those properties and convenience methods for converting between the two types of objects.

The project is still missing a few essentials before being useful, such as documentation, proper examples and being uploaded to a Maven repo.

In the following example, a TestModel class is converted to a WebTestModel class:

TestModel.java

package net.dataforte.test.model;

@Model(pattern = "Web#", classPackage = "net.dataforte.test.webmodel")
public class TestModel {
	String s;
	int i;

	@ModelAttribute
	public String getS() {
		return s;
	}

	public void setS(String s) {
		this.s = s;
	}

	public int getI() {
		return i;
	}

	public void setI(int i) {
		this.i = i;
	}
}

WebTestModel.java

package net.dataforte.test.webmodel;

public class WebTestModel {

	private java.lang.String s;

	public WebTestModel() {}

	public WebTestModel(net.dataforte.test.model.TestModel src) {
		this.fromTestModel(src);
	}

	java.lang.String getS() {
		return s;
	}

	void setS(java.lang.String s) {
		this.s = s;
	}

	public WebTestModel fromTestModel(net.dataforte.test.model.TestModel src) {
		this.s = src.getS();
		return this;
	}

	public net.dataforte.test.model.TestModel toTestModel() {
		net.dataforte.test.model.TestModel that = new net.dataforte.test.model.TestModel();
		that.setS(this.s);
		return that;
	}

}
Share

Java and Large Memory Pages on Linux

Recently I helped configure a system for an application running under Tomcat on Linux with very large memory requirements: a minimum heap of 6GB with a maximum of 11GB. The JVM was initially configured to use the Parallel garbage collector. With this configuration garbage collection of the “Young Generation” was fine, but the “Old Generation” GC was taking over 30 seconds (and blocking all other threads while doing this). We looked into enabling Large Memory Pages, a feature of modern CPUs which allow memory-hungry applications to allocate memory in 2MB chunks instead of the standard 4KB. Documentation on the web on how to do this exactly is sparse and missing some details we ran into. Here’s the sequence of steps we had to take:

  1. configure the kernel’s maximum shared memory to span the whole address space (via the kernel.shmmax and kernel.shmall parameters)
  2. configure the kernel’s allocated large memory pages (via the vm.nr_hugepages parameter)
  3. configure the user limits to ensure that the user running Tomcat can allocate the necessary memory (via the maxlock parameter)
  4. ensure that PAM applies the security limits to users who “login” via su and sudo
  5. configure the JVM for Large Memory Pages

Add the following lines to /etc/sysctl.conf and use sysctl -p to reload the changes into the running kernel although I recommend rebooting the system so that the Large Memory pages can be properly allocated (they have to be contiguous).

# Maximum size of a shared memory segment (in bytes)
kernel.shmmax=17179869184
# Maximum total size of all shared memory segments (in pages of 4KB)
kernel.shmall=3145728
# Number of allocated Large Memory Pages (each one takes up 2MB)
vm.nr_hugepages=6144

Edit /etc/security/limits.conf so that the user running the Java application can lock the correct amount of memory.

tomcat soft memlock 12884901888
tomcat hard memlock 12884901888

Edit /etc/pam.d/su and /etc/pam.d/sudo and ensure that they contain the following line so that the above memory limits are applied:

session required pam_limits.so

Next add the relevant options to the JVM’s command-line:

-XX:+UseLargePages -Xmx11g -Xms6g

Share

Too-OneBee

I have started yet another open-source project: Too-OneBee. It’s both an embeddable and standalone web console for monitoring a J2EE application. It lives at Google Code: http://code.google.com/p/too-onebee/
It embeds a javax.scripting console for running scripts within the webapp, a JMX browser, a session/request/application/server context viewer, a JNDI browser.
I would like to add pluggable handlers so that, for example, one could add Spring integration, a JDBC tool (a la phpMyAdmin), etc.

Too-One Bee - Scripting
Too-One Bee - Scripting
Share

Canyon: cleaning up

I am currently cleaning up the various Canyon SPIs (Service Provider Interface) so that the canyon object which is exposed to the scripts now has a standard interface for the various methods therein (show(), hide(), get(), etc). Next up is writing the much needed documentation, tutorial and website. I’ll try to do that when I release 0.6.0

Share

Canyon Core 0.5.1

I released Canyon Core 0.5.1 today. It is available either from the Dataforte Maven Repository at http://www.dataforte.net/listing/maven/releases/ and tagged in SVN at https://canyon.svn.sourceforge.net/svnroot/canyon/tags/canyon-core/canyon-core-0.5.1/

This is the first stable release of Canyon Core, and I’ve put quite a bit of work into it over the past few years. Releases of Canyon Echo2, Canyon Cooee and Canyon Echo3 will follow shortly.

Share

Canyon: more JSR-223 and the SimpleBindings

In order to make the widgets available as object during script execution, I have extended javax.script.SimpleBindings so that I lookup keys in the widget hierarchy before handing it down to the backing map by overriding the get() method. This was how I did it when using the Groovy implementation directly. I have discovered that javax.script requires that I override containsKey() as well as get(), otherwise it doesn’t even attempt to invoke get(). Now that is fixed in SVN and I am nearing the first public release of Canyon.

Share

Canyon: JSR-223

Canyon is my implementation of the concepts behind various kinds of XML-based application interface tools (XAML, Flash MXML, ZK, etc).
Canyon started off as a way to describe Echo2 (http://echo.nextapp.com) applications using XML bound together with Groovy (http://groovy.codehaus.org) scripts.
Recently I’ve been working on refactoring Canyon to support multiple widget libraries (Echo2, Cooee, Echo3, Swing, SWT, etc) and JSR-223 (aka javax.script) instead of just Groovy. Currently canyon and its bits are available on Sourceforge SVN (http://sourceforge.net/projects/canyon).

I promise I will hopefully get a half-way decent site for all these projects of mine and create some sort of community going.

Share

WARShield: progress

The WARShield stuff progresses nicely: it now implements most of the feature-set I had in mind.

I have also implemented an event/listener method so that other ContextListeners / Servlets /Filters can get notified when the configuration is readily available, changes, etc.

Currently the configuration is stored in a single properties file, but I am adding a “multi-resource” method so that applications supply a set of template files which get populated with the configuration data (similar to the maven resource filtering by using the ${} notation).

I plan on adding hooks for containers such as Spring, so that they look for their data in the appropriate place and wait until the configuration is ready before reading it.

Share