mserver.log

Has been a while since I last blogged, but I have been rather busy lately. Today I have done what I wanted to be doing a long time ago. I wrote a ztemplates service for sending out emails, using the powers of the rendering engine to create HTML emails. So I came up with the following service (note that I wrote this for version 0.9.9.6):

public class ZEmailService implements ZIEmailService {
 
	@Override
	public void render(Object obj, final String from, final String[] recipients, final String subject) throws Exception {
		render(obj, from, recipients, subject, "text/html");
	}
 
	@Override
	public void render(Object obj, final String from, final String[] recipients, final String subject, String mimeType) throws Exception {
 
		if(obj.getClass().isAnnotationPresent(ZEmail.class) == false) {
			throw new Exception("@ZEmail annotation not present on " + obj.getClass().getCanonicalName());
		}
 
		final ZIRenderService renderService = ZTemplates.getRenderService();
		final ZEmail emailAnnotation = obj.getClass().getAnnotation(ZEmail.class);
		final Properties properties = new Properties();
		final String body = renderService.render(obj);
 
		properties.put("mail.smtp.host", emailAnnotation.host());
 
		if(emailAnnotation.username() != null) {
			properties.put("mail.user", emailAnnotation.username());
		}
		if(emailAnnotation.username() != null) {
			properties.put("mail.password", emailAnnotation.password());
		}
		sendMail(properties, from, recipients, subject, mimeType, body);
	}
 
	private static void sendMail(final Properties properties, final String from, final String[] recipients, final String subject, final String mimeType, final String body) throws MessagingException {
		final Session session = Session.getDefaultInstance(properties, null);
		final Message message = new MimeMessage(session);
		final InternetAddress addressFrom = new InternetAddress(from);
		final InternetAddress[] addressTo = new InternetAddress[recipients.length];
 
		for (int i = 0; i < recipients.length; i++) {
			addressTo[i] = new InternetAddress(recipients[i]);
		}
 
		message.setFrom(addressFrom);
		message.setRecipients(Message.RecipientType.TO, addressTo);
		message.setSubject(subject);
		message.setContent(body, mimeType);
 
		Transport.send(message);
	}
}

Here is the interface:

public interface ZIEmailService extends ZIService {
 
	  public void render(Object obj, final String from, final String[] recipients, final String subject) throws Exception;
	  public void render(Object obj, final String from, final String[] recipients, final String subject, String mimeType) throws Exception;
}

And the code for the @ZEmail annotation:

@Retention(RetentionPolicy.RUNTIME)
public @interface ZEmail {
	String host() default "localhost";
	String username();
	String password();
}

Let’s say we have a view like:

@ZRenderer(value = ZJspRenderer.class)
@ZEmail(password = "secret", host = "somehost", username = "admin")
public class SomeView {
 
    @ZExpose
    public String someValue() {
    }
}

As you can see the @ZEmail annotation makes it possible to set mail server settings. To actually send the mail using that view we only have to do the following:

new ZEmailService().render(new SomeView(), "from@example.org", new String[] { "to@example.org" }, "subject");

As you can see I’m not using the ServiceRepository in this example, I added the service to ztemplates myself, so I’m running a custom version. Just instantiating the service works as well.

There are of course a lot of improvements possible, but this version matches my needs. I hope someone will have some use of it.

Note: Will probably also work using other renderers, didn’t test it

Required external libraries:

• mail.jar
• activation.jar

Hope to be blogging some more soon.

Quite recently I have been in a situation where I wanted to expose some XML using ZTemplates. So I used the servlet serivce to get the HttpServletResponse instance:

final HttpServletResponse response = ZTemplates.getServletService().getResponse();
response.setCharacterEncoding("utf-8");
response.setContentType("application/xml");

Followed by rendering a view:

ZTemplates.getServletService().render(view);

When I inspected the HTTP traffic using wireshark I noticed that I saw:

Content-Type: text/html;charset=utf-8

Rather than the expected application/xml. So I started looking into the code of the ZServletService that is responsible for rendering:

public void render(Object obj) throws Exception {
    render(obj, "text/html");
}
 
public void render(Object obj, String mimeType) throws Exception {
    String html = renderService.render(obj);
    // response.setCharacterEncoding(UTF8);
    response.setContentType(mimeType);
    response.getWriter().print(html);
}

That explained a lot, so now you could go ahead and do:

ZTemplates.getServletService().render(view, "application/xml");

However that does not give you control on the encoding, so I decided to create a custom rendering method instead:

public static void render(Object view, String encoding, String contentType ) throws Exception {
    final HttpServletResponse response = ZTemplates.getServletService().getResponse();
    final String renderResponse = ZTemplates.getRenderService().render(view);
    response.setCharacterEncoding(encoding);
    response.setContentType(contentType);
    response.getWriter().print(renderResponse);
}

So to fix my problem I only have to do the following now:

ZRenderUtils.render(view, "utf-8", "application/xml");

p.s. If you are wondering in this example, view, is just a String containing XML.

This is the first article in a series of really short articles with problems I come across in my day to day development life. You can recognize these articles by the ‘[Quicky]‘ reference in the title and of course a ‘quicky’ tag is supplied. These ‘quickies’ don’t supply much explanation on why the problem exists, but more how to solve it.

Today I had to compare a variable with a string value using JSTL. Something like:

<c:choose>
  <c:when test="${variable eq "expected value"}">
    something
  </c:when>
</c:choose>

will obviously not work, because the XML becomes invalid. The way to do it however is:

<c:choose>
  <c:when test="${variable eq 'expected value'}">
    something
  </c:when>
</c:choose>

Replacing the double quotes surrounding the string value by single quotes does the trick.

My previous article about ztemplates was about exposed values inside JSP code blocks. There is a also another possible approach and that is to use the JavaServer Pages Standard Tag Library. So let’s rewrite my last code example in the previous article.

Previous example:

<%
if(condition == true) {
 
  // notice the cast to the correct class type
  String message = (String)request.getAttribute("message");
 
  // When debugging it would be nice to check if the object is valid.
  if(message == null) {
 
    throw new Exception("DEBUG: message not exposed by render POJO.");
  }
 
  out.println(message);
}
%>

New example to see if the exposed ‘message’ field was set (with complete XHTML this time) :

<?xml version="1.0" encoding="UTF-8" ?>
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
 
<!-- NOTICE: load the tag library -->
<%@taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>Example</title>
  </head>
  <body>
    <c:choose>
      <c:when test="${message is null}">
        <p>You didn't provide a message!</p>
      </c:when>
      <c:otherwise>
        <p>Here is your message: ${message}</p>
      </c:otherwise>
    </c:choose>
  </body>
</html>

As you can see this is a much more robust way of doing things. Also take a look at this page for a more thorough introduction to JSTL.

Previously I worked with Apache Velocity as a rendering engine when using ZTemplates. However, I decided to use JSP instead.

Imagine having the following render POJO:

SampleView.java

@ZRenderer(ZJspRenderer.class)
public class SampleView {
 
  String message;
 
  @ZExpose
  public String getMessage() {
    return this.message;
  }
}

In the JSP template this could be used when doing something like:

SampleView.jsp

<html>
  <body>
    <h1>${message}</h1>
  </body>
</html>

However what would you do when having special conditions triggering the rendering of the message? You would try to do something like:

<%
if(condition == true) {
 
  out.println(${message});
}
%>

However since Java tries to compile the JSP it will give you an error saying there is a syntax error on the line where you print the ${message}. After having a look in the ZTemplates source code I discovered that ServletRequest.setAttribute() is being used for exposing values with a ZJspRenderer. This would mean JSP’s native request object would provide us all the exposed values for usage in our code blocks.

This would leave us with doing:

<%
if(condition == true) {
 
  // notice the cast to the correct class type
  String message = (String)request.getAttribute("message");
 
  // When debugging it would be nice to check if the object is valid.
  if(message == null) {
 
    throw new Exception("DEBUG: message not exposed by render POJO.");
  }
 
  out.println(message);
}
%>

It would be nice if ZTemplates would provide a wrapper for this, including a checker that throws an exception when the object is null. With this I mean a custom exception and not the normal NullPointerException, because most of the times it gives very big stack traces.

However this would also mean you’d have to import it in your JSP file as an import, because ZTemplates would have to provide a class for this. Not sure if this is what I would like to see.

Abstract

This article introduces ztemplates, a multi-tier open source web application framework for Java. ztemplates is licensed under the Apache 2.0 license. It uses Java annotations, thus meaning at least Java 1.5 or higher is required. ztemplates has a lot of features, but we will only introduce the basics in this post. This article will not describe how to set-up a project. Instead, it will explain the basics of the architecture of ztemplates.

ztemplates is a good framework. Its missing features are not critical issues nor will they cause problems in your day-to-day work.

public final Word = "See what ztemplates can do for you!";

Continue here

Still working with hibernate for use with an Asterisk machine and a Postgres database server. This time I am try to get extensions.conf in the database. There is a predefined model for this for use with postgres+asterisk:

CREATE TABLE extensions (
  id serial,
  context varchar(40) NOT NULL DEFAULT '',
  exten varchar(40) NOT NULL DEFAULT '',
  priority int4 NOT NULL DEFAULT 0,
  app varchar(40) NOT NULL DEFAULT '',
  appdata varchar(256) NOT NULL DEFAULT '',
  PRIMARY KEY  (context,exten,priority),
  UNIQUE (id)
);

Now I wanted a POJO with a composite key(context, exten, priority) for this. So I started coding:

@Entity
@Table(name = "asterisk_extensions")
public class Extension implements Serializable {
 
    @GeneratedValue(strategy = GenerationType.TABLE)
    @NotNull
    @Column(name = "id")
    private int id;
 
    @Id
    @NotNull
    @Column(name = "context", length = 40)
    private String context;
 
    @Id
    @NotNull
    @Column(name = "exten", length = 40)
    private String extension;
 
    @Id
    @NotNull
    @Column(name = "priority")
    private int priority;
 
    @NotNull
    @Column(name = "app", length = 40)
    private String application;
 
    @NotNull
    @Column(name = "appdata", length = 256)
    private String applicationData;
 
    // getters and setters
    [...]
}

This won’t do the trick. I can’t specify three @Id annotations for this POJO. Also the @GeneratedValue doesn’t work since it’s not annotated as @Id but I will come to that later on.

So to get a composite we need something else. We will need the @IdClass annotation for the calls to specify a class to use as a reference by Hibernate for handling the composite primary key. That class needs to be @Embeddable. And also have all the fields available for the composite key. Furthermore it should implement a hashCode() and equals() method.

Let’s put all of this together:

Extension.java:

@Entity
@Table(name = "asterisk_extensions")
@IdClass(ExtensionPK.class)
public class Extension implements Serializable {
 
    @GeneratedValue(strategy = GenerationType.TABLE)
    @NotNull
    @Column(name = "id")
    private int id;
 
    @Id
    @NotNull
    @Column(name = "context", length = 40)
    private String context;
 
    @Id
    @NotNull
    @Column(name = "exten", length = 40)
    private String extension;
 
    @Id
    @NotNull
    @Column(name = "priority")
    private int priority;
 
    @NotNull
    @Column(name = "app", length = 40)
    private String application;
 
    @NotNull
    @Column(name = "appdata", length = 256)
    private String applicationData;
 
    // getters and setters
    [...]
}

ExtensionPK.java

@Embeddable
public class ExtensionPK implements Serializable {
 
    @NotNull()
    @Column(name = "context", length = 40)
    private String context;
 
    @NotNull
    @Column(name = "exten", length = 40)
    private String extension;
 
    @NotNull
    @Column(name = "priority")
    private int priority;
 
    @Override
    public boolean equals(Object object) {
 
        if (object instanceof ExtensionPK) {
 
            ExtensionPK extensionPK = (ExtensionPK)object;
 
            return (this.context.equals(extensionPK.context) && this.extension.equals(extensionPK.extension) && this.priority == extensionPK.priority);
        }
 
        return false;
    }
 
    @Override
    public int hashCode() {
 
        String hash = this.context + this.extension + this.priority;
 
        // let String do the job for us.
        return hash.hashCode();
    }
 
    // getters and setters
    [...]
 
}

This will solve our first problem regarding the composite key. That wasn’t too difficult, now was it?

Ok, let’s move on to our second problem. Since the ‘id’ field is not @Id annotated, somehow Hibernate will not create a sequence in the database for auto incrementing the value. So since it’s an application using only postgres anyway I see no problem in getting our friend ‘columnDefinition’ back again:

@NotNull
@Column(name = "id", columnDefinition = "bigserial")
private int id;

Which will do the trick!

Note: As a reminder, I let hibernate create my database schema. Important not to forget to recreate the schema!

It has been a while since I last posted here. Meanwhile I learned lots of stuff at Lunatech Research.

Anyway, let’s talk about some of the more technical stuff. As a hobby project I’m writing a Java application that is able to interface with an Asterisk instance. I do this by mapping Asterisk to a database using unixODBC. Via the database I can control Asterisk. Asterisk does require specific column properties on the database tables.

For the Java application part I use Hibernate and PostgreSQL. I use the annotation support in hibernate to define my POJO’s. Also I am letting Hibernate create my database schema. Asterisk however, needs some special column properties which are database specific. So using Hibernate to create the schema also caused a bit of a problem for me, because there is no annotation for setting the ‘DEFAULT NOW()’ property for a column. This is just one example of course.

Here’s the SQL statement for use with asterisk+postgres as found here:

CREATE TABLE cdr (
  calldate timestamp WITH time zone NOT NULL DEFAULT now(),
  clid varchar(80) NOT NULL DEFAULT '', 
  src varchar(80) NOT NULL DEFAULT '', 
  dst varchar(80) NOT NULL DEFAULT '', 
  dcontext varchar(80) NOT NULL DEFAULT '', 
  channel varchar(80) NOT NULL DEFAULT '', 
  dstchannel varchar(80) NOT NULL DEFAULT '', 
  lastapp varchar(80) NOT NULL DEFAULT '', 
  lastdata varchar(80) NOT NULL DEFAULT '', 
  duration bigint NOT NULL DEFAULT '0', 
  billsec bigint NOT NULL DEFAULT '0', 
  disposition varchar(45) NOT NULL DEFAULT '', 
  amaflags bigint NOT NULL DEFAULT '0', 
  accountcode varchar(20) NOT NULL DEFAULT '', 
  uniqueid varchar(32) NOT NULL DEFAULT '', 
  userfield varchar(255) NOT NULL DEFAULT '' 
);

This became a bit of an annoyance for me. I do understand their point that it’s database specific. I was more upset with the fact that it wasn’t obvious which annotation I had to use to tackle this problem.

Finally after reading this page thoroughly enough I concluded I had to use the ‘columnDefinition’ in the ‘@Column’ annotation.

Here is an example for the ‘calldate’ field:

@Entity
@Table(name = "asterisk_cdr")
public class CallDetailRecord implements Serializable {
 
    /** The time the call was placed */
    @Column(name = "calldate", columnDefinition = "timestamp with time zone NOT NULL default now()")
    @Temporal(TemporalType.TIMESTAMP)
    private Date callDate;
 
    [...]
}

So in the end it’s not all that bad since I’m using only one database engine for my application anyway. So, no need to switch later on, meaning that for me the ‘columnDefinition’ is something I will be using a lot more in this application.

As said in my previous post I have been writing a simple webcrawler. Today I decided to share it with you.

It’s licensed under the GPL license. And also I would like you to know that I’m not responsible for any damage caused by this piece of code to your properties or others. Use at your own risk!

Features:

• serial crawling
• queuing of url’s
• url trackback (parent field in database table crawler_url)
• sha-1 hashing of page contents (see crawler_url_cache table)

Feel free to extend the features. It’s not a complete crawler at all, it’s just a very basic one.

If you found my previous post interesting you should definitely have a look at: src/Database/Database.java. It’s a class that handles all database requests. Also notice that it only allows “DAO” objects for database requests. This I did to create an abstraction layer between code and database.

The tar.gz file contains the following:

• Netbeans project
• jar file in dist/
• sql for the sql database in src/Database/database.sql
• code in src/
• GPL license in src/
• JavaDoc in dist/javadoc

You can simple run the jar (compiled against 1.6 JDK) file in the dist directory as:

java -jar JWebCrawler.jar http://blog.mserver.nl

Where http://blog.mserver.nl is the url to use to start crawling from.

Note that you should have a database setup with the following connection parameters:

host: localhost
user: crawler
password: crawler
database: crawler

The database should be filled with the correct tables by inserting the sql in src/Database/database.sql. You will probably like to recompile this package yourself.

What you will need to build/run the netbeans project yourself:

• At least netbeans 6
• at least JDK 1.5
• Postgres JDBC connector
• A running postgres database server
• A database with correct sql, see src/Database/database.sql

In the netbeans project I’ve set http://blog.mserver.nl as the parameter to use when running the project from netbeans if you are wondering.

Enough said, it’s a basic crawler and you can find the package here.

Things you could do to improve operations:

• Parallel instead of serial crawling
• No sha-1 caching but instead caching entire contents
• Set user agent
• Add more non-crawlable types filters like .avi files
• Some other 1337 function

Enjoy!

Due to my internship at Lunatech that will start in September, I have been paying more attention to Java programming these last weeks. You probably noticed some articles about Netbeans and Tomcat. This time I’m going somewhat more in dept.

In the previous school year we did some projects with Java. Those projects incorporated web applications and a banking application. As a datastore for our data we used Postgres. This was nice and all but we we were not using the Postgres JDBC driver thoroughly enough. So today I decided to create a web crawler with a Postgres database as the data backend with using some more JDBC functions, instead of creating wrappers. This time I focused on transactions specifically.

After a hard days work, the crawler is successfully getting data. That’s fine and all but I would like to show how I interfaced with the database. It’s like a tutorial. I will assume you have a valid JDBC driver installed.

We will need the following imports:

1
2
3
4
5
6
7

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Savepoint;
import java.sql.Statement;
import java.util.Properties;

The first thing you should do is to load the driver:

1
2
3
4
5
6
7
8
9
10
11
12

try {
    // load the driver
    Class.forName("org.postgresql.Driver");
 
} catch (ClassNotFoundException e) {
 
    // print on console
    System.err.println(e.toString);
 
    // take appropriate actions
    [...]
}

OK that looks easy, but don’t forget to catch the exceptions otherwise you will be faced with compiler errors. Let’s connect to the database now:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

// using lot's of variable to stay on the clear side
String _username = "jdbctest";
String _password = "jdbctest";
String _address = "localhost"; // the server address
String _database = "jdbctest"; // the database to use
 
// create a new connection url
String _url = "jdbc:postgresql://" + _address + "/" + _database;
 
// create a property manager
String _properties = new Properties();
 
// create properties
String _properties.setProperty("user", _username);
String _properties.setProperty("password", _password);
 
Connection _connection = null;
 
try {
    // get the connection
    _connection = DriverManager.getConnection(_url, _properties);
 
    // in my case I want serializable transaction isolation 
    _connection.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE);
 
} catch (SQLException e) {
 
    // print on console
    System.err.println(e.toString());
 
    // take appropriate actions
    [...]
}

For extra connection properties, see this page. Now we should have a working connection. Now, the fun part, creating transactions and do queries in those transactions.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

// a savepoint for starting a transaction
Savepoint _savepoint = null;
 
// a controller            
boolean _transactionEnded = false;
 
try {
    // in order to start a transaction we need to set AutoCommit to false
    _connection.setAutoCommit(false);
 
    // create a statement, could also be a prepared statement, whichever you prefer
    Statement _statement = _connection.createStatement();
 
    // create a new savepoint to use for the transaction (this is the start of the transaction)
    _savepoint = _connection.setSavepoint("somerandomvalue");
 
    // BEGIN
 
    // execute the query
    _statement.execute("SELECT * FROM TEST");
 
    // try commit
    _connection.commit();
 
    // COMMIT
 
    // notify by changing controller state
    _transactionEnded = true;
 
} catch (Exception e) {
 
    if(!_transactionEnded) {
 
        // transaction didn't end
        try {
 
            // rollback and cleanup!
            _connection.rollback(_savepoint);
            _connection.releaseSavepoint(_savepoint); 
        } 
        catch (Exception err) {
 
            // just print the error
            System.err.println(err.toString());
 
            // take appropriate actions
            [...]
        }
    }
 
    // print on console
    System.err.println(e.toString());
 
    // take appropriate actions
    [...]
}

Now this example should be self-explanatory, if not, leave a comment. Also I only showed how to do an “update” query. Everything between the BEGIN and the COMMIT comments is a transaction. Now that you know that, it’s very easy to write some code for “select” queries etc.

Some of you may ask why I don’t use Persistence. The answer to this question is: “Next blog post”. This was just a little “fun” project and I couldn’t get Persistence working properly yet. This is just an article for the ordinary keep-it-simple people like me. I do see the advantages of Persitence, Seam etc. for RAD, but this little home project was just a freshen-up to get more into Java programming. Java programming with Persistence, Hibernate, Seam, Facelets, JSF, etc. is what I will be doing at least for the next 6 months, so now I will spend some time getting to know them better.