Java Expressions Library

Structure of this manual

This is the first version of JEL manual. It is mostly examples based. There are two examples(with static and dynamic libraries) discussed here step by step. The main purpose of this document is to supplementjavadoc generated documentation and provide several starting points to the exploration of it. There are following sections in the manual:
Features and design goals of JEL.
Highlighting JEL main features.
How to use JEL.
Walking through the example of use of static libraries.
Using libraries
Walking through the example of use of dynamic (virtual) libraries.
Error detection and reporting
Describing how JEL reacts on various errors in expressions.
Making things faster
Describing a little bit more complicated interface to JEL compiled expressions.
Limitations of JEL
There are some.
Summary
This manual is by no means compherensive, but I feel it to be enough to start using JEL. If I'm wrong, please send me suggestions on what You would like to see in the next version of this manual.

Features and design goals of JEL.

How to use JEL.

Simplicity of use was one of the design goals. Fortunately the interface to JEL can be made very simple, because it is well defined. Let's follow step by step a simple program, evaluating the expression given in it's command line arguments. Similar program exists in the distribution under the name ./samples/Calculator.java

public static void main(String[] args) {

    // Assemble the expression
    StringBuffer expr_sb=new StringBuffer();
    for(int i=0;i<args.length;i++) {
      expr_sb.append(args[i]);
      expr_sb.append(' ');
    };
    String expr=expr_sb.toString();

This first part of the program is not related to JEL. It's purpose is to assemble the expression, possibly, containing spaces into the single line. This has to be done, because shells tend to tokenize parameters... well that's their job, but we don't need it here.

    // Set up library
    Class[] staticLib=new Class[1];
    try {
      staticLib[0]=Class.forName("java.lang.Math");
    } catch(ClassNotFoundException e) {
      // Can't be ;)) ...... in java ... ;)
    };
    Library lib=new Library(staticLib,null);
    try {
    lib.markStateDependent("random",null);
    } catch (NoSuchMethodException e) {
      // Can't be also
    };

As JEL allows to call functions, from expression it certainly needs to know which ones You allow it to call. All information about functions, available for use in expressions, is contained within the Library object we just have constructed.

There are two classes of functions in the Library : static and virtual (dynamic).

Functions of the first class are assumed (by default) to be dependent only on their arguments i.e. they do not save any information from call to call(they are "stateless")... Examples of such functions are mathematical functions like sin, cos, log, for such functions it does not matter how many times (and when) they will be called they will always return the same result, provided their arguments are the same. Static functions can be evaluated by JEL at compile time if their arguments are constants (known at compile time). To define set of static functions it is needed to pass the array of Class objects, defining those functions, as the first parameter of the library constructor (see example above). Note ONLY STATIC functions of the Classes, passed in the first argument of the gnu.jel.Library constructor will be defined. Moreover, by default all static functions are marked as stateless.

Some static functions still save their state (in static variables) in between calls. Thus having different results, depending on when they are is called (even if arguments are the same). If such function is evaluated at compile time, we have troubles , because it will be evaluated only once during expression lifetime and it's state dependence will be lost. Typical example of the static function, having a state is java.lang.Math.random(). JEL has special mechanism, provided by gnu.jel.Library class to mark static functions as state dependent. (see the above example to find out how it was done for the java.lang.Math.random())

There is the separate class of functions, understood by JEL, which are EXPLICITLY state dependent. We shall discuss these functions later in the same document, they are not used in the example we currently consider. However, virtual functions are, actually, MOST IMPORTANT to JEL. This is because expression, containing all stateless functions is a constant, it will be evaluated at compile time, and there is absolutely no sense to evaluate such expression repeatedly (this is what JEL was designed for). Still we shall continue with this simple example as the following code is mostly independent of whether we use virtual functions or not...


    // Compile
    CompiledExpression expr_c=null;
    try {
      expr_c=Evaluator.compile(expr,lib);
    } catch (CompilationException ce) {
      System.err.print("–––COMPILATION ERROR :");
      System.err.println(ce.getMessage());
      System.err.print("                       ");
      System.err.println(expr);
      int column=ce.getColumn(); // Column, where error was found
      for(int i=0;i<column+23-1;i++) System.err.print(' ');
      System.err.println('^');
    };

This chunk of code is for the expression compilation. The crucial line is the call to Evaluator.compile(...), it is the point, where expression gets transformed into Java bytecode, loaded into the Java Virtual Machine using JEL Classloader and returned to caller as an instance of the subclass of gnu.jel.CompiledExpression. Typical user of JEL is not required to know what magic is going on inside of Evaluator.compile(...). Other code in this chunk is for the error reporting and will be discussed in the specialized section, below.

    if (expr_c !=null) {
      
      // Evaluate (Can do it now any number of times FAST !!!)
      Number result=null;
      try {
	result=(Number)expr_c.evaluate(null);
      } catch (Throwable e) {
	System.err.println("Exception emerged from JEL compiled"+
			   " code (IT'S OK) :");
	System.err.print(e);
      };

Now we came to the evaluation of the function. It is done by calling the evaluate method of the JEL compiled class. This method is defined abstract in gnu.jel.CompiledExpression but it is redefined in the class, compiled by JEL. The argument of this method is discussed in the section on virtual functions below. If only static functions are present in the library it is safe to pass the null pointer as the argument to evaluate.

Result of the evaluate is always an object. JEL converts primitive numeric types into instances of the java.lang.Number subclasses. Actually, if the result of evaluation has the integral type it is returned as instance of the java.lang.Long, if it is a floating point number it is returned as java.lang.Double, if result is an arbitrary Java object it is returned as the reference to that object. The possible type of the result depends on the library You use. For the library, consisting of java.lang.Math methods only, result will always be either java.lang.Long or java.lang.Double. In other words it will always be a java.lang.Number.

The try ... catch clause around the call to evaluate() will be enforced by the Java compiler. It is required as errors can appear during evaluation. The general rule is : syntax, types incompatibility and function resolution errors will be reported at compile time ( as thrown instance of gnu.jel.CompilationException), while the errors in the values of numbers will be reported at the execution time. For example expression "1/0" will generate no error at compile time (nevertheless it is the constant expression and its evaluation is attempted), but at the time of calling execute You will get a java.lang.ArithmeticError (division by zero) as it should be.

      
      // Print result
      if (result==null) 
	System.out.println("void");
      else
	System.out.println(result.toString());
    };
};
This last piece of code will print the result. And is concluding our brief tour of the JEL usage.

Using libraries

It was already said in this document, that the dynamic libraries are the most important for JEL because functions, from those libraries will be called each time the expression is evaluated (as opposed to once, when expression is compiled, for static libraries ).

To be able to call the method from the JEL expressions it is needed to define (export) it to JEL. This is done by constructing the library object, which hashes exported functions.

From the point of view of JEL there are, in fact three classes of functions. These classes of functions are defined below together with the hints on how to export them :

Static stateless functions
These are static methods of Java classes, which do not save state in between calls. Evaluation of these functions will be attempted at compile time. Most of Java static functions are stateless. In JEL all static methods of classes in the array, supplied as a first argument of gnu.jel.Library constructor are exported and assumed stateless.
Static functions, saving the state
These are static methods of Java classes which DO save state (in class static variables). Evaluation of these functions will not be attempted at compile time, they will be called each time the expression is evaluated. In JEL this kind of functions must be explicitly marked stateDependent after the gnu.jel.Library is created.
Virtual functions
These functions save their state explicitly in the instance variables of some object (requiring the reference to that object to be supplied at execution time). Virtual functions are never evaluated at compile time.In JEL all virtual methods of classes in the array, supplied as a second argument of gnu.jel.Library constructor are exported as virtual functions.

Because class designation is omitted in JEL ( user enters "sin(x)" instead of "java.lang.Math.sin(x)") all methods, in the constructed Library are subjected to the same constraints as the methods of the single Java class (see § 8.4 in the Java Language Specification (JLS) ). For example, overloading of methods is allowed, but having two methods with the same names and parameter types is not (§ 8.4.2 in JLS). Java will not let You create such methods in single class, but when You construct a JEL library methods of several classes are merged making such conflicts possible. Be careful, when constructing libraries.

Let's finally walk through the example, which calculates function of the single variable many times, usage of dynamic libraries is necessary here. This example will consist of two classes : a user written class( which provides access to the variable) and the main class. First start with the variable provider :

public class VariableProvider {
  public double x;
  
  public double x() {return x;};
};

This class is trivial, it just defines the function, returning the value of the variable x.

In the main class (see the first JEL example for headers) the code, constructing the library will be replaced with :

    // Set up library
    Class[] staticLib=new Class[1];
    try {
      staticLib[0]=Class.forName("java.lang.Math");
    } catch(ClassNotFoundException e) {
      // Can't be ;)) ...... in java ... ;)
    };

    Class[] dynamicLib=new Class[1];
    VariableProvider variables=new VariableProvider();
    Object[] variableObjects=new Object[1];
    variableObjects[0]=variables;
    dynamicLib[0]=variables.getClass();
    
    Library lib=new Library(staticLib,dynamicLib);
    try {
    lib.markStateDependent("random",null);
    } catch (NoSuchMethodException e) {
      // Can't be also
    };

Note the new piece of code, which creates the variable provider class with the reference, named variables, array of objects variableObjects (to be passed to the evaluate method of the compiled expression) and array of classes dynamicLib for the library construction.

Now the code for compilation will be exactly the same as in the example 1, but we have additional function x defined for use in expressions. JEL has the special notation for the functions, having no arguments, namely, brackets in "x()" can be omitted to be "x". This allows to compile now ( with the above defined library) the expressions like "sin(x)", "exp(x*x)", "pow(sin(x),2)+pow(cos(x),2)"...

The code for evaluation will be replaced with :


    if (expr_c !=null) {
      
      try {
         for(int i=0;i<100;i++) {
            variables.x=i;      // <- Value of the variable
            System.out.println(expr_c.evaluate(variableObjects));
                             //^^^^^^^^^^^^^^^ evaluating 100 times
	 };
      } catch (Throwable e) {
	System.err.println("Exception emerged from JEL compiled"+
			   " code (IT'S OK) :");
	System.err.print(e);
      };
   };
Note the two major differences: 1. we have explicitly assigned the value to the variable; 2. the array of object references ( consisting of one element in this example) is passed to the evaluate method. This piece of code will evaluate expressions for x=0..99 with step 1.

Object, in the variableObjects[i] should be possible to cast to the class in the dynamicLib[i] array for any i, otherwise ClassCastException will be thrown from evaluate.

This concludes our dynamic library example. Try to modify the ./Calculator.java sample yourself to allow compilation of virtual functions as described above.

Error detection and reporting

Expressions are made by human, and this human is not assumed to be the programmer. Of course he should know what functions are, but no Java experience is needed. Making errors is the natural property of humans, consequently JEL has to be aware of that.

There are two places, where errors can appear. First are the compilation errors, which are thrown in the form of gnu.jel.CompilationException by the gnu.jel.Evaluator.compile(...). These errors signal about syntax problems in the entered expressions, wrong function names, illegal types combinations, but NOT about illegal values of arguments of functions. The second source of errors is the compiled code itself, Throwables, thrown out of gnu.jel.CompiledExpression.evaluate() are primarily due to the invalid values of function arguments.

Compilation errors are easy to process. Normally, You should surround compilation by the try ... catch (CompilationException )... block. Caught CompilationException can be interrogated, then on the subject of WHERE error has occurred (gnu.jel.CompilationException.getCol()) and WHAT was the error gnu.jel.CompilationException.getMessage(). This information should then be presented to user. It is wise to use information about error column to position the cursor automatically to the erroneous place in the expression.

Errors of the second type are appearing during the function evaluation and can not be so nicely dealt with by JEL. They depend on the actual library, supplied to the compiler. For example methods of java.lang.Math do not generate any checked exceptions at all (still, Errors are possible), but You may connect library, of functions throwing exceptions. As a general rule : exceptions thrown by functions from the library are thrown from evaluate method.

Making things faster

In the above text the result of the computation, returned by gnu.jel.CompiledExpression.evaluate() was always an object. While this is very flexible it is not very fast. Objects have to be allocated on heap, and , garbage collected. When the result of computation is the Java primitive type it can be desirable to retrieve it without creation of the object. This can be done with evaluateXX() family of calls (see gnu.jel.CompiledExpression. There is an evaluateXX() method for each Java primitive type, if You know what type expression has You can just call the corresponding method.

If You do not know the type You can query it using gnu.jel.CompiledExpression.getType(). Be warned, that the call to wrong evaluateXX() method will result in exception. Another tricky point is that JEL always selects smallest data type for constant representation. Namely, expression "1" has type byte and not int, thus in most cases You will have to query the type, and only then, call the proper evaluateXX() method.

It is anyway possible to eliminate type checks at evaluation time completely. There is a version of gnu.jel.Evaluator.compile(...) method, which allows to fix the type of the result. It directs the compiler to perform the widening conversion to the given type, before returning the result. For example : if You fix the type to be int (passing java.lang.Integer.TYPE as an argument to compile) all expressions such as "1", "2+5", "2*2" will be evaluated by evaluate_int method of the compiled expression. Nevertheless, in the above case attempt to evaluate "1+2L" will be rejected by compiler, asking to insert the explicit narrowing conversion ( such as "(int)(1+2L)").

Limitations of JEL

There is one serious limitation, which should be mentioned. Actually it is not a JEL limitation but rather a limitation of the typical Java run-time.

To load compiled expressions into the Java virtual machine menory JEL uses a custom java.lang.ClassLoader. While there is nothing wrong with that, setting up a ClassLoader is a privileged operation in Java. This means either JEL should run in a Java application ( there are no security restrictions on Java applications), or , if JEL is distributed in some custom applet the applet should be signed.

Summarizing remarks

I hope You found JEL useful. Don't hesitate to contact me if there are any problems with JEL, please, report BUGS, suggest tests, send me Your patches,... There are still many improvements to be done.

Most current information about JEL should be available at http://galaxy.fzu.cz/JEL/.

JEL is the "free software" and is distributed to You under terms of GNU General Public License. Find the precise terms of the license in the file ./COPYING in the root of this distribution.

JEL © 1998 by Konstantin Metlov (metlov@fzu.cz).