CHAPTER 23
Julian is built on top of a mature managed platform which is already equipped with everything a programmer might need. Instead of re-inventing the wheel, it's in a developer's best interest to leverage on the existing components. This chapter will talk about how to get access to the platform types from Julian.
Julian provides three approaches for inter-operating with the platform. They are bridging, mapping and binding. Bridging, analogous to Java's JNI, requires more works on both script and the platform end, but is generally faster and can be more flexible. Mapping, similar to C#'s P/Invoke and Java's JNA, requires only minimal works in the script and no works on platform side, but is a little slower and sometimes may not meet the expectation in some extremely complex scenario. Conformant to JSE-223 API, binding allows direct access to a platform-borne object, sparing any extra code. For most programmers, mapping and binding should suffice, and will be the focus of this chapter.
Platform resource access through any of these approaches brings security risk. Julian provides a simple sandbox mechanism to disable such access and other related features, including mapping and binding themselves.
Mapping script API against a platform class can be a very smooth experience. Let's start from the simplest.
On the platform side, define a class:
// (Java)
package info.julang.tests;
public class MyClass {
public MyClass(){
}
public byte getByte(byte b) {
return b;
}
public char getChar(char c) {
return c;
}
public static int getSInt(int i) {
return i;
}
public static String getSString(String str) {
return str;
}
}
On the script side, define a class
[Mapped(className="info.julang.tests.MyClass")]
class MyClass {
}
That's it. You merely declare a class with a single attribute to map to the full name of your platform class. The script class is otherwise empty. Upon loading, the class would automatically get all the public constructors and methods (including static and instance-scoped) from the platform class.
MyClass mc = new MyClass();
byte b1 = mc.getByte((byte)1);
char c1 = mc.getChar('c');
int i2 = MyClass2.getSInt(-1);
string s2 = MyClass2.getSString("world");
The rules for type mapping between the two systems are straightforward:
int, byte, boolean, char, float map to Julian's int, byte, bool, char, float, respectively.java.lang.String maps to Julian's string type.Platform's static constants of certain types (primitive and string) will be mapped to script class by value copy.
// (Java)
package info.julang.tests;
public class Stat {
public final static int A = 10;
public final static String B = "STR";
}
[Mapped(className="info.julang.tests.Stat")]
class Stat {
static string C;
static Stat(){
Stat.C = Stat.A + Stat.B;
}
}
string c = Stat.C; // 10STR
Julian also supports mapping to platform interfaces. The following Java code imitates Git's data model, which then gets fully mirrored into Julian.
// (Java)
package info.julang.tests;
public interface GObj {
int hash();
}
////////////////////////////////////
package info.julang.tests;
public class GBlob implements GObj {
private int hash;
public GBlob(int hash){
this.hash = hash;
}
public int hash(){
return hash;
}
}
////////////////////////////////////
package info.julang.tests;
public class GTree implements GObj {
private GObj[] objs;
public GTree(GObj[] objs){
this.objs = objs;
}
public int hash(){
int total = 0;
for(GObj o : objs){
total += o.hash();
}
return total;
}
}
[Mapped(className="info.julang.tests.GObj")]
interface IGObj {
}
[Mapped(className="info.julang.tests.GTree")]
class GTree: IGObj {
}
[Mapped(className="info.julang.tests.GBlob")]
class GBlob: IGObj {
}
IGObj o1 = new GBlob(10);
IGObj o2 = new GBlob(20);
IGObj o3 = new GBlob(30);
GTree t1 = new GTree(new IGObj[]{o1, o2});
GTree t2 = new GTree(new IGObj[]{t1, o3});
int v = t2.hash();
Keep in mind that a script interface must map to a platform interface, while a script class a platform class. They cannot be mixed.
The order in which the mapped types are loaded is quite critical to the final runtime definition. Consider an example.
// (Java)
package info.julang.tests;
public class Node {
private Node next;
private int value;
public Node(int value, Node next){
this.value = value;
this.next = next;
}
public int getValue(){
return value;
}
public Node getNext(){
return next;
}
public void setNext(Node next){
this.next = next;
}
}
/////////////////////////////////////
package info.julang.tests;
public class Cluster {
private Cluster[] clusters;
private Node[] nodes;
public Cluster(Cluster[] clusters, Node[] nodes){
this.clusters = clusters;
this.nodes = nodes;
}
public Cluster[] getClusters(){
return clusters;
}
public Node[] getNodes(){
return nodes;
}
}
Note in Java, class Cluster's constructor and methods all have references to class Node. Now define a set of script classes to mirror the inheritance.
[Mapped(className="info.julang.tests.Node")]
class Node {
}
[Mapped(className="info.julang.tests.Cluster")]
class Cluster {
}
The order in which script classes are defined is less important. The key factor is in what order they get loaded. Let's load Node first.
// Load Node
Node n1 = new Node(10, null);
// Load Cluster
Cluster c2 = new Cluster(new Cluster[0], new Node[]{ n1 });
In the code snippet above, we loaded Node first, so by the time we load Cluster, the engine will find that info.julang.tests.Node, a type used by Cluster, is already mapped. It will happily just use that type to add method definition to the mapped type. In other words, the signature of script Cluster's constructor is Cluster(Cluster[], Node[]).
This is awesome. But what if we load Cluster first?
// Load Cluster
Cluster c2;
// Load Node
Node n1 = new Node(10, null);
// Call Cluster
c2 = new Cluster(new Cluster[0], new Node[]{ n1 });
This time, when the engine loads Cluster, it sees that the platform class has a reference to info.julang.tests.Node, which is not mapped so far. The engine will NOT try to find the script type mapping to it. That would be too costly and perhaps there isn't a user type mapping to info.julang.tests.Node at all. Fortunately, such missing mapping is totally allowed in Julian, which simply resorts to script's Object type with a special flag set to it, indicating that the object is expected to come from the platform (a mapped object has this flag too). In this case, the engine would actually substitute an Object array.
Then, when we call the constructor, the Node array is passed in as the second argument. The constructor is able to drill down to each element on the array, realizing that the objects held there are indeed platform-born. It would then create a platform array and place all the objects into it, and finally hand it over to JVM.
If you find this too hard to consume, it's absolutely OK. The take-away point is that you should not worry about the loading order.
Java's Object has several methods with same or similar name to Julian's: toString() is one example. When declaring a mapped type, these methods would not be mapped. This design can help avoiding conflicts between the two runtimes, but what if you want to leverage on these platform methods? The answer is PlatformObject.
To use System.PlatformObject, simply declare that your class implements the interface. But do not really implement it, as they will be automatically fulfilled by engine upon type loading using the corresponding platform methods. Your job is then merely to implement script methods in which you may feel free to call methods provided by PlatformObject.
[Mapped(className="info.julang.tests.MyObj")]
class MyObj : PlatformObject {
String toString(){
return 'a' + this.pfToString();
}
int hashCode(){
return 1 + this.pfHashCode();
}
bool equals(Object obj){
return !this.pfEquals(obj);
}
}
In this example, we explicitly implement three Object methods for MyObj, each calling into the mapped platform object's corresponding method and building the result on top of the return value.
It often occurs that the programmers already possess some variables in the underlying platform (JVM) and they need to access to them from Julian. This can be done by binding the variable with a Julian engine instance. Here is an example using JSR-223 API.
// (Java)
public static class Car {
int _speed;
Car(int speed) { _speed = speed; }
public int getSpeed() { return speed; }
public void setSpeed(int speed) { _speed = speed; }
}
/////////////////////////////////
ScriptEngine se = new ScriptEngineManager().getEngineByName("julian");
Car car = new Car(50);
se.put("mycar", car);
Object result = se.eval("int oldSpeed = car.getSpeed(); car.setSpeed(60); return oldSpeed;");
// Verify
assert result instanceof Integer;
assert ((int)result) == 50;
assert car.getSpeed() == 60;
This example is straightforward. It creates a Java object (car) outside JSE, which is then put into the engine via Java scripting API. The script that got evaluated is comprised of three statements. First, it reads an integer from the Java object into Julian's local variable of the corresponding type; second, it calls the method the setter method with a Julian integer as the argument; at the end, it returns the value back to the external runtime. We can see how this series of actions has caused real changes to the Java object under the line of // Verify.
Having understood this simple example, it's time we dove into a few more details surrounding the practicalities of this feature. In particular, we will put a little more emphasis on its limits and restrictions.
Binding for primitives are no different from the example above in terms of API usage. As long as there is a corresponding type in Julian, the value can be bound without a problem. Binding a value of some primitive type that is not supported in Julian will result in no-op. No error will be thrown, but attempt to access to the variable from the script will incur undefined symbol exception.
// (Java)
ScriptEngine se = new ScriptEngineManager().getEngineByName("julian");
int i = 5;
se.put("ival", i);
se.put("cval", 'a');
se.put("zval", false);
Object result = se.eval("return ((string)ival) + cval + zval;");
// Verify
assert result instanceof String;
assert ((String)result) == "5afalse";
Of important note, beware that a binding for primitive values involves value copy. This has two-fold implications: (1) it really doesn't matter whether you bind a variable or a literal value. Both of them will be copied into the JSE runtime; (2) the original variable from the Java runtime won't be affected afterwards. The only way to get the updated value back to the original variable is by manual assignment.
// (Java)
ScriptEngine se = new ScriptEngineManager().getEngineByName("julian");
int i = 5;
se.put("ival", i);
se.eval("ival = 7;");
// Verify
assert se.get("ival") == 7; // This has been changed inside the script.
assert i == 5; // This hasn't.
Another relatively subtle behavior of primitive binding relates to the timing when a bound variable is updated. If you call jse.get("ival") while the script is being evaluated, you won't get the updated value in real time. This is because the update for bound primitives will only occur at the end of evaluation, no matter the result.
To look at primitive bindings from another angle, these are not really bindings with a JVM local per se. It's essentially a means of creating certain basic variables before Julian runs. More than often, this is the most convenient interop option that regular programmers want to choose when they need to convey external information into the JSE runtime.
Binding objects, including the special java.lang.String, does more or less live up to the name of binding. A JVM object will be wrapped into a Julian variable, its member methods exposed as Julian methods. When their fields are updated in Julian, the underlying JVM object are updated as well. We have seen this from the introductory example.
To be summarized here, however, are the limitations of object binding. Let's go through them.
Methods can take arguments. A bound object's method would expect arguments that's also introduced into the JSE runtime through binding. The only exception here are primitive values and java.lang.String type, where you can pass along a Julian's primitive/string typed value just fine.
// (Java)
public static abstract class Vehicle {
protected int _speed;
private String _name;
Vehicle(int speed) { _speed = speed; }
public void setName(String name) { _name = name; }
public String toString { return getVehicleType() + ":" + name; };
protected abstract String getVehicleType();
}
public static class Car {
Car(int speed) { super(speed); }
public boolean isFasterThan(Vehicle another) { return this._speed > another._speed; }
@Override protected String getVehicleType() { return "CAR"; };
}
public static class Airplane {
Airplane(int speed) { super(speed); }
@Override protected String getVehicleType() { return "AIRPLANE"; };
}
/////////////////////////////////
ScriptEngine se = new ScriptEngineManager().getEngineByName("julian");
Car car = new Car(50);
Airplane plane = new Airplane(480);
se.put("mycar", car);
se.put("myplane", plane);
Object result = se.eval("mycar.setName("McQueen"); return mycar.isFasterThan(myplane);");
// Verify
assert mycar.equals("CAR:McQueen");
assert !((boolean)result);
In the example above, we passed another bound value (myplane) as the argument to a method defined on the first bound value (mycar), whose Java signature requires a parameter of class Vehicle. JSE perfectly matches the bound value's Julian type and its JVM counterpart. The same technique applies to return value as well.
The object binding is essentially built on top of Mapping APi we covered earlier in this chapter. So one may wonder what it means for the relationship between a bound object of class A, and a Julian-native object that is also mapped from class A. The short answer is they are unrelated. You cannot assign one to another, or pass one as the argument where the type of the other is expected. We will talk more about this inconvenience later.
Array binding is easy to learn, but can be hard to understand its internal mechanism. This is because JSE implements two arrays: a "native" Julian array, and a JVM-backed bound array. They are not mutually assignable, but in most use cases the orthogonality is expected so that you can swap the two types without trouble.
As always, let's first see an example where we bind two arrays: one of primitive component type, the other class type. We will use a script file this time since there are a few more lines.
// (Java)
public static class Car {
int _speed;
Car(int speed) { _speed = speed; }
public int getSpeed() { return speed; }
public void setSpeed(int speed) { _speed = speed; }
}
/////////////////////////////////
ScriptEngine se = new ScriptEngineManager().getEngineByName("julian");
int length = 5;
int[] iarray = new intlength;
Car[] carry = new Carlength;
for (int i = 0; i < length; i++) {
iarrayi = i;
carryi = new Car(i * 10);
}
se.put("iarray", iarray);
se.put("carry", carry);
FileReader reader = new FileReader("array_test_1.jul");
se.eval(reader);
//////// array_test_1.jul ////////
// Iterability
for (int i : iarray) {
Console.println(i);
}
// Indexability
for (int i = 0; i < carray.length; i++) {
Console.println(carray[i].getSpeed());
}
// Extension API
int total = carray.filter(c => c.getSpeed() > 25).map(c => c.getSpeed()).reduce(0, (acc, val) => acc + val);
All of these are well expected. But the following may come as a surprise:
// (With the same bindings as the previous example)
Console.println(iarray is int[]); // false
No doubt, the array is not a Julian int array. But if it has its own type, how do I pass it to a method which accepts int[] for the argument? In the case of primitive one-dimension array, you may cast the variable to the desired type.
// (With the same bindings as the previous example)
int[] julIntArray = (int[])iarray;
void process(int[] arr) { ... }
process(julIntArray);
The casting is a cheap operation. It directly overlaps the JVM heap memory into JSE's memory, so no value copy will be incurred.
This technique, however, won't work for arrays of non-primitive types, even for java.lang.String. There are two options here as a rescue. First, if you have the power to control the function's signature, you can use Array, or var, as the argument type. Second, you may copy the original bound value to another array of the required type. At first, this may sound quite inconvenient. But in reality, it's a pretty rare situation. Since the bound types are non-referrable (we will shed more light onto this in the next section), you cannot declare a new function in the script to make it explicitly require a certain type to be bound. The methods which would take arrays of such type as the argument are themselves bound as well, thus meeting the type requirement perfectly.
// (Java)
public static class Car {
int _speed;
Car(int speed) { _speed = speed; }
}
public static class ParkingLot {
ParkingLot() { ... }
void admit(Car[] cars) { ... }
}
/////////////////////////////////
ScriptEngine se = new ScriptEngineManager().getEngineByName("julian");
int length = 5;
Car[] carry = new Carlength;
for (int i = 0; i < length; i++) {
carryi = new Car(i * 10);
}
se.put("carry", carry);
se.put("lot1", new ParkingLot());
se.eval("lot1.admit(carry);"); // The array type matches the required signature.
In fact the only scenario where an array copy is needed are: (1) multi-dimentional array of primitive Julian types; (2) array of Julian string type. Fortunately, you can use Array.copy() utility method to achieve this with relative ease and more desired performance. After all, the syntax and API for array operations are designed orthogonally for both Julian and JVM arrays.
// (sarray is a bound array of type java.lang.String[])
int length = sarray.length;
String[] tarray = new String[length];
Array.copy(sarray, 0, tarray, 0, length);
We have looked at multiple examples for binding so far. Our comprehension of how binding works may be sufficient for daily use already. But for the readers with a curious mind, it might be worth to understand a bit more about the internal mechanism of binding. The major benefit of this little digging is a much reduced chance of surprise when dealing with more complex bound types.
We have mentioned this before: Julian's object and array binding is built on top of Mapping API. Following is what happens when you throw a bunch of objects into the engine instance, before evaluation:
Collects the JVM types to which these objects belongs. For array types, collects the innermost component type instead.
For each type collected above, synthesizes a class declaration in Julian. Each class will be named in a mangled fashion, deriving from the platform type name. All of these classes will be defined under a special module: <implicit>.
// (Java)
package com.example;
public class Car {
... ...
}
/////////////////////////////////
ScriptEngine se = new ScriptEngineManager().getEngineByName("julian");
se.put("car", new Car()); // => Contribute to a synthesized script shown below
... ...
//////// (synthesized script) ////////
module <implicit>; // Note this is not a legal identifier.
[Mapped(className="com.example.Car")]
class Mapped_com_example_Car { }
Evaluates the synthesized Julian class definitions. This will register each type with the engine runtime.
For each bound object, creates a Julian object of the mapped type and puts it into the global scope. For array-typed objects, creates an array value of the corresponding element type instead. This completes the binding process.
So in its essence, object binding is not much different from mapping. It declares a Julian type mirroring a platform type, but instead of calling its constructor, it internally converts the JVM typed object into a Julian one. Therefore, constructors are not exposed since they are not needed.
The type declared through binding is not syntactically referrable due to its module name: <implicit> is not a legal identifier in Julian. Therefore, you cannot refer to its static constants directly. The workaround is to use reflection.
// (Java)
public static class Machine {
public static int MAX_VAL = 100;
Machine() { }
}
/////////////////////////////////
ScriptEngine se = new ScriptEngineManager().getEngineByName("julian");
se.put("mymac", new Machine());
se.eval(new FileReader("array_test_2.jul"));
//////// array_test_2.jul ////////
System.Reflection.Field f = mymac.getType().getField("MAX_VAL");
int cv = f.get(null); // 100
The last bit about the binding internals has to do with what we now may call explicit mapping that is covered in the first part of this chapter. Bear in mind that there is absolutely no correlation or other relationship whatsoever between a bound object of Java type T and another Julian-native variable whose class is mapped to T as well. First off, they have different full name, since a Julian programmer cannot create a module of name <implicit>; second, and of a more fundamental difference, the explicitly mapped variable is loaded from a special class loader managed by Julian runtime, while the class of implicitly bound object is already loaded by whatever class loader that is outside the management of Julian.
// (Java)
package com.example;
public class Car {
... ...
}
/////////////////////////////////
ScriptEngine se = new ScriptEngineManager().getEngineByName("julian");
se.put("mycar", new Car());
se.eval(new FileReader("array_test_3.jul"));
//////// array_test_3.jul ////////
module My.Module;
[Mapped(className="com.example.Car")]
class Car { }
Car urcar = new Car(); // A car of explicit mapping
Console.println(mycar.getType().getFullName()); // <implicit>.Mapped_com_example_Car
Console.println(urcar.getType().getFullName()); // My.Module.Car
mycar = urcar; // ERROR!
urcar = mycar; // ERROR!
Class bridging is widely used by the system classes but, in contrast to mapping API, it can be tricky to configure and implement for common users. As of 0.1.34, the support for defining bridged API is not open yet. This feature is scheduled to deliver in late 2018.
To bridge a Julian class with a platform class, first define the Julian class with Bridged attribute.
[Bridged(apiset="System.Console")]
class PlatformCalculator {
[Bridged(name="calculate")]
hosted static void calculate(Object msg); // Must specify hosted modifier so that the method can be defined without a body.
}
Note the two fields in the attribute: apiset and name. The former is used to register the platform class, the latter associating with a particular method on that class.
Next, define the platform class in Java.
@Bridged(apiset="System.Console")
public class Calculator {
@Bridged(name="calculate")
private static class CalculateExecutor extends StaticNativeExecutor<Calculator> {
@Override
protected JValue apply(ThreadRuntime rt, Argument[] args) throws Exception {
return calculate(rt);
}
}
private static JValue calculate(ThreadRuntime rt){
// Perform your logic
// ... ...
// Return a JValue at the end
}
}
Last, register the class through either command-line or Java scripting engine API.
Exposing a script engine to the end user could be a dangerous act. To protect the hosting machines from the user's destructive urge, Julian engine provides a simple API for configuring what can and cannot be done when it comes to platform access. This API is less powerful than Java's security mechanism, and cannot overwrite it either. Nevertheless it's still desired in certain circumstances. For example, the platform (JVM, for example) owner may want grant little permissions to the engines running on it, but doing so by messing with the security manager is a tedious work that can be prone to errors. Also, in the future JVM may not be the only underlying platform to run Julian, and wherever it ends up running may miss security management feature altogether.
Platform access is grouped by category, under each of which a few __operation__s are defined. When a certain platform API is called (such as File.create() for example), the engine would check if the required category/operation policy is enabled. In the case of File.create(), it would require System.IO/write, where "System.IO" is the name of the category, and "write" that of the operation. If the required policy has been denied, the engine would throw a System.UnderprivilegeException.
By default, all policies are allowed. A JSE user may configure the policies right before the execution:
public static void runEngine(){
JulianScriptEngine engine = ...;
engine.deny("*"); // Deny all policies
engine.allow("System.Socket", "*"); // But allow all operations under category "System.Socket"
engine.deny("System.Socket", "write", "connect"); // Except for "write" and "connect"
...
engine.runFile("script.jul"); // Go!
}
If you prefer using JSE-223 API, you can also configure to the same effect by using context key JulianScriptingEngine.ALLOW_POLICIES and JulianScriptingEngine.DENY_POLICIES:
public static void runEngine(){
ScriptContext context = ...;
context.setAttribute( // Deny all policies
JulianScriptingEngine.DENY_POLICIES,
new String[] { "*" },
ScriptContext.ENGINE_SCOPE);
context.setAttribute( // But allow all operations under category "System.Socket"
JulianScriptingEngine.ALLOW_POLICIES,
new String[] { "System.Socket/*" },
ScriptContext.ENGINE_SCOPE);
context.setAttribute( // Except for "write" and "connect"
JulianScriptingEngine.DENY_POLICIES,
new String[] { "System.Socket/write,connect" },
ScriptContext.ENGINE_SCOPE);
...
}
The series of calls to allow and deny have an accumulated effect. So with the three calls above it would end up with an engine instance that denies all policies except for "System.Socket/read" and "System.Socket/listen".
In fact, back to the main topic of this chapter, to use System.Mapped attribute, access policy System.Interop/map must be enabled on the engine. This applies to object binding as well, since that is built on top of the Mapping API.
// (Java)
ScriptEngine se = new ScriptEngineManager().getEngineByName("julian");
ScriptContext context = se.getContext();
context.setAttribute(
JulianScriptingEngine.DENY_POLICIES,
new String[] { "System.Interop/map" },
ScriptContext.ENGINE_SCOPE);
se.put("obj", new Object());
se.eval("... (any code)"); // Throw EngineInvocationError with UnderprivilegeException as the cause.