[NTLUG:Discuss] C++ calls Java which calls C++

[steve]

Stephen Davidson wrote:

> Having done much JNI programming (mostly to control HW devices), I can 
> give you a hand, although it has been a while.

Thanks!  I really need some help here.

> The first question I would have, is the main program accessible to the 
> .so library?

That's what I want to happen - but it's not.

If I load the library from within the main C++ application using dlopen
then entrypoints in the application are accessible from within the
library - but when Java loads the library they are not.

I presume it's using some kind of loading mechanism that somehow doesn't
recognise the existance of the application that invoked the JNI/JVM
environment.

> Specifically, are the functions you are looking for in the 
> main program exported?

Well, this is Linux - everything that's not 'static' is exported -
right?

> Otherwise, I am not sure that the .so can see 
> them.  If my memory serves, when the native library is loaded, it is 
> loaded into its own address space, and must dlopen any other libraries 
> that it needs.

That's what seems to be the problem.

> If it needs to call back into the VM, it needs to use 
> the *env pointer -- which may give you the handle you need.

It's not the JVM I want to call.  It's *my* application.

Essentially, we have the main application in C++ - it's a game
engine ("Botzilla" - find it at http://botzilla.sf.net) - which
has a C++ function inside it

   int fetchRobotGameEvent () ;

...elsewhere in that program, I create the JNI/JVM stuff and load
a Java '.class' file into it.  The Java class is going to be written
by the kids in my son's Java class and it's going to look something
like this:

import java.io.*   ;
import java.util.* ;

/* My utility stuff - including class GameInterface */
import botzilla.*   ;

public class MyClass extends GameInterface
{
   public MyClass ()
   {
     System.out.println ( "==> MyClass constructed OK" ) ;
   }

   public void update ()
   {
     /* getGameEvent is a part of class GameInterface */
     System.out.println ( getGameEvent () ) ;
   }
}

...I have them extend my base class 'GameInterface' so that
I can hide all of the nasty uliness associated with interfacing
to the game engine inside the base class.

So I provide the base class 'GameInterface':

package botzilla ;

import java.io.* ;
import java.lang.* ;
import java.util.* ;

public class GameInterface
{
   public native int    getGameEvent () ;

   ....lots and lots of other native functions....

   public GameInterface ()
   {
     System.load ( "/usr/local/lib/libBotzillaInterface.so" ) ;
   }
}


...and then, inside libBotzillaInterface, I have some horrible
JNI-compatible function written in C++:

extern "C" JNIEXPORT jint JNICALL
    Java_botzilla_GameInterface_getGameEvent
    (JNIEnv *env, jobject _jobj)
{
   printf ( "Inside Botzilla Interface!\n" ) ;
}


...OK - so the botzilla application calls the 'MyClass'
constructor - then repeatedly calls the 'update' function.
Because MyClass 'extends' the GameInterface class, the
MyClass constructor function implicitly invokes the GameInterface
constructor - which in turn loads my C++ library.   When the
MyClass update function calls 'getGameEvent', that calls the
Java_botzilla_GameInterface_getGameEvent inside my C++ library
and everything seems to work just fine.

The problem comes when 'getGameEvent' wants to call a function
inside the main game engine - for example by trying to call

   int fetchRobotGameEvent () ;

If I change the Java_botzilla_GameInterface_getGameEvent function
to call 'fetchRobotGameEvent()', I get a runtime error:

symbol lookup error: /usr/local/lib/libBotzillaInterface.so: undefined 
symbol: _Z13fetchRobotGameEventv

...which means that the Java 'System.out.load' function was unable
to resolve the symbols that the library needed from the main C++
application.

This is odd because if I make the C++ code explicitly load
the library using

  dlopen("/usr/local/lib/libBotzillaInterface.so". RTLD_NOW);

...then the symbols all resolve OK - but I still get the runtime
error from Java's System.out.load call.

So it looks like Java is somehow creating an entirely new
namespace for things it loads.

> http://java.sun.com/j2se/1.5.0/docs/index.html
> http://java.sun.com/j2se/1.5.0/docs/guide/jni/index.html
> http://java.sun.com/j2se/1.5.0/docs/guide/jni/spec/jniTOC.html

That's where I got my information to get this far.  There doesn't
seem to be any mention of this kind of thing.

All of the examples I could find were very simple and involved EITHER
C++ calling Java OR Java calling C++.  My problem is with C++ calling
Java which in turn calls C++ which in turn calls the main application.

None of the examples I could find were anywhere near that complex.

> Q: Any reason why they can't just port the whole thing into Java?

Well, I'd have to do that port - and:

a) I'm no good at Java - the kids are still learning what a 'class'
    is - so they can't do it.
b) It's a HUGE C++ program (over 150,000 lines of code including all
    of the game libraries it uses) and porting it to Java would be an
    insane effort.
c) Java just isn't fast enough to write things like intense 3D
    computer games in which robotic dinosaurs rampage through the
    streets with explosions and stuff.  The only way to make Java
    fast enough is to pull out the heavy duty stuff and write it
    in C++...which is what I'm doing.

The idea was just to plug in some SIMPLE Java 'scripting' so the
kids could write short programs to drive the robotic dinosaurs
around the city.  This would be a very satisfying thing for them
to work on compared to the dry problems they are given in their
Java programming class at school.

All I want here is to dump some really simple 'plugins' into my
game - without changing the game hardly at all.

> It has been a compiled native language since 1.3 came out, over 5 years 
> ago.  The Swing (or maybe the SWT - stay away from AWT though, unless 
> you a sado-masochist!) libraries are probably more than up to the task.  
> And if they are not, OpenGL has been available since 1.4 (about 3 years 
> ago).  For me to port a medium size C-Module (for Serial Device Control) 
> into Java took about two weeks, including debugging (those command codes 
> and global arrays were the killers!).

Well, I was hoping that this would be a two evening effort.

> If you are still stuck after this, let me know, and I will be happy to 
> take a closer look.

Thanks!  I'd appreciate it.

It really boils down to understanding how Java loads native libraries
and whether I can control how it does that to any useful extent.