How to solve conflicting java class files
13 views (last 30 days)
Show older comments
Hi, I have the following issue and I'm currently not sure, how to solve it.
I would like to use a java library in my matlab code. The library comes as a jar file with included dependencies, so it should work. adding it with javaaddpath is no issue, but when calling some functions, I get NoMethodFound errors. From all I could figure out this is due to some parts of dependencies are present on the matlab java path, but not in a version that is compatible with the library I want to use. Since this is for a toolbox that should be distributed, modifying the static class path is not really an option, and I was wondering, whether it is possible to dynamically shadow the static path in some way for a specific function.
Any ideas would be appreciated
1 Comment
Kyle Huggins
on 5 Nov 2020
Ok. So. I know this is super old, but i'm going to put my solution here because I worked damn hard on it to make it functional. This may not 100% apply to you, but it might work for someone else.
Problem
Matlab depends on certain java packages. Foremost among them are jackson and apache log4j2. If you have a java application of your own that you want to run within matlab, you more than likely are using one or both (as in my case). This creates problems becase as of R2020a, matlab is using 2.9.8 of jackson, while the current version is 2.13.3. So if you add your jar to the dynamic path via javaaddclasspath, you will have a very unhappy experience like Thomas here and get a NoMethodFound exception, since the static path is read before the dynamic path, and java provides ZERO ability to modularize or otherwise quarantine things on the classpath. So what's a dev to do?
Solution
Shade your java jar. This is where there might be issues, because you may not have full control over the jar that is included. But, assuming that you are, you should have the ability to shade the jar. With maven, one can use the maven-shade-plugin to shade packages that overlap with ones matlab uses. Here's an example from what I ended up using:
<plugin>
<!-- adding shade to packages b/c of version collision when importing to matlab !-->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.4</version>
<configuration>
<!-- There is a bug in how log4j works with fat jars. This transformation solves the problem
but is a workaround. See https://issues.apache.org/jira/browse/LOG4J2-673. It looks like
they're targeting a fix for 3.0, which I have no clue when it'll release. !-->
<transformers>
<transformer
implementation="com.github.edwgiz.maven_shade_plugin.log4j2_cache_transformer.PluginsCacheFileTransformer">
</transformer>
</transformers>
<relocations>
<!-- only shading what I think is necessary. Matlab might have other collisions !-->
<relocation>
<pattern>org.apache.logging</pattern>
<shadedPattern>org.shaded.apache.logging</shadedPattern>
</relocation>
<relocation>
<pattern>com.fasterxml</pattern>
<shadedPattern>com.shaded.fasterxml</shadedPattern>
</relocation>
</relocations>
<finalName>${project.artifactId}-${project.version}.${buildDateTime}.r${buildRevision}-shaded
</finalName>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>com.github.edwgiz</groupId>
<artifactId>maven-shade-plugin.log4j2-cachefile-transformer</artifactId>
<version>2.13.3</version>
</dependency>
</dependencies>
</plugin>
Let's break this down:
<relocations>
<!-- only shading what I think is necessary. Matlab might have other collisions !-->
<relocation>
<pattern>org.apache.logging</pattern>
<shadedPattern>org.shaded.apache.logging</shadedPattern>
</relocation>
<relocation>
<pattern>com.fasterxml</pattern>
<shadedPattern>com.shaded.fasterxml</shadedPattern>
</relocation>
</relocations>
This is how the shading actually takes place. The shade plugin will produce 1 (one) jar that includes all dependencies. Pick your poison on what to name it, but adding shaded is a pretty common pattern. Now, the combined fat jar will have completely renamed packages according to the rules here.
<finalName>${project.artifactId}-${project.version}.${buildDateTime}.r${buildRevision}-shaded
</finalName>
This is up to you. by default the shade plugin will copy the original jar to original-<whatyounamedit>.jar. If you're like me, and you want the shaded jar to be built in addition to the other, dependent free one, this is how i did it.
<transformers>
<transformer
implementation="com.github.edwgiz.maven_shade_plugin.log4j2_cache_transformer.PluginsCacheFileTransformer">
</transformer>
</transformers>
What the hell is michael bay doing in this pom?! Basically there is a bug in log4j2. It makes it so that it does not correctly instantiate the logger when used within a fat jar (aka jar with dependencies). This is a workaround to deal with that issue. Read the comment above (and corresponding LOG4J2 ticket) for more details.
This will create an uber jar that contains all your packages with the conflicting ones shaded. Use them as you normally would within your code.
Things that didn't work
I tried really really hard to use my own classloader with my own set of URLs that would exclude the system static path. This did not work. Here's the code for posterity:
shadedJar = 'pathToYourJar.jar';
shadedFile = java.io.File(shadedJar);
url = shadedFile.toURI().toURL();
argArray = javaArray('java.net.URL',1);
argArray(1) = url;
sysClassLoader = java.lang.ClassLoader.getSystemClassLoader();
%ostensilbly this creates a class loader with only the URL of the jar I want to load
myURLClassLoader = java.net.URLClassLoader.newInstance(argArray,sysClassLoader);
yourClass = myURLClassLoader.loadClass('com.path.to.YourClass');
tmp = yourClass.getConstructors();
% i could not figure out how to find exactly the constructor i wanted. So i resorted
% to just grabbing them all then finding the right one by inspection
yourClassConstructor = tmp(2);
% I could never get this to work consistently. It wouldn't return an object
% and I have no clue why.
yourObject = yourClassConstructor.newInstance(yourArg1, yourArg2, ...)
I didn't want to prefix the static path with my versions of jackson and log4j because presumably matlab needs them for something. So that was a non-starter. If that rocks your socks, then check out here.
Other options?
Presumably things like proguard and and OSGI are also meant to solve this type of problem. Those both seemed too complicated so i didn't pursue them.
Answers (0)
See Also
Categories
Find more on Call Java from MATLAB in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!