Java policy and ABI changes

Mike Hommey · External Since: Nov 09, 2004 Posts: 966

Hi,

I have a java library package, called libmozillainterfaces-java, that is
provided by xulrunner. I'm currently working on a new upstream release
of xulrunner which changed the java interfaces: some interfaces changed
namespaces, so you have to do changes to your source code, and xpcom
initialization is not handled the same way (you have to initialize the
Mozilla instance before initializing xpcom).

Which means classes built with the older version won't build nor run
as is with the newer version.

What should be done in such case, package-wise ? Change name ? Change
jar name ? Both ? Other ?

Also note that for the moment it is not very widely used, and it has no
reverse dependencies. So it may be fine to leave it alone, but I would
also like to know what to do if it happens again when it becomes used by
eclipse, for example.

Cheers,

Mike

PS: Please Cc: me, I'm not subscribed.

--
To UNSUBSCRIBE, email to debian-java-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

Andrew Haley · External Since: May 24, 2007 Posts: 1

Mike Hommey writes:

> I have a java library package, called libmozillainterfaces-java,
> that is provided by xulrunner. I'm currently working on a new
> upstream release of xulrunner which changed the java interfaces:
> some interfaces changed namespaces, so you have to do changes to
> your source code, and xpcom initialization is not handled the same
> way (you have to initialize the Mozilla instance before
> initializing xpcom).
>
> Which means classes built with the older version won't build nor run
> as is with the newer version.
>
> What should be done in such case, package-wise ? Change name ? Change
> jar name ? Both ? Other ?

Shoot the maintainers? Well, OK, that would be a little extreme, but
urge the maintainers not to break binary compatibility.

"Despite all of its promise, software reuse in object-oriented
programming has yet to reach its full potential. A major impediment
to reuse is the inability to evolve a compiled class library without
abandoning the support for already compiled applications. . . . [A]n
object-oriented model must be carefully designed so that class-library
transformations that should not break already compiled applications,
indeed, do not break such applications.'

? Ira Forman, Michael Conner, Scott Danforth, and Larry Raper,
Release-to-Release Binary Compatibility in SOM (1995) as quoted in the
JLS.

In my opinion, Java libraries without stable interfaces shouldn't be
deployed in free OSes. If they are to be used, you're going to have
to change the jar name, but even that may not work: if you use such a
library Mozilla, some other version of the same package might be used
by some other Java application running in the same process, and unless
it's firewalled by some ClassLoader trickery it'll break. If that
happens, some trickery like Jar Jar Links may be your only hope.

Andrew.

--
To UNSUBSCRIBE, email to debian-java-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

manfred · External Since: Feb 26, 2007 Posts: 21

Quoting Andrew Haley <aph RemoveThis @littlepinkcloud.COM>:

> Mike Hommey writes:
>
> > I have a java library package, called libmozillainterfaces-java,
> > that is provided by xulrunner. I'm currently working on a new
> > upstream release of xulrunner which changed the java interfaces:
> > some interfaces changed namespaces, so you have to do changes to
> > your source code, and xpcom initialization is not handled the same
> > way (you have to initialize the Mozilla instance before
> > initializing xpcom).
> >
> > Which means classes built with the older version won't build nor run
> > as is with the newer version.
> >
> > What should be done in such case, package-wise ? Change name ? Change
> > jar name ? Both ? Other ?
>
> Shoot the maintainers? Well, OK, that would be a little extreme, but
> urge the maintainers not to break binary compatibility.

> In my opinion, Java libraries without stable interfaces shouldn't be
> deployed in free OSes. If they are to be used, you're going to have
> to change the jar name, but even that may not work: if you use such a
> library Mozilla, some other version of the same package might be used
> by some other Java application running in the same process, and unless
> it's firewalled by some ClassLoader trickery it'll break. If that
> happens, some trickery like Jar Jar Links may be your only hope.

Hm. All this is a bit extreme. Even the Linux kernel changes its API
all the time and things are working okay. Especially with the use of
the java classloaders the whole binary compatibility is a bit
overrated IMHO.

In Maven you can have all sorts of versions of the same jar in
parallel and each application can use whichever it needs.

I think one way or another it will be necessary to have different
versions of various libraries in Debian in parallel. I do not think it
will be possible to force all upstreams to always use the latest
version (or most secure one or whatever.. how would you select anyway).

Just my 2c (and definitely not more..)

manfred

Marcus Better · External Since: Nov 19, 2004 Posts: 393

Andrew Haley wrote:
> In my opinion, Java libraries without stable interfaces shouldn't be
> deployed in free OSes.

That's a nice goal but unfortunately the world is not so perfect, because
users occasionally require new software with shiny new bells and whistles.
Besides we cannot control upstream and prevent them from breaking ABI. As a
distribution we need to find a balance between features and stability.

I think the Java policy needs to be tweaked to allow for multiple versions
of the same library. The problem is much easier than for C libraries, since
we don't have a dynamic linker, so the user is responsible for adding the
correct library to the classpath. We just need to make sure the different
versions don't conflict, which usually means that both of them cannot
install the generic symlink "/usr/share/java/foo.jar".

It seems it would suffice to have the symlink created by postinst, which
would point it at the latest installed version (similar to ldconfig).

Note that I'm not suggesting we should package several versions of
libraries. That should be avoided, but when necessary there should be a way
to do it.

Marcus

--
To UNSUBSCRIBE, email to debian-java-REQUEST.DeleteThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.DeleteThis@lists.debian.org

manfred · External Since: Feb 26, 2007 Posts: 21

Quoting Marcus Better <marcus.RemoveThis@better.se>:

> Andrew Haley wrote:
>> In my opinion, Java libraries without stable interfaces shouldn't be
>> deployed in free OSes.
>
> That's a nice goal but unfortunately the world is not so perfect, because
> users occasionally require new software with shiny new bells and whistles.
> Besides we cannot control upstream and prevent them from breaking ABI. As a
> distribution we need to find a balance between features and stability.
>
> I think the Java policy needs to be tweaked to allow for multiple versions
> of the same library. The problem is much easier than for C libraries, since
> we don't have a dynamic linker, so the user is responsible for adding the
> correct library to the classpath.

I could not agree more. I assume you mean the packager needs to
reference the right version of a library.

> We just need to make sure the different
> versions don't conflict, which usually means that both of them cannot
> install the generic symlink "/usr/share/java/foo.jar".

> It seems it would suffice to have the symlink created by postinst, which
> would point it at the latest installed version (similar to ldconfig).

I actually have a question about that. What do we need the symlink
without the version for. It actually looks dangerous to me to have
that. An application could use the symlink and then another app gets
updated which changes the symlink to a newer version with a different
API. That would then actually break the first application.

Why do we actually need the symlink and not just link to the correct
version only.

In terms of separating the artifacts i think an option would be to
include the package structure and the version into directory structure
so e.g. commons-cli could be located in org/apache/commons/cli/0.9 and
in parallel have a org/apache/commons/cli/1.0 and so on.

The jars could then be in /usr/share/java/org/apache/commons/cli/0.9.
Note that this kind of reproduces the structure in Maven repositories
with the advantage that Debian packagers could control that structure
(whereas the Maven repo is a big tangled mess).

> Note that I'm not suggesting we should package several versions of
> libraries. That should be avoided, but when necessary there should be a way
> to do it.

Would it be possible for one package to actually create different
versions? Otherwise it would need to be possible to have different
versions of the same package installed at the same time without file
conflicts (which could be dealt with as suggested above).

manfred

Marcus Better · External Since: Nov 19, 2004 Posts: 393

manfred.DeleteThis@mosabuam.com wrote:
> I could not agree more. I assume you mean the packager needs to
> reference the right version of a library.

That too, but also the _user_ who runs third-party code using the library,
and needs to set the classpath.

> I actually have a question about that. What do we need the symlink
> without the version for. It actually looks dangerous to me to have
> that.

It is, but I guess it's convenient, so we just hope it doesn't break on
upgrades...

Wonder if it would be possible to use the alternatives system to provide the
symlink? Then it could be controlled by the sysadmin.

>> Note that I'm not suggesting we should package several versions of
>> libraries. That should be avoided, but when necessary there should be a
>> way to do it.
>
> Would it be possible for one package to actually create different
> versions?

No, not in a sane way. Each package comes from one source release.

> Otherwise it would need to be possible to have different
> versions of the same package installed at the same time without file
> conflicts

If multiple ABIs are needed simultaneously, then the package names must
differ. I.e. libfoo0-java, libfoo1-java etc.

I don't think there has been much pressure to support different versions
though. Usually we just package the latest version available. We should
probably take care to check API/ABI compatibility on each upgrade.

Marcus

--
To UNSUBSCRIBE, email to debian-java-REQUEST.DeleteThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.DeleteThis@lists.debian.org

Florian Weimer · External Since: Nov 10, 2004 Posts: 648

* Marcus Better:

> I think the Java policy needs to be tweaked to allow for multiple versions
> of the same library. The problem is much easier than for C libraries, since
> we don't have a dynamic linker, so the user is responsible for adding the
> correct library to the classpath.

Not quite true, there is in fact a dynamic linker (Class-Path:
attribute in the manifest). I think it should be used, otherwise you
can't add a new dependency without breaking all the wrapper scripts in
reverse dependencies.

--
To UNSUBSCRIBE, email to debian-java-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

Andrew Haley · External Since: Nov 14, 2005 Posts: 17

manfred.RemoveThis@mosabuam.com writes:
> Quoting Andrew Haley <aph.RemoveThis@littlepinkcloud.COM>:

> > In my opinion, Java libraries without stable interfaces shouldn't be
> > deployed in free OSes. If they are to be used, you're going to have
> > to change the jar name, but even that may not work: if you use such a
> > library Mozilla, some other version of the same package might be used
> > by some other Java application running in the same process, and unless
> > it's firewalled by some ClassLoader trickery it'll break. If that
> > happens, some trickery like Jar Jar Links may be your only hope.
>
> Hm. All this is a bit extreme. Even the Linux kernel changes its API
> all the time and things are working okay.

This really is grossly unfair to the kernel deveopers, who go to great
lengths to avoid breaking the ABI. Would that Java package
mantainers were so careful.

Andrew.

--
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903

--
To UNSUBSCRIBE, email to debian-java-REQUEST.RemoveThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.RemoveThis@lists.debian.org

Andrew Haley · External Since: Nov 14, 2005 Posts: 17

Marcus Better writes:
> Andrew Haley wrote:
> > In my opinion, Java libraries without stable interfaces shouldn't be
> > deployed in free OSes.
>
> That's a nice goal but unfortunately the world is not so perfect,
> because users occasionally require new software with shiny new
> bells and whistles. Besides we cannot control upstream and prevent
> them from breaking ABI. As a distribution we need to find a balance
> between features and stability.
>
> I think the Java policy needs to be tweaked to allow for multiple
> versions of the same library. The problem is much easier than for C
> libraries, since we don't have a dynamic linker, so the user is
> responsible for adding the correct library to the classpath. We
> just need to make sure the different versions don't conflict, which
> usually means that both of them cannot install the generic symlink
> "/usr/share/java/foo.jar".

As I pointed out before, that doesn't work for Java in the general
case because a single running VM can load multiple libraries, which
come from different sources, which may need different versions of the
same library. You're into dependency hell very quickly.

> It seems it would suffice to have the symlink created by postinst,
> which would point it at the latest installed version (similar to
> ldconfig).
>
> Note that I'm not suggesting we should package several versions of
> libraries. That should be avoided, but when necessary there should
> be a way to do it.

I'm not going to argue with that. Sometimes everybody has to do bad
things. Smile

Andrew.

--
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903

--
To UNSUBSCRIBE, email to debian-java-REQUEST RemoveThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster RemoveThis @lists.debian.org

Manfred Moser · External Since: Mar 04, 2007 Posts: 9

On Saturday May 26 2007, Andrew Haley wrote:
> manfred.DeleteThis@mosabuam.com writes:

> > Hm. All this is a bit extreme. Even the Linux kernel changes its
> > API all the time and things are working okay.
>
> This really is grossly unfair to the kernel deveopers, who go to
> great lengths to avoid breaking the ABI. Would that Java package
> mantainers were so careful.

Sorry if I had the wrong impression and made a bit of an extreme
statement. I certainly don't know enough about this and apologize.

I do believe that API changes are necessary here and there and there
are valid reasons to support the old and the new API at the same
time.

manfred
--
Manfred Moser
http://www.mosabuam.com
skype: mosabua
http://www.linkedin.com/in/manfredmoser

--
To UNSUBSCRIBE, email to debian-java-REQUEST.DeleteThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.DeleteThis@lists.debian.org