Introduction to multiarch: What maintainers must do

Steve Langasek · External Since: Dec 13, 2004 Posts: 2140

On Wed, Jul 29, 2009 at 07:38:22PM -0500, Peter Samuelson wrote:
> We have mostly settled the /usr/share/locale question, and apparently
> /usr/share/doc is a special exception anyway....

No, it is not. The ubiquity of /usr/share/doc provides the *rationale* for
multiarch handling /usr/share in a way that doesn't require splitting out
common packages, but dpkg will not handle /usr/share/doc differently than
any other files.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek.DeleteThis@ubuntu.com vorlon.DeleteThis@debian.org

--
To UNSUBSCRIBE, email to debian-devel-REQUEST.DeleteThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.DeleteThis@lists.debian.org

Charles Plessy · External Since: Mar 03, 2009 Posts: 43

Le Wed, Jul 29, 2009 at 03:57:31PM +0200, Goswin von Brederlow a écrit :
>
> I got some good feedback from my previous Introduction to multiarch
> post and some good questions. I'm singling out Manoj Srivastava here
> because he was the most recent to ask this on irc but there where
> others with the same or simmilar question.
>
> The full multiarch proposal can be read on:
> https://wiki.ubuntu.com/MultiarchSpec

Dear Goswin,

thank you very much for your vulgarisation efforts. Multi-architecture settings
will be very useful for scientific computing, where we only want the
applications that really need 64 bits to use this, as there can be a
performance cost to use 64 bits in cases where 32 would be enough (which means:
spending taxpayer money to heat the atmosphere).

I have three questions about Multi-arch:

1) What is the advantage of adding a new field over simply using something like
'Arch: multi'?

2) On the Ubuntu page I read: "It is worth noting that existing package
management tools will be unable to interpret and satisfy package relationships
of this format, even when the desired package is available. Consequently, it is
recommended to defer use of such package relationships in the archive for a
full release cycle following the package management implementation." Does it
mean that we need to postpone the implementation of multi-arch in perl packages
containing compiled code until Squeeze + 2, to keep our promise to support
upgrades from Lenny?

Have a nice day,

--
Charles Plessy
Tsurumi, Kanagawa, Japan

--
To UNSUBSCRIBE, email to debian-devel-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

Samuel Thibault · External Since: May 08, 2009 Posts: 47

Charles Plessy, le Thu 30 Jul 2009 13:13:59 +0900, a écrit :
> 1) What is the advantage of adding a new field over simply using something like
> 'Arch: multi'?

Err, I believe it makes sense to mark an i386/amd64-only library as
multiarch-capable.

Samuel

--
To UNSUBSCRIBE, email to debian-devel-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

Santiago Vila · External Since: Dec 01, 2004 Posts: 389

On Wed, 29 Jul 2009, Goswin von Brederlow wrote:

> > However, you *can* share the same .mo files on each platform, since the
> > gettext code copes with endianness issues at runtime if need be.
>
> So we would have to define a default endianness for creating such
> files. A patch to gettext to create them allways little endian (or the
> other way) seems in order.
>
> Thoughts from the maintainer?

I have very good news Smile

There is already a default for that, as
msgfmt always creates little endian .mo files by default.

In fact, lots of source packages have pre-built .mo files, which are
just copied when doing "make install".

So, as far as we don't try to change msgfmt behaviour, we will not
need to patch it at all.

As noted by Cyril, there is a small discussion about this in Bug#468209.

Thanks.

--
To UNSUBSCRIBE, email to debian-devel-REQUEST.DeleteThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.DeleteThis@lists.debian.org

Goswin von Brederlow · External Since: Feb 09, 2009 Posts: 90

Steve Langasek <vorlon.TakeThisOut@debian.org> writes:

> On Wed, Jul 29, 2009 at 07:38:22PM -0500, Peter Samuelson wrote:
>> We have mostly settled the /usr/share/locale question, and apparently
>> /usr/share/doc is a special exception anyway....
>
> No, it is not. The ubiquity of /usr/share/doc provides the *rationale* for
> multiarch handling /usr/share in a way that doesn't require splitting out
> common packages, but dpkg will not handle /usr/share/doc differently than
> any other files.

Well, the execption is for /usr/share. So locale, docs, whatever is
covered there.

MfG
Goswin

--
To UNSUBSCRIBE, email to debian-devel-REQUEST.TakeThisOut@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.TakeThisOut@lists.debian.org

Goswin von Brederlow · External Since: Feb 09, 2009 Posts: 90

"Eugene V. Lyubimkin" <jackyf.devel DeleteThis @gmail.com> writes:

> Goswin von Brederlow wrote:
>> "Eugene V. Lyubimkin" <jackyf.devel DeleteThis @gmail.com> writes:
>>>> 2) Tagging package relationships instead of packages means extending
>>>> the syntax of package relationsships, trusting the binary packages to
>>>> get the depends right
>>> You'll have to do it sooner or later. At least for already mentioned perl,
>>> python and others. Or no?
>>
>> Yes, but how many are there. Perl for example has 2627 reverse
>> depends. How many of those are plugins?
> Don't matter. If even there is literally one package, the new syntax has to be
> defined. Once you add it, it doesn't matter - one package uses it or thousand
> of ones.

Sure. But do you want to alter 10 plugin packages or 2627 packages?
Considering how hard it is to transition has gone into the
considerations too.

MfG
Goswin

--
To UNSUBSCRIBE, email to debian-devel-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

Goswin von Brederlow · External Since: Feb 09, 2009 Posts: 90

Charles Plessy <plessy.RemoveThis@debian.org> writes:

> Le Wed, Jul 29, 2009 at 03:57:31PM +0200, Goswin von Brederlow a Ã©crit :
>>
>> I got some good feedback from my previous Introduction to multiarch
>> post and some good questions. I'm singling out Manoj Srivastava here
>> because he was the most recent to ask this on irc but there where
>> others with the same or simmilar question.
>>
>> The full multiarch proposal can be read on:
>> https://wiki.ubuntu.com/MultiarchSpec
>
> Dear Goswin,
>
> thank you very much for your vulgarisation efforts. Multi-architecture settings
> will be very useful for scientific computing, where we only want the
> applications that really need 64 bits to use this, as there can be a
> performance cost to use 64 bits in cases where 32 would be enough (which means:
> spending taxpayer money to heat the atmosphere).
>
> I have three questions about Multi-arch:
>
> 1) What is the advantage of adding a new field over simply using something like
> â€˜Arch: multiâ€™?

Then how you you know what architecture the package actualy is?

> 2) On the Ubuntu page I read: â€œIt is worth noting that existing package
> management tools will be unable to interpret and satisfy package relationships
> of this format, even when the desired package is available. Consequently, it is
> recommended to defer use of such package relationships in the archive for a
> full release cycle following the package management implementation.â€ Does it
> mean that we need to postpone the implementation of multi-arch in perl packages
> containing compiled code until Squeeze + 2, to keep our promise to support
> upgrades from Lenny?

Depending wether squeeze has support for the new syntax or not, yes,
it would mean that. Unless we can show that it will be possible to
upgarde the package management to squeeze+2 prior to updating perl and
that the new syntax will only make package uninstallable but not cause
parse errors. I don't think requiring people to upgrade dpkg/apt
before upgrading the rest of the system when skipping a release would
be too much of a burden. But it wouldn't be as nice as it could be.

One large question that should be answered first is: Do we desperately
need a multiarch perl? Are there reverse dependencies on perl where
one must be i386 and the other amd64? If it just a "would be nice"
feature then putting it off some might be better than making upgrades
more complex.

> Have a nice day,

MfG
Goswin

--
To UNSUBSCRIBE, email to debian-devel-REQUEST.RemoveThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.RemoveThis@lists.debian.org

Steve Langasek · External Since: Dec 13, 2004 Posts: 2140

On Wed, Jul 29, 2009 at 11:20:20PM +0200, Goswin von Brederlow wrote:
> You raise an interesting point there with -dbg packages. Esspecially
> considering the Google SoC project that wants to automatically build
> -dbg packages for everything in debian. Those .ddeb packages. Too me
> it seems that .ddeb packages seem like a really good idea and teaching
> the package management system that .ddeb packages must match the
> architecture no matter what the .deb says seems like the solution to
> your example. The -dbg packages could be handled all in one go
> magically. So do you have another example besides -dbg packages?

In the specific case of external .ddeb packages, there are additional
options available to us. In particular, since .ddeb archives don't exist
today for Debian, and would presumably not be subject to the same sort of
archive consistency checks, then if dpkg adds support for
architecture-constrained dependencies (Depends: fwibble:i386) in the squeeze
time frame, these could be used for ddebs immediately in squeeze even though
they're not supported in the main archive.

> I myself am not yet happy with the "Multi-Arch: allowed" feature as
> solution. And I haven't heard all the rational behind it. Why it is
> better than other suggestions from the past. It is something that has
> been added to the specs recently and I think you make a point that
> maybe it needs to be thought of or explained some more. The existing
> -dbg packages certainly make a point for allowing "Depends: foo:same"
> or "Depends: foo:arch" no matter what foo has as "Multi-Arch: ...".

Which past suggestions are you referring to? Ephemeral ideas that no one
saw fit to record aren't of much help.

I think the rationale is already laid out in the spec.

I also think that -dbg packages are a wart on our archive and our packaging,
and am not overly concerned about whether these packages remain consistent
on transition to multiarch - unlike interpreters, which need to be handled
right for the sanity of our users.

> Another option would be for foo to
> Provides: foo-$(DEB_HOST_GNU_TYPE)-${binary:version}
> and for foo-dbg to depend on that. Or for plugins
> Provides: foo-$(DEB_HOST_GNU_TYPE)-$(PLUGIN_ABI).

I don't think that's acceptable.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek.DeleteThis@ubuntu.com vorlon.DeleteThis@debian.org

--
To UNSUBSCRIBE, email to debian-devel-REQUEST.DeleteThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.DeleteThis@lists.debian.org

Goswin von Brederlow · External Since: Feb 09, 2009 Posts: 90

"Eugene V. Lyubimkin" <jackyf.devel.RemoveThis@gmail.com> writes:

> Goswin von Brederlow wrote:
>> "Eugene V. Lyubimkin" <jackyf.devel.RemoveThis@gmail.com> writes:
>>
>>> Goswin von Brederlow wrote:
>>>> "Eugene V. Lyubimkin" <jackyf.devel.RemoveThis@gmail.com> writes:
>>>>>> 2) Tagging package relationships instead of packages means extending
>>>>>> the syntax of package relationsships, trusting the binary packages to
>>>>>> get the depends right
>>>>> You'll have to do it sooner or later. At least for already mentioned perl,
>>>>> python and others. Or no?
>>>> Yes, but how many are there. Perl for example has 2627 reverse
>>>> depends. How many of those are plugins?
>>> Don't matter. If even there is literally one package, the new syntax has to be
>>> defined. Once you add it, it doesn't matter - one package uses it or thousand
>>> of ones.
>>
>> Sure. But do you want to alter 10 plugin packages or 2627 packages?
>> Considering how hard it is to transition has gone into the
>> considerations too.
> The best I would imagine is to alter 'Arch: any' to 'Arch: multi' (as

You can be "Arch: amd64 i386" and still be "Multi-Arch: same" or
"Multi-Arch: foreign". The two fields are completly independent other
than the arch: all case. "Arch: multi" really won't work.

> Charles suggested) or insert 'Multi-Arch: yes' automatically by the some
> tool (dak?), as checking co-installableness can be done automatically by
> simply diffing 'dpkg -c package.deb' for produced arches (one and tricky
> way), or add them manually to the ~200 libraries you want to transition
> in the first round - not very hard. For thousand of Perl libraries

The flag needs to be in the packages DEBIAN/control or various tools
would have to duplicate the automatic magic and I really don't want
DAK to mess with uploaded debs. The only palce for this would be
during build, which means dpkg-gencontrol logically. Detecting
packages that should have "Multi-Arch: same" could probably be done
savely that way. But not "Multi-Arch: foreign" as there are two many
cases where "Multi-Arch: foreign" is not appropriate (yet) even if the
package has no libraries. So you would only cover some packages.

Add to that that library package have to adjust their libdir then
asking them to add one line to debian/control as well is really no
extra work.

> inserting 'Depends: perl:foreign' could be inserted by ${perl:Depends}
> substitute requiring binNMUs only. I am not sure for python modules or
> modules for other interpreters though.

How will ${perl:Depends} know when there are only scripts in the
package and when plugins? Figure that out and by all means make
dh_perl capable of setting the depends right automatically.

But not everything is using debhelper. The specs only lay out the
changes that need to be done in the resulting deb. Not who has to make
those changes. Things like debhelper and cdbs will hopefully automate
some things. I could imagine that you could have the following rules
file:

#!/usr/bin/make

%:
dh --with multiarch $@

or at some point --with multiarch would be default.

> I would still want that multi-arch dependencies would be specified at
> one straight place, not two.

For most things it will be the depended on package. Your suggestion
would make it always be in 2 places (co-installability in the library,
depends in the dependee). I think the proposed way and having it in
90-99% only in the depended on package is better.

MfG
Goswin

--
To UNSUBSCRIBE, email to debian-devel-REQUEST.RemoveThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.RemoveThis@lists.debian.org

Goswin von Brederlow · External Since: Feb 09, 2009 Posts: 90

Steve Langasek <vorlon.TakeThisOut@debian.org> writes:

> On Wed, Jul 29, 2009 at 11:20:20PM +0200, Goswin von Brederlow wrote:
>> You raise an interesting point there with -dbg packages. Esspecially
>> considering the Google SoC project that wants to automatically build
>> -dbg packages for everything in debian. Those .ddeb packages. Too me
>> it seems that .ddeb packages seem like a really good idea and teaching
>> the package management system that .ddeb packages must match the
>> architecture no matter what the .deb says seems like the solution to
>> your example. The -dbg packages could be handled all in one go
>> magically. So do you have another example besides -dbg packages?
>
> In the specific case of external .ddeb packages, there are additional
> options available to us. In particular, since .ddeb archives don't exist
> today for Debian, and would presumably not be subject to the same sort of
> archive consistency checks, then if dpkg adds support for
> architecture-constrained dependencies (Depends: fwibble:i386) in the squeeze
> time frame, these could be used for ddebs immediately in squeeze even though
> they're not supported in the main archive.

And with .ddebs being a release goal (if I read the latest
announcement right) I consider the -dbg packages covered.

>> I myself am not yet happy with the "Multi-Arch: allowed" feature as
>> solution. And I haven't heard all the rational behind it. Why it is
>> better than other suggestions from the past. It is something that has
>> been added to the specs recently and I think you make a point that
>> maybe it needs to be thought of or explained some more. The existing
>> -dbg packages certainly make a point for allowing "Depends: foo:same"
>> or "Depends: foo:arch" no matter what foo has as "Multi-Arch: ...".
>
> Which past suggestions are you referring to? Ephemeral ideas that no one
> saw fit to record aren't of much help.
>
> I think the rationale is already laid out in the spec.

For example why "Multi-Arch: allowed" and "Depends: python:any"
instead of "Depends: python:amd64"? The 'any' case means ~2500 perl
depending packages need to specify it. The 'amd64' case would only
need plugins to specify it.

If one looks closely then right at the end you have "Permit flag days
for interpreters, reducing the number of packages to touch", which
covers this case I think. It would be nice if such rationals would be
linked directly to the parts of the proposal they affect.

> I also think that -dbg packages are a wart on our archive and our packaging,
> and am not overly concerned about whether these packages remain consistent
> on transition to multiarch - unlike interpreters, which need to be handled
> right for the sanity of our users.
>
>> Another option would be for foo to
>> Provides: foo-$(DEB_HOST_GNU_TYPE)-${binary:version}
>> and for foo-dbg to depend on that. Or for plugins
>> Provides: foo-$(DEB_HOST_GNU_TYPE)-$(PLUGIN_ABI).
>
> I don't think that's acceptable.

Then that could also be linked as background information as to why
"allowed" is used. Might be good to document what has been ruled out
as unacceptable so people don't come with the same idea next month.

And I do mean linked as in to a seperate document. All the not choosen
paths don't need to be on the specs itself.

MfG
Goswin

--
To UNSUBSCRIBE, email to debian-devel-REQUEST.TakeThisOut@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.TakeThisOut@lists.debian.org

Steve Langasek · External Since: Dec 13, 2004 Posts: 2140

On Wed, Jul 29, 2009 at 11:30:33PM +0200, Goswin von Brederlow wrote:
> >> Moreover, this is not the only exception. Thousands of desktop and server
> >> packages that contains executable binaries (applications) compiled from
> >> C/C++/Pascal/etc. also have arch-dependent reverse dependencies - packages
> >> with debug info, '-dbg' ones. So, they are not 'Multi-Arch: foreign' too.

> > First, why do these packages need to be cross-installed? If they don't need
> > to be, then there's no reason to set the Multi-Arch field on them at all.

> I want zsh for be "Multi-Arch: foreign" so zomg:i386 and
> flowscan:amd64 can be installed. But then zsh:amd64 and zsh-dbg:i386
> would be installable, which clearly does not give functional debugging
> symbols.

All well and good that you want this, but both zomg and flowscan are
arch:any packages present on both architectures. I don't see any compelling
reason to support this case on the first pass (i.e., make this supportable
in squeeze without incompatible changes to Depends:), this looks like a
wishlist request and not something critical - as is, say, getting rid of
ia32-libs.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek DeleteThis @ubuntu.com vorlon DeleteThis @debian.org

--
To UNSUBSCRIBE, email to debian-devel-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

Goswin von Brederlow · External Since: Feb 09, 2009 Posts: 90

Steve Langasek <vorlon DeleteThis @debian.org> writes:

> On Wed, Jul 29, 2009 at 11:30:33PM +0200, Goswin von Brederlow wrote:
>> >> Moreover, this is not the only exception. Thousands of desktop and server
>> >> packages that contains executable binaries (applications) compiled from
>> >> C/C++/Pascal/etc. also have arch-dependent reverse dependencies - packages
>> >> with debug info, '-dbg' ones. So, they are not 'Multi-Arch: foreign' too.
>
>> > First, why do these packages need to be cross-installed? If they don't need
>> > to be, then there's no reason to set the Multi-Arch field on them at all.
>
>> I want zsh for be "Multi-Arch: foreign" so zomg:i386 and
>> flowscan:amd64 can be installed. But then zsh:amd64 and zsh-dbg:i386
>> would be installable, which clearly does not give functional debugging
>> symbols.
>
> All well and good that you want this, but both zomg and flowscan are
> arch:any packages present on both architectures. I don't see any compelling
> reason to support this case on the first pass (i.e., make this supportable
> in squeeze without incompatible changes to Depends:), this looks like a
> wishlist request and not something critical - as is, say, getting rid of
> ia32-libs.

This was just an example. I can probably do the same with bash and
google-earth. Do you want to force everything that depends on bash to
be 32bit now?

Anything named *-dbg must be handled special in apt/dpkg till they all
go away and become .ddebs. And they must be handled special too.

If that is not done then no package with *-dbg flavour or .ddeb can be
"Multi-Arch: foreign", which put a serious dampner on multiarch
functionality.

MfG
Goswin

--
To UNSUBSCRIBE, email to debian-devel-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org