Help!

Please test dpkg with multiarch support

 
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9
Post new topic   General Reply to Topic (not reply to a specific post)    Forums Home -> Development RSS
Next:  Accepted python-visual 1:5.12-1.4 (source amd64)  
Author Message
Guillem Jover
External


Since: Nov 13, 2004
Posts: 674



PostPosted: Wed Feb 29, 2012 11:10 pm    Post subject: Re: Multiarch file overlap summary and proposal [Login to view extended thread Info.]
Archived from groups: linux>debian>devel (more info?)

On Wed, 2012-02-15 at 19:31:10 -0800, Russ Allbery wrote:
> I agree that it's asymmetric. apt-get install libfoo means libfoo:native,
> but apt-get remove libfoo means libfoo:*. And asymmetric is bad, all
> things being equal. But I think this may be one place where asymmetric is
> still the right thing to do; I would argue that it means you're
> implementing the most common operation in both cases. apt-get install
> libfoo generally means "give me a native libfoo" since non-native libfoo
> is going to be an unusual case, and apt-get remove libfoo generally means
> "I have no more interest in libfoo, make it go away." I think that people
> who want to get rid of one architecture of libfoo but keep the other are
> already going to be thinking about architectures, and it's natural to ask
> them to qualify their request.
>
> If removing the non-native architecture has cascading effects, apt is
> obviously going to warn them about that already and they'll realize what's
> going on.

This was already contemplated at least as part of one of the threads David
referenced:

<http://lists.debian.org/debian-dpkg/2011/12/msg00068.html>

> David Kalnischkies writes:
> > (Note though that e.g. APT is not able to handle installed architectures
> > as an 'attribute'. It not only has to handle them as 'different'
> > packages (and more specific different versions) to keep
> > backward-compatibility, also different dependencies on different
> > architectures would make it pretty messy in practice. But double-think
> > is a requirement for APT development anyway. Wink )
>
> Yes, definitely the internals of our package management software can't
> fully compress the packages together; at the least, the dependencies are
> going to be different between architectures and have to be stored
> separately. [...]

> But I think what we should be telling the *user*, regardless of our
> internals, is "don't think of libfoo:i386 and libfoo:amd64 as two separate
> packages that you can maintain independently; think of libfoo as being
> installed for one or more architectures."

The thing is, in practice they cannot share much at all, because even
if they might end up being at the same version, they need to go
through different versions inbetween. For dpkg, only the package name
and the reverse dependencies are shared, assuming any other field is
equal will only come down to lost metadata.

And while I have initially actually been working with the mental model
of pkgname with multiple arch instances as an internal detail, the fact
is that this abstraction just leaks everywhere, and trying to shield
the users and maintainers from that reality will only cause pain. It's
just a nice illusion coming from the fact that those packages share the
package name. But considering pkgname == pkgname:* means for example that
all query commands have to output information for multiple instances, so
packages/scripts/etc have to be adapted anyway to handle those, and
while I don't consider that a problem, just another side of the changes
needed for multiarch, it shows how the interface can only possibly be
transparent on one side of the interface, if at all.

Finally, the thing is, those packages are really independent, they just
happen to share a name and a common source ancestor, but they can contain
different files per arch instance, different metadata, even different
maintainer script behaviour per arch, etc. And only packages from the
same arch can depend on them.

> > Mhh. The current spec just forbids binNMU for M-A:same packages -
> > the 'sync' happens on the exact binary version.
> > Somewhere else in this multiarch-discussion was hinted that we could
> > sync on the version in (optional) Source tag instead to allow binNMU.
>
> I think that the best long-term way to handle binNMUs may be to move the
> build number into a different piece of package metadata from the version.
> So a binNMU of a package with version 1.4-1 would still have version 1.4-1
> but would have a build number of 2 instead of 1. I think this would be
> way cleaner in the long run, and not just for multiarch.

That means then we cannot state a relationship based on the binNMU
version. And while that might be desirable most of the times, it makes
it impossible when it might be desirable. Without considering this
deeper, it also reminds me of when Revision was a distinct field. In
any case how to handle binNMUs is something that should be carefully
considered and not be rushed out now, just because suddently they cannot
be used...

regards,
guillem


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/20120229215831.GB4852@gaara.hadrons.org
Back to top
Russ Allbery
External


Since: Nov 17, 2005
Posts: 1209



PostPosted: Wed Feb 29, 2012 11:10 pm    Post subject: Re: Multiarch file overlap summary and proposal [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Guillem Jover writes:
> On Wed, 2012-02-15 at 19:31:10 -0800, Russ Allbery wrote:

>> I think that the best long-term way to handle binNMUs may be to move
>> the build number into a different piece of package metadata from the
>> version. So a binNMU of a package with version 1.4-1 would still have
>> version 1.4-1 but would have a build number of 2 instead of 1. I think
>> this would be way cleaner in the long run, and not just for multiarch.

> That means then we cannot state a relationship based on the binNMU
> version. And while that might be desirable most of the times, it makes
> it impossible when it might be desirable.

Good point.

> Without considering this deeper, it also reminds me of when Revision was
> a distinct field. In any case how to handle binNMUs is something that
> should be carefully considered and not be rushed out now, just because
> suddently they cannot be used...

I agree with this sentiment. Personally, I'm fine with moving forward
with a multiarch approach that doesn't allow for binNMUs on a subset of
arches as the first cut, and then go back and figure out what we're doing
with binNMUs later.

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/87boohxpa8.fsf@windlord.stanford.edu
Back to top
Guillem Jover
External


Since: Nov 13, 2004
Posts: 674



PostPosted: Thu Mar 01, 2012 3:10 am    Post subject: Re: Multiarch file overlap summary and proposal [Login to view extended thread Info.]
Archived from groups: linux>debian>devel, others (more info?)

On Thu, 2012-02-16 at 10:43:53 -0800, Russ Allbery wrote:
> I was thinking more about this, and I was finally able to put a finger on
> why I don't like package splitting as a solution.
>
> We know from prior experience with splitting packages for large
> arch-independent data that one of the more common mistakes that we'll make
> is to move the wrong files: to put into the arch-independent package a
> file that's actually arch-dependent.

This was brought up by Steve in the thread, my reply:

<http://lists.debian.org/debian-devel/2012/02/msg00497.html>

regards,
guillem


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/20120301020029.GA8180@gaara.hadrons.org
Back to top
Russ Allbery
External


Since: Nov 17, 2005
Posts: 1209



PostPosted: Thu Mar 01, 2012 3:10 am    Post subject: Re: Multiarch file overlap summary and proposal [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Guillem Jover writes:
> On Thu, 2012-02-16 at 10:43:53 -0800, Russ Allbery wrote:

>> I was thinking more about this, and I was finally able to put a finger
>> on why I don't like package splitting as a solution.

>> We know from prior experience with splitting packages for large
>> arch-independent data that one of the more common mistakes that we'll
>> make is to move the wrong files: to put into the arch-independent
>> package a file that's actually arch-dependent.

> This was brought up by Steve in the thread, my reply:

> <http://lists.debian.org/debian-devel/2012/02/msg00497.html>

Thanks for the pointer, Guillem, but I'm afraid I don't think this reply
addresses my concerns. See the specific enumeration of things that we
would have to split, and the ways in which they can break. I think the
issue with C headers is particularly severe.

I don't think this mirrors an existing problem. The sorts of things we
split into arch: all packages are nowhere near as intrusive or as tightly
coupled as the things we're talking about splitting to avoid refcounting;
for example, right now, splitting out C headers into arch: all packages is
very rare. The sort of package splitting that we would do to avoid
refcounting would run a serious risk of introducing substantial new
problems that we don't currently have.

The situation with refcounting seems much less fragile than the situation
without refcounting to me. Refcounting puts the chance of error in the
right place (on people who want to use the new feature, since the
situation will not change for users who continue using packages the way
they do today), provides a clear error message rather than silent
corruption, and fails safely (on any inconsistency) rather than appearing
to succeed in situations that are not consistent. Those are all good
design principles to have.

I think the principle of not changing things for people who are not using
multiarch is particularly important, and is inconsistent with either
package splitting or with moving files into arch-qualified paths. We
should attempt to adopt new features in a way that puts most of the risk
on the people who are making use of the new features, and tries to be as
safe as possible for existing users. I agree that we should not pursue
that to an extreme that leads to an unmaintainable system, but I don't
believe refcounting has that problem.

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/874nu9vzbb.fsf@windlord.stanford.edu
Back to top
Guillem Jover
External


Since: Nov 13, 2004
Posts: 674



PostPosted: Thu Mar 01, 2012 4:10 am    Post subject: Re: Multiarch file overlap summary and proposal [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On Wed, 2012-02-15 at 16:32:38 -0800, Russ Allbery wrote:
> Guillem Jover writes:
> > If packages have to be split anyway to cope with the other cases, then
> > the number of new packages which might not be needed otherwise will be
> > even smaller than the predicted amount, at which point it makes even
> > less sense to support refcnt'ing.
>
> I don't think the package count is really the interesting metric here,
> unless the number of introduced packages is very large (see below about
> -dev packages). I'm more concerned with maintainer time and with
> dependency complexity, and with the known problems that we introduce
> whenever we take tightly-coupled files and separate them into independent
> packages.

Well, people have been using the amount of packages as a metric, I've
just been trying to counter it. It also in a way represents the amount
of work needed.

About tightly-coupled files, they can cause serious issues also with
refcounting, consider that there's always going to be a point when
unpacking one of the new instances will have a completely different
vesion than the other already unpacked instance(s). So packages could
stop working for a long time if say unpacked libfoo0:i386 1.0 has
file-format-0, but new libfoo0:amd64 4.0 has file-format-2, and the
file didn't change name (arguably this could be considered an upstream
problem, depending on the situation), this would be particularly
problematic for pseudo-essential packages.

> I just posted separately about version lockstep: I think this is a
> feature, not a bug, in our multiarch implementation. I think this is the
> direction we *should* go, because it reduces the overall complexity of the
> system. [...]

I've replied to that separately, in any case I think the best compromise
would be to add version lockstep to dpkg, but not refcounting. Because
the first is a restriction that can always be lifted if it's confirmed
to cause issues (which I think it will), and the second can always be
added later because it's something that allows things not permitted
previously.

But at this point it seems I'm alone in thinking that refcounting has
more negative implications than positive ones, and I cannot get myself
to care enough any longer to push for this. So some weeks ago I added
back both those things to my local repo.

regards,
guillem


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/20120301030201.GB8180@gaara.hadrons.org
Back to top
Russ Allbery
External


Since: Nov 17, 2005
Posts: 1209



PostPosted: Thu Mar 01, 2012 4:10 am    Post subject: Re: Multiarch file overlap summary and proposal [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Guillem Jover writes:

> About tightly-coupled files, they can cause serious issues also with
> refcounting, consider that there's always going to be a point when
> unpacking one of the new instances will have a completely different
> vesion than the other already unpacked instance(s). So packages could
> stop working for a long time if say unpacked libfoo0:i386 1.0 has
> file-format-0, but new libfoo0:amd64 4.0 has file-format-2, and the file
> didn't change name (arguably this could be considered an upstream
> problem, depending on the situation), this would be particularly
> problematic for pseudo-essential packages.

Yes, I agree. Refcounting does complicate the upgrade situation, since
you really want to upgrade all installed architectures in lockstep to
ensure that we maintain as many of the guarantees of file consistency as
we do now with single-arch upgrades.

> I've replied to that separately, in any case I think the best compromise
> would be to add version lockstep to dpkg, but not refcounting. Because
> the first is a restriction that can always be lifted if it's confirmed
> to cause issues (which I think it will), and the second can always be
> added later because it's something that allows things not permitted
> previously.

I definitely understand where you're coming from, and I would be lying if
I said that introducing refcounting doesn't make me nervous. You're
right, it's something that's going to be very difficult to back out of if
we decide it's a mistake.

I do think it's the best solution to a complex set of issues, but we're
going to have to use it in conjunction with pretty tight version lockstep
to avoid problems with file inconsistency.

> But at this point it seems I'm alone in thinking that refcounting has
> more negative implications than positive ones, and I cannot get myself
> to care enough any longer to push for this. So some weeks ago I added
> back both those things to my local repo.

Well... no one likes to win an argument under those terms. I'd much
rather have us all agree. But I do want to wholeheartedly second
Christian's thanks for all your work on dpkg in the middle of a really
difficult situation, and your willingness to make compromises like this
even when you think they're the wrong technical decision. That's really
hard to do, and I think it's also very admirable.

If this all turns out to be a horrible mistake, I for one will try to help
us back out of it as needed, to put my resources where my advocacy has
been.

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/87sjhtuidr.fsf@windlord.stanford.edu
Back to top
Russ Allbery
External


Since: Nov 17, 2005
Posts: 1209



PostPosted: Thu Mar 01, 2012 7:10 pm    Post subject: Re: Multiarch file overlap summary and proposal [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

md@Linux.IT (Marco d'Itri) writes:
> On Mar 01, Russ Allbery wrote:

>> The situation with refcounting seems much less fragile than the situation
>> without refcounting to me.

> I totally agree.

> Also, why does refcounting have to be "perfect"?
> What would break if it did not actually check that the two files
> provided by the same package for different architectures are identical?

Well, it would break most of the things that make it less fragile. Smile

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/87399sgog4.fsf@windlord.stanford.edu
Back to top
Thorsten Glaser
External


Since: Feb 02, 2006
Posts: 225



PostPosted: Fri Mar 02, 2012 6:10 pm    Post subject: Re: Summary: dpkg shared / reference counted files and version match [Login to view extended thread Info.]
Archived from groups: linux>debian>devel (more info?)

Jakub Wilk dixit:

>> * binNMUs for the same version cannot be co-installed anyway as their
>> changelogs differ.
>> ↓
>> That could be “fixed” by using the same email address and a hardcoded
>> date, or not including the binNMU entry at all, or moving that entry to a new
>> field, etc. All of which seem like ugly hacks, or a possible loss of
>> information.
>
> http://lists.debian.org/debian-devel/2012/02/msg00316.html

Hrm. But where is the line between files that are the same and files
that differ in, for example, a binNMU?

Worse: think of using M-A as porting toolkit. Say, you’re working on
arm64 and want to cross-compile using M-A (for example because the
current¹ approach is deprecated – already, dpkg-cross needs a flag
to convert libc6-dev as it’s M-A). But some package does not build
unmodified, so the target architecture has it uploaded to d-ports.org
“unreleased” with local changes.

With Guillem’s refusal of file sharing, this would work as-is, but
not otherwise because of differences between the package versions.
(You could build, say, eglibc_2.13-27+arm64.1.dsc, locally for your
host system and install it, but that’s just ouch.) Wasn’t M-A intended
to be a porting tool (among others like a biarch/triarch replacement)
just for this purpose?

> Please don't throw into the mud work of individual developers (including me)

That’s what I thought at first when I read about this, too (except
I’m not affected). But Guillem does seem to have a point.

What do our “new architecture upbringers” (e.g. armhf people) say?
(Maybe take powerpcspe, as it seems to require more changed packages.)

bye,
//mirabilos
--
“Having a smoking section in a restaurant is like having
a peeing section in a swimming pool.”
-- Edward Burr


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/Pine.BSM.4.64L.1203021620450.24179@herc.mirbsd.org
Back to top
Goswin von Brederlow
External


Since: Feb 09, 2009
Posts: 303



PostPosted: Sun Mar 04, 2012 7:10 am    Post subject: Re: Multiarch file overlap summary and proposal [Login to view extended thread Info.]
Archived from groups: linux>debian>devel, others (more info?)

md@Linux.IT (Marco d'Itri) writes:

> On Mar 01, Russ Allbery wrote:
>
>> The situation with refcounting seems much less fragile than the situation
>> without refcounting to me.
> I totally agree.
>
> Also, why does refcounting have to be "perfect"?
> What would break if it did not actually check that the two files
> provided by the same package for different architectures are identical?

Everything that can go wrong when splitting packages. You would loose
the stability advantage.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/87aa3wkhpv.fsf@frosties.localnet
Back to top
Goswin von Brederlow
External


Since: Feb 09, 2009
Posts: 303



PostPosted: Sun Mar 04, 2012 7:10 am    Post subject: Re: Multiarch file overlap summary and proposal [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Guillem Jover writes:

> On Wed, 2012-02-15 at 16:32:38 -0800, Russ Allbery wrote:
>> Guillem Jover writes:
>> > If packages have to be split anyway to cope with the other cases, then
>> > the number of new packages which might not be needed otherwise will be
>> > even smaller than the predicted amount, at which point it makes even
>> > less sense to support refcnt'ing.
>>
>> I don't think the package count is really the interesting metric here,
>> unless the number of introduced packages is very large (see below about
>> -dev packages). I'm more concerned with maintainer time and with
>> dependency complexity, and with the known problems that we introduce
>> whenever we take tightly-coupled files and separate them into independent
>> packages.
>
> Well, people have been using the amount of packages as a metric, I've
> just been trying to counter it. It also in a way represents the amount
> of work needed.
>
> About tightly-coupled files, they can cause serious issues also with
> refcounting, consider that there's always going to be a point when
> unpacking one of the new instances will have a completely different
> vesion than the other already unpacked instance(s). So packages could
> stop working for a long time if say unpacked libfoo0:i386 1.0 has
> file-format-0, but new libfoo0:amd64 4.0 has file-format-2, and the
> file didn't change name (arguably this could be considered an upstream
> problem, depending on the situation), this would be particularly
> problematic for pseudo-essential packages.

That is not an argument for or against refcounting. If at all it would
be marginally for refcounting:

The same situation would occur with libfoo0:i386 1.0, libfoo0:amd64 4.0
and libfoo0-common:all 2.0. But now even worse because you have 3
versions that can be out-of-sync.

Actualy if the file is shipped in the package then ref counting would
automatically detect the difference in contents and fail to install the
new libfoo0:amd64 4.0. And if the file is not shipped in the package
then ref counting has no effect on it. Again ref counting comes out
better.

Ref counting would catch some of those cases but not all and it never
makes it worse. What solves this problem is the same version requirement
or simply adding Breaks: libfoo0 (<< 4.0~) to libfoo0:* 4.0. The only
point you've made is that ref counting isn't a magic bullet that brings
us world peace.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/8762ekkh3g.fsf@frosties.localnet
Back to top
Display posts from previous:   
Post new topic   General Reply to Topic (not reply to a specific post)    Forums Home -> Development All times are: Eastern Time (US & Canada)
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9
Page 9 of 9

 
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum