[gentoo-dev] baselayout-2 stablisation plans

Roy Marples · External Since: Jul 07, 2005 Posts: 130

This is just a heads up for getting baselayout-2 stable. Next week I
plan to put baselayout-2.0.0_rc1 into the tree without any keywords and
it will be removed from package.mask (keeping the current alphas masked
though). Arch teams will then be pinged on a bug to keyword
baselayout-2.

Now there are, as usual, some daemons and init scripts that probably
won't work. For those that don't, I have either fixed or there's patches
in our bugzilla. Here's a rough summary.

Regarding init scripts.
Init scripts should now be strictly bourne or POSIX shell. ie, no
bashisms. bash init scripts will work, but ONLY if /bin/sh is bash.
Shells as /bin/sh that I've tested and found to be working are
bash
dash
busybox
zsh
FreeBSD sh

Also, as there's no bashisms, that also means no bash arrays. This
includes our net config.
config_eth0=( "1.2.3.4/24" "5.6.7.8 netmask 255.255.255.0" )
becomes
config_eth0="'1.2.3.4/24' '5.6.7.8 netmask 255.255.255.0'"
Pay attention to the quoting there.
We still support bash arrays if and only if /bin/sh is really bash, so
your existing net config should work as is.

Regarding start-stop-daemon. It will now test to see if the daemon is
running after it returns as some daemons love to fork and then error on
a bad config. So we trap this. However, there are also some false
positives where there is a big gap between forking and writing the
pidfile. Examples of this are ntpd (patch in portage) and nscd (part of
glibc).

Also, when running in parallel, we now prefix all output. This looks
very pretty on screen and is almost perfect. However, this also comes at
a cost - daemons are now expected to close stdin, stdout and stderr
after forking like all well written daemons should. Some don't - like
gpm (latest rev is patched) and noip (patch in bugs). If they don't,
then it will hang rc. Sadly, there isn't much I can do about this.

Lastly, reiserfs3 and jfs users should comment on bug #116016 to ensure
that my proposed patches to the userland tools actually work as I don't
use those fs's much aside from basic testing.

Well, that's about it. It's been a fun journey making baselayout-2 and
we're almost at the end of this road Smile

Thanks

Roy

PS - Now is a very good time to test it if you're concerned about it
running on your obscure arch or use some funky networking foo.

--
gentoo-dev DeleteThis @gentoo.org mailing list

Daniel Drake · External Since: May 02, 2004 Posts: 278

Roy Marples wrote:
> This is just a heads up for getting baselayout-2 stable. Next week I
> plan to put baselayout-2.0.0_rc1 into the tree without any keywords and
> it will be removed from package.mask (keeping the current alphas masked
> though). Arch teams will then be pinged on a bug to keyword
> baselayout-2.

You should find someone from the GDP to write some user migration docs.
The two things that spring to mind are having to use init scripts for
device-mapper and crypto fs stuff (I can see lots of unbootable systems
for those who don't realise this...) and having to compile the new
splashutils *after* baselayout to keep fbsplash working.

Thanks for all your work on baselayout-2!

Daniel

--
gentoo-dev DeleteThis @gentoo.org mailing list

Roy Marples · External Since: Jul 07, 2005 Posts: 130

On Sat, 2007-07-21 at 12:15 -0400, Daniel Drake wrote:
> Roy Marples wrote:
> > This is just a heads up for getting baselayout-2 stable. Next week I
> > plan to put baselayout-2.0.0_rc1 into the tree without any keywords and
> > it will be removed from package.mask (keeping the current alphas masked
> > though). Arch teams will then be pinged on a bug to keyword
> > baselayout-2.
>
> You should find someone from the GDP to write some user migration docs.

Good idea. Any volunteers?

> The two things that spring to mind are having to use init scripts for
> device-mapper and crypto fs stuff (I can see lots of unbootable systems
> for those who don't realise this...) and having to compile the new
> splashutils *after* baselayout to keep fbsplash working.

I don't actually know how to set those up or what the migration path
would be. Maybe devzero and strerror could document this as I understand
they do this.

Thanks

Roy

--
gentoo-dev RemoveThis @gentoo.org mailing list

Daniel Drake · External Since: May 02, 2004 Posts: 278

Roy Marples wrote:
> I don't actually know how to set those up or what the migration path
> would be. Maybe devzero and strerror could document this as I understand
> they do this.

I manage systems with a single RAID 0 stripe (not dmraid) managed by
device-mapper. When upgrading baselayout, we also have to upgrade to a
recent device-mapper version which provides the device-mapper init
script. Then we must run:

# rc-update add device-mapper boot

If we don't, we get an unbootable system.

Daniel

--
gentoo-dev.DeleteThis@gentoo.org mailing list

Roy Marples · External Since: Jul 07, 2005 Posts: 130

On Sat, 2007-07-21 at 12:45 -0400, Daniel Drake wrote:
> Roy Marples wrote:
> > I don't actually know how to set those up or what the migration path
> > would be. Maybe devzero and strerror could document this as I understand
> > they do this.
>
> I manage systems with a single RAID 0 stripe (not dmraid) managed by
> device-mapper. When upgrading baselayout, we also have to upgrade to a
> recent device-mapper version which provides the device-mapper init
> script. Then we must run:
>
> # rc-update add device-mapper boot
>
> If we don't, we get an unbootable system.

Probably a good idea to add that to the ebuild output too?

Roy

--
gentoo-dev DeleteThis @gentoo.org mailing list

Denis Dupeyron · External Since: Jul 09, 2006 Posts: 28

On 7/21/07, Roy Marples wrote:
> Now there are, as usual, some daemons and init scripts that probably
> won't work. For those that don't, I have either fixed or there's patches
> in our bugzilla. Here's a rough summary.

About the nscd issue we discussed on irc on friday (i.e. daemon not
playing nice with parallel startup), your patch worked. It stayed on
my work laptop, though, so I can't file a bug right now. But I'll do
so monday morning.

Oh, and by the way, baselayout-2 is great stuff, thanks !

Denis.
--
gentoo-dev.DeleteThis@gentoo.org mailing list

Roy Marples · External Since: Jul 07, 2005 Posts: 130

On Sat, 2007-07-21 at 11:48 -0700, Josh Saddler wrote:
> (GDP): you give us the info, we'll document it for you. Or I will at least.

Well, the changes are as outlined in my first email.
The user changes are mainly a few variables in the /etc/conf.d/* files
that baselayout ships. For example a few have been removed and a few
have been added, and a few have changed.

>From our perspective, /etc/conf.d/* is quite well documented, so GDP
could easily diff the files to see what has changed.

> Of equal concern to me, however are a few issues:
>
> 1) How will stabilization work? Is it a forced upgrade from stable 1.x
> to 2.x, or can it be slotted?

It cannot be slotted in any way or form.
Also, the daemon state data is non transferable as the format has
changed in baselayout-2. This is the data that records how a daemon was
started by s-s-d so we can check if it's running or not. However, most
end users won't be concerned by this.

I've tested the ebuilds for upgrading and downgrading quite extensively,
with the following notes.
baselayout-1 *requires* bash. As such /bin/sh should point to bash
before downgrading.
If the /etc/init.d files are not immediately updated by etc-update or
other means then errors will happen. What errors and how severe entirely
depend on the scripts the user has in /etc/init.d

> 2) It will be completely unmanageable to have more than one set of
> baselayout instructions in the handbook & other docs, so there
> definitely is a need for the migration doc.

That's your call.

> 3) How long will 1.x be kept stable? (This affects how long the old
> instructions are in the handbooks etc.)

Good question. We normally keep at least one major revision prior to the
current stable in the tree. They can stay in the tree indefinitely I
suppose, but the GDP should only follow the current stable. Maybe
archive the handbook?

> 4) What baselayout will be used in the next release? (Maybe that's more
> of a releng question.)

baselayout team just makes baselayout releases. If you mean the LiveCD
then ask releng.

> 5) Do you have a rough estimate (month, 3 weeks, 5 weeks, what?) on when
> the first arches might be stabilizing 2.x?

No.
If the RC's prove stable and no serious regressions are reported for a
month then we'll probably release a final 2.0.0 and get arch teams to
mark stable a week later, or right away if no last minute changes have
been made.

> This is all from the GDP's perspective, almost none of it is of interest
> to me as a user; I expect this sh** to work just as well as
> baselayout-1.x when I hit the upgrade myself. Smile

In theory it should just work. Especially as Gentoo/FreeBSD has been
running it as our standard baselayout for around 6 months now. So
hopefully this should be a very smooth release.

> Documenting this will be a major arsepain--er, effort--since baselayout
> 1.x directions are already mixed in so well with pretty much every doc
> we have. I'm not at all looking forward to fixing the docs when the time
> comes, but I will do it provided I get to borrow your brain for a good
> long time. Smile

Most of the documentation should still apply. I've tried to be as
compatible as possible - the one possible exception being networking as
baselayout-1 used bash arrays extensively. But we still support that
if /bin/sh is bash, which it is by default for Gentoo/Linux

Thanks

Roy

--
gentoo-dev.DeleteThis@gentoo.org mailing list

Roy Marples · External Since: Jul 07, 2005 Posts: 130

On Sat, 2007-07-21 at 22:22 +0200, Tobias Klausmann wrote:
> Is there a common bug to report snags to? I've hit one:
> /etc/init.d/net.eth0 used to be a symlink to net.lo. After
> installing, it was gone (I figure it went with baselayout-1).
> Luckily, I have direct console access, otherwise the machine
> would have been gone after the reboot. Definitely something to
> yell about during merging.

We don't ship with net.eth0 because there is no guarantee an interface
called eth0 exists. net.lo (net.lo0 in *BSDs) always exists so we just
ship that.

Thanks

Roy

--
gentoo-dev.RemoveThis@gentoo.org mailing list

Tobias Klausmann · External Since: May 13, 2006 Posts: 7

Hi!

[... baselayout-2 is on the horizon ...]

Is there a common bug to report snags to? I've hit one:
/etc/init.d/net.eth0 used to be a symlink to net.lo. After
installing, it was gone (I figure it went with baselayout-1).
Luckily, I have direct console access, otherwise the machine
would have been gone after the reboot. Definitely something to
yell about during merging.

Regards,
Tobias

PS: If said bug exists, I'll gladly re-report there, but my
cursory search didn't turn up anything.
--
gentoo-dev DeleteThis @gentoo.org mailing list

Roy Marples · External Since: Jul 07, 2005 Posts: 130

On Sat, 2007-07-21 at 21:28 +0100, Roy Marples wrote:
> On Sat, 2007-07-21 at 22:22 +0200, Tobias Klausmann wrote:
> > Is there a common bug to report snags to? I've hit one:
> > /etc/init.d/net.eth0 used to be a symlink to net.lo. After
> > installing, it was gone (I figure it went with baselayout-1).
> > Luckily, I have direct console access, otherwise the machine
> > would have been gone after the reboot. Definitely something to
> > yell about during merging.
>
> We don't ship with net.eth0 because there is no guarantee an interface
> called eth0 exists. net.lo (net.lo0 in *BSDs) always exists so we just
> ship that.

One very good idea would be to get udev to create any net links that do
not exist to net.lo

Then we truly have plug and play Smile

--
gentoo-dev DeleteThis @gentoo.org mailing list

Rémi Cardona · External Since: Dec 09, 2006 Posts: 18

Denis Dupeyron a écrit :
> About the nscd issue we discussed on irc on friday (i.e. daemon not
> playing nice with parallel startup), your patch worked. It stayed on
> my work laptop, though, so I can't file a bug right now. But I'll do
> so monday morning.

I'll add mine about the dhcdbd patch you gave me 2 weeks ago. Works great.

> Oh, and by the way, baselayout-2 is great stuff, thanks !

+1 on that as well.

Rémi
--
gentoo-dev RemoveThis @gentoo.org mailing list

Roy Marples · External Since: Jul 07, 2005 Posts: 130

On Sat, 2007-07-21 at 17:25 -0700, Josh Saddler wrote:
> Archiving the handbook isn't possible.

Why? Unless I'm missing something, it should be fairly trivial.

> What'd really be nice is if it goes stable for all arches (or at least
> all of the ones that matter, subjectively) either in time or after the
> next release.

That's kinda the plan. When the base system team was 4 people, we all
agreed that we just "make it stable" as it was mostly script based and
needed to be done at the same time.

I'd like to treat baselayout-2 the same, but as we're mostly C based
it's up to the arch teams to decide.

>
> > Most of the documentation should still apply. I've tried to be as
> > compatible as possible - the one possible exception being networking as
> > baselayout-1 used bash arrays extensively. But we still support that
> > if /bin/sh is bash, which it is by default for Gentoo/Linux
>
> Yeah. Though I still don't know the whole story, I anticipate that
> updating the networking configs will be the biggest headache.

Only for non bash /bin/sh - like FreeBSD.
But we don't have a handbook for that yet so it's no worry Smile

Thanks

Roy

--
gentoo-dev RemoveThis @gentoo.org mailing list

Andrew Gaffney · External Since: Nov 02, 2004 Posts: 67

Roy Marples wrote:
>> 4) What baselayout will be used in the next release? (Maybe that's more
>> of a releng question.)
>
> baselayout team just makes baselayout releases. If you mean the LiveCD
> then ask releng.

It'll be whatever version of baselayout is stable at the time we take the
initial release snapshot. However, if baselayout-2 will go stable a week after
we take the initial snapshot, it would probably be incorporated.

--
Andrew Gaffney http://dev.gentoo.org/~agaffney/
Gentoo Linux Developer Catalyst/Installer + x86 release coordinator
--
gentoo-dev DeleteThis @gentoo.org mailing list

Doug Goldstein · External Since: Dec 24, 2006 Posts: 37

Roy Marples wrote:
> On Sat, 2007-07-21 at 12:45 -0400, Daniel Drake wrote:
>> Roy Marples wrote:
>>> I don't actually know how to set those up or what the migration path
>>> would be. Maybe devzero and strerror could document this as I understand
>>> they do this.
>> I manage systems with a single RAID 0 stripe (not dmraid) managed by
>> device-mapper. When upgrading baselayout, we also have to upgrade to a
>> recent device-mapper version which provides the device-mapper init
>> script. Then we must run:
>>
>> # rc-update add device-mapper boot
>>
>> If we don't, we get an unbootable system.
>
> Probably a good idea to add that to the ebuild output too?
>
> Roy
>

I added that ages ago.

--
Doug Goldstein
http://dev.gentoo.org/~cardoe/
--
gentoo-dev.RemoveThis@gentoo.org mailing list

Michiel de Bruijne · External Since: Jul 23, 2007 Posts: 1

On Saturday 21 July 2007 16:36:03 Roy Marples wrote:
> This is just a heads up for getting baselayout-2 stable. Next week I
> plan to put baselayout-2.0.0_rc1 into the tree without any keywords and
> it will be removed from package.mask (keeping the current alphas masked
> though). Arch teams will then be pinged on a bug to keyword
> baselayout-2.
>
> Well, that's about it. It's been a fun journey making baselayout-2 and
> we're almost at the end of this road Smile

>
> Thanks
>
> Roy

Hereby a friendly reminder that gcc-config should be keyworded as well,
current gcc-config isnâ€™t compatible with baselayout-2. According to Mike
gcc-config-1.4.0 is compatible with baselayout-2. (doesnâ€™t work on my system,
but I havenâ€™t investigated yet).
--
gentoo-dev.RemoveThis@gentoo.org mailing list

Roy Marples · External Since: Jul 07, 2005 Posts: 130

On Mon, 2007-07-23 at 13:30 -0700, Chris Gianelloni wrote:
> We'll definitely want the same version stable across the board. I'll be
> sure to work with Roy and you to ensure we come to an agreement on what
> to use and that we're all on the same page.

Fair enough.

Should I open a bug (when the time comes) just requesting the blessing
of the arch teams or just unmask it anyway?

Note that the following arch's have been tested by people other than me
amd64
arm
ppc
ppc64
sparc (fbsd only i think so far)
x86

I did compile test it on a s390 a few months ago, but I seem to have
lost my account there now.

Thanks

Roy

--
gentoo-dev.DeleteThis@gentoo.org mailing list

"José Luis Rivero · External Since: May 03, 2007 Posts: 2

Roy Marples escribió:
> On Mon, 2007-07-23 at 13:30 -0700, Chris Gianelloni wrote:
>> We'll definitely want the same version stable across the board. I'll be
>> sure to work with Roy and you to ensure we come to an agreement on what
>> to use and that we're all on the same page.
>
> Fair enough.
>
> Should I open a bug (when the time comes) just requesting the blessing
> of the arch teams or just unmask it anyway?
>
> Note that the following arch's have been tested by people other than me
> amd64
> arm
> ppc
> ppc64
> sparc (fbsd only i think so far)
> x86
>

In alpha is under testing by Tobias Klausman (B|ackbird) but, a part
from the net.eth0 lost link issue, seems to be sane AFAIK. We will be
ready very soon.

Thanks.

--
Jose Luis Rivero
Gentoo/Doc Gentoo/Alpha
--
gentoo-dev.TakeThisOut@gentoo.org mailing list

Alon Bar-Lev · External Since: Dec 09, 2006 Posts: 9

On 7/21/07, Roy Marples wrote:
> This is just a heads up for getting baselayout-2 stable. Next week I
> plan to put baselayout-2.0.0_rc1 into the tree without any keywords and
> it will be removed from package.mask (keeping the current alphas masked
> though). Arch teams will then be pinged on a bug to keyword
> baselayout-2.

Hi!
Just an issue I thought a long while ago...
What about adding USE flags for all optional networking components...
So that they installed without manually merging them one by one?

Best Regards,
Alon Bar-Lev.
--
gentoo-dev.DeleteThis@gentoo.org mailing list

Roy Marples · External Since: Jul 07, 2005 Posts: 130

On Tue, 2007-07-24 at 21:31 +0300, Alon Bar-Lev wrote:
> Just an issue I thought a long while ago...
> What about adding USE flags for all optional networking components...
> So that they installed without manually merging them one by one?

Too many use flags - simply install the package.
In the future, once the API for out network scripts doesn't change then
we will punt the scripts to the packages.

I don't want baselayout to rival PHP in terms of USE flags - heh.

Thanks

Roy

--
gentoo-dev.TakeThisOut@gentoo.org mailing list