Migrating unmanaged Python library extension from ?python-..

Ben Finney · External Since: May 12, 2005 Posts: 51

Howdy all,

As per bug#523965 <URL:http://bugs.debian.org/523965>, I'm attempting to
migrate the 'docutils-manpage-writer' package from using
'python-central' to using 'python-support'.

The original source currently has no installation method or build
system; I am working with the upstream author to rectify that. In the
meantime, I'm trying to fix the breakage by migrating to
'python-support'.

Currently (because there is no upstream build system specifically for
this code) the package needs the following rules:

=====
PROGRAM_DIR = usr/bin
PROGRAM_NAME = rst2man
WRITERS_DIR = writers
MANPAGE_WRITER_DIR = usr/share/pyshared/docutils/${WRITERS_DIR}

[…]

..PHONY: install
install: build
install -d ${MANPAGE_WRITER_DIR}
install -m 644 writers/manpage.py ${MANPAGE_WRITER_DIR}
install -d ${PROGRAM_DIR}
install -m 755 ${PROGRAM_NAME} ${PROGRAM_DIR}
dh --with python_central install
=====

I'm not very familiar with using 'python-support', so I'm not clear what
I need to change in these rules. Where do I need to install the library
files for them to be properly discovered by 'dh_pysupport'?

--
\ "Don't fight forces, use them." —Richard Buckminster Fuller, |
`\ _Shelter_, 1932 |
_o__) |
Ben Finney

--
To UNSUBSCRIBE, email to debian-python-REQUEST.DeleteThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.DeleteThis@lists.debian.org

Adeodato Simó · External Since: Dec 09, 2005 Posts: 127

+ Ben Finney (Wed, 15 Apr 2009 20:19:55 +1000):

> * Copy the file from 'writers/manpage.py' into 'usr/lib/$(shell
> pyversions -d)/site-packages/docutils/writers/.'. If I omit this step,
> there is no indication that the modules should be in the
> 'docutils/writers/' system library directory.

You're not qualifying the destination directory. Do you mean
debian/tmp/usr/lib/...? In that case, just use debian/<pkgname>/usr/lib...
instead, and skip the dh_install step.

> * Tell 'dh_install' to install
> 'usr/lib/python*/site-packages/docutils/writers/'. If I omit this
> step, 'dh_pysupport' ignores the module and it doesn't end up in the
> package at all.

> * Tell 'dh_pysupport' to do its thing.

> I would love to collapse the first two steps somehow, but none of my
> experiments have led to 'dh_pysupport' actually finding and installing
> the module to the right place.

See above.

--
- Are you sure we're good?
- Always.
-- Rory and Lorelai

--
To UNSUBSCRIBE, email to debian-python-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

Piotr OÅ¼arowski · External Since: Apr 15, 2009 Posts: 12

[Ben Finney, 2009-04-15]
> The only way that I know of so far to get 'dh_pysupport' to find and
> install the module correctly is:
>
> * Copy the file from 'writers/manpage.py' into 'usr/lib/$(shell
> pyversions -d)/site-packages/docutils/writers/.'. If I omit this step,
> there is no indication that the modules should be in the
> 'docutils/writers/' system library directory.
>
> * Tell 'dh_install' to install
> 'usr/lib/python*/site-packages/docutils/writers/'. If I omit this
> step, 'dh_pysupport' ignores the module and it doesn't end up in the
> package at all.
>
> * Tell 'dh_pysupport' to do its thing.

dh_pysupport should be as small as possible, it should not do
dh_install's job, IMO. Just fill in .install file, and then call
`dh_install; dh_pysupport` in debian/rules

--
To UNSUBSCRIBE, email to debian-python-REQUEST.DeleteThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.DeleteThis@lists.debian.org

Ben Finney · External Since: May 12, 2005 Posts: 51

Piotr OÅ¼arowski <piotr.DeleteThis@debian.org> writes:

> [Ben Finney, 2009-04-15]
> > The only way that I know of so far to get 'dh_pysupport' to find and
> > install the module correctly is:
> >
> > * Copy the file from 'writers/manpage.py' into 'usr/lib/$(shell
> > pyversions -d)/site-packages/docutils/writers/.'. If I omit this step,
> > there is no indication that the modules should be in the
> > 'docutils/writers/' system library directory.
> >
> > * Tell 'dh_install' to install
> > 'usr/lib/python*/site-packages/docutils/writers/'. If I omit this
> > step, 'dh_pysupport' ignores the module and it doesn't end up in the
> > package at all.
> >
> > * Tell 'dh_pysupport' to do its thing.
>
> dh_pysupport should be as small as possible, it should not do
> dh_install's job, IMO.

I'm not sure why you think I'm expecting that.

> Just fill in .install file, and then call `dh_install; dh_pysupport`
> in debian/rules

As I said in the original message, I'd love to just do that; but it
doesn't work, as explained in the material you quoted above.

--
\ "My classmates would copulate with anything that moved, but I |
`\ never saw any reason to limit myself." —Emo Philips |
_o__) |
Ben Finney

--
To UNSUBSCRIBE, email to debian-python-REQUEST.DeleteThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.DeleteThis@lists.debian.org

Ben Finney · External Since: May 12, 2005 Posts: 51

Adeodato Simó <dato DeleteThis @net.com.org.es> writes:

> + Ben Finney (Wed, 15 Apr 2009 20:19:55 +1000):
>
> > * Copy the file from 'writers/manpage.py' into 'usr/lib/$(shell
> > pyversions -d)/site-packages/docutils/writers/.'. If I omit this step,
> > there is no indication that the modules should be in the
> > 'docutils/writers/' system library directory.
>
> You're not qualifying the destination directory. Do you mean
> debian/tmp/usr/lib/...?

I'm not sure what you mean by "qualify"; all the directory names
(including the ones you suggest?) are relative, if that's what you mean.

I'm cribbing from Cyril Brulebois's suggestion, which puts the module in
'usr/lib/$(shell pyversions -d)/site-packages/docutils/writers/' and has
that directory specified to 'dh_install'.

> In that case, just use debian/<pkgname>/usr/lib... instead, and skip
> the dh_install step.

That does work, thank you.

This goes against the suggestions earlier that 'dh_install' is the right
tool; I'm curious as to why you advise the opposite.

--
\ "To punish me for my contempt of authority, Fate has made me an |
`\ authority myself." —Albert Einstein, 1930-09-18 |
_o__) |
Ben Finney

--
To UNSUBSCRIBE, email to debian-python-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

Ben Finney · External Since: May 12, 2005 Posts: 51

Ben Finney <ben+debian@benfinney.id.au> writes:

> Adeodato Simó <dato RemoveThis @net.com.org.es> writes:
>
> > In that case, just use debian/<pkgname>/usr/lib... instead, and skip
> > the dh_install step.
>
> That does work, thank you.

I spoke too soon; 'dh_pysupport' fails to find the module.

The install process now shows the following output:

=====
# Manual installation, until upstream uses a build system
install -d debian/docutils-writer-manpage/usr/lib/python2.5/site-packages/docutils/writers
install -m 644 writers/manpage.py debian/docutils-writer-manpage/usr/lib/python2.5/site-packages/docutils/writers/
dh install
dh_testroot
dh_prep
dh_installdirs
dh_auto_install
dh_install
dh_installdocs
dh_installchangelogs
dh_installexamples
dh_installman
dh_installcatalogs
dh_installcron
dh_installdebconf
dh_installcatalogs
dh_installemacsen
dh_installifupdown
dh_installinfo
dh_installinit
dh_installmenu
dh_installmime
dh_installmodules
dh_installlogcheck
dh_installlogrotate
dh_installpam
dh_installppp
dh_installudev
dh_installwm
dh_installxfonts
dh_bugfiles
dh_lintian
dh_desktop
dh_gconf
dh_icons
dh_perl
dh_pysupport
dh_scrollkeeper
dh_usrlocal
dh_link
dh_compress
dh_fixperms
=====

This doesn't seem to be sufficient for 'dh_pysupport', which gives no
output. The resulting binary package does not have the module at all.

--
\ "Corporation, n. An ingenious device for obtaining individual |
`\ profit without individual responsibility." —Ambrose Bierce, |
_o__) _The Devil's Dictionary_, 1906 |
Ben Finney

--
To UNSUBSCRIBE, email to debian-python-REQUEST RemoveThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster RemoveThis @lists.debian.org

Piotr OÅ¼arowski · External Since: Apr 15, 2009 Posts: 12

[Ben Finney, 2009-04-15]
> > Just fill in .install file, and then call `dh_install; dh_pysupport`
> > in debian/rules
>
> As I said in the original message, I'd love to just do that; but it
> doesn't work, as explained in the material you quoted above.

otherwise I don't know why you can't just use:
(note that in -3 you're not using .install file to install files,
docutils-writer-manpage.install doesn't do what you expect it to do,
that's why you have to call `install` in debian/rules first)

diff -u docutils-writer-manpage-0.1~svn.r5663/debian/changelog docutils-writer-manpage-0.1~svn.r5663/debian/changelog
--- docutils-writer-manpage-0.1~svn.r5663/debian/changelog
+++ docutils-writer-manpage-0.1~svn.r5663/debian/changelog
@@ -1,3 +1,9 @@
+docutils-writer-manpage (0.1~svn.r5663-4) UNRELEASED; urgency=low
+
+ * Switch from python-central to python-support
+
+ -- Piotr OÅ¼arowski <piotr DeleteThis @debian.org> Wed, 15 Apr 2009 13:19:37 +0200
+
docutils-writer-manpage (0.1~svn.r5663-3) unstable; urgency=low

* debian/rules:
diff -u docutils-writer-manpage-0.1~svn.r5663/debian/rules docutils-writer-manpage-0.1~svn.r5663/debian/rules
--- docutils-writer-manpage-0.1~svn.r5663/debian/rules
+++ docutils-writer-manpage-0.1~svn.r5663/debian/rules
@@ -55,11 +55,11 @@
install -m 644 writers/manpage.py ${MANPAGE_WRITER_DIR}
install -d ${PROGRAM_DIR}
install -m 755 rst2man ${PROGRAM_DIR}
- dh --with python_central install
+ dh install

.PHONY: binary-indep
binary-indep: build install
- dh --with python_central binary-indep
+ dh binary-indep

.PHONY: binary-arch
binary-arch: build install
diff -u docutils-writer-manpage-0.1~svn.r5663/debian/control docutils-writer-manpage-0.1~svn.r5663/debian/control
--- docutils-writer-manpage-0.1~svn.r5663/debian/control
+++ docutils-writer-manpage-0.1~svn.r5663/debian/control
@@ -5,7 +5,7 @@
Homepage: http://docutils.sourceforge.net/sandbox/manpage-writer/
VCS-bzr: http://bzr.debian.org/collab-maint/docutils-writer-manpage/
Build-Depends: debhelper (>= 7.0.14),
- python-central (>= 0.6. Cool

,
+ python-support (>= 0.7.2),
python-docutils
Standards-Version: 3.8.0
XS-Python-Version: all
only in patch2:
unchanged:
--- docutils-writer-manpage-0.1~svn.r5663.orig/debian/preinst
+++ docutils-writer-manpage-0.1~svn.r5663/debian/preinst
@@ -0,0 +1,11 @@
+#!/bin/sh
+# TODO: remove this file after releasing Squeeze
+set -e
+if [ "$1" = upgrade ]
+then
+ if dpkg --compare-versions "$2" lt 0.1~svn.r5663-4; then
+ pycentral pkgremove docutils-writer-manpage
+ fi
+fi
+
+#DEBHELPER#

--
To UNSUBSCRIBE, email to debian-python-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

Adeodato Simó · External Since: Dec 09, 2005 Posts: 127

+ Ben Finney (Wed, 15 Apr 2009 21:21:01 +1000):

> > > * Copy the file from 'writers/manpage.py' into 'usr/lib/$(shell
> > > pyversions -d)/site-packages/docutils/writers/.'. If I omit this step,
> > > there is no indication that the modules should be in the
> > > 'docutils/writers/' system library directory.

> > You're not qualifying the destination directory. Do you mean
> > debian/tmp/usr/lib/...?

> I'm not sure what you mean by "qualify"; all the directory names
> (including the ones you suggest?) are relative, if that's what you mean.

No, with "qualify" I don't mean "make absolute", of course. But when
talking about a path when packaging, it's very relevant if it's
debian/tmp/usr/lib/foo, or debian/foo/usr/lib/foo. Writing "usr/lib/foo"
alone misses very important information.

Except that, in this case, it was really "usr/lib/foo", which is just...
well, unusual.

> I'm cribbing from Cyril Brulebois's suggestion, which puts the module in
> 'usr/lib/$(shell pyversions -d)/site-packages/docutils/writers/' and has
> that directory specified to 'dh_install'.

> > In that case, just use debian/<pkgname>/usr/lib... instead, and skip
> > the dh_install step.

> That does work, thank you.

> This goes against the suggestions earlier that 'dh_install' is the right
> tool; I'm curious as to why you advise the opposite.

dh_install would have been the tool of choice for this task, except that
your destination directory is not a static string: it depends on the
output of some command. Because of this, it can't be sensibly used for
this. (If pysupport looked in some directory which does not embed the
version number, that'd be another story; but I don't know about that.)

Cheers,

--
- Are you sure we're good?
- Always.
-- Rory and Lorelai

--
To UNSUBSCRIBE, email to debian-python-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

Ben Finney · External Since: May 12, 2005 Posts: 51

Adeodato Simó <dato RemoveThis @net.com.org.es> writes:

> dh_install would have been the tool of choice for this task, except
> that your destination directory is not a static string: it depends on
> the output of some command. Because of this, it can't be sensibly used
> for this. (If pysupport looked in some directory which does not embed
> the version number, that'd be another story; but I don't know about
> that.)

Josselin earlier suggested that 'dh_pysupport' will still look in
'usr/share/python-support/' for modules. Should I expect it to work with
just 'dh_install ; dh_pysupport' if I use the following install file:

=====
writers/manpage.py usr/share/python-support/docutils/writers/
=====

Unfortunately when I try this now, the module ends up in
'/usr/share/pyshared/writers/manpage.py', when it needs instead to be in
'/usr/share/pyshared/docutils/writers/manpage.py'.

--
\ "We have to go forth and crush every world view that doesn't |
`\ believe in tolerance and free speech." —David Brin |
_o__) |
Ben Finney

--
To UNSUBSCRIBE, email to debian-python-REQUEST RemoveThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster RemoveThis @lists.debian.org

Piotr OÅ¼arowski · External Since: Apr 15, 2009 Posts: 12

[Ben Finney, 2009-04-15]
> Josselin earlier suggested that 'dh_pysupport' will still look in
> 'usr/share/python-support/' for modules. Should I expect it to work with
> just 'dh_install ; dh_pysupport' if I use the following install file:
>
> =====
> writers/manpage.py usr/share/python-support/docutils/writers/
^ you're missing $package here
(see patch from my previous mail)
> =====
>
> Unfortunately when I try this now, the module ends up in
> '/usr/share/pyshared/writers/manpage.py', when it needs instead to be in
> '/usr/share/pyshared/docutils/writers/manpage.py'.

--
To UNSUBSCRIBE, email to debian-python-REQUEST.RemoveThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.RemoveThis@lists.debian.org

Ben Finney · External Since: May 12, 2005 Posts: 51

Piotr OÅ¼arowski <piotr.RemoveThis@debian.org> writes:

> [Ben Finney, 2009-04-15]
> > Josselin earlier suggested that 'dh_pysupport' will still look in
> > 'usr/share/python-support/' for modules. Should I expect it to work with
> > just 'dh_install ; dh_pysupport' if I use the following install file:
> >
> > =====
> > writers/manpage.py usr/share/python-support/docutils/writers/
> ^ you're missing
> > $package here

If I change this to

=====
writers/manpage.py usr/share/python-support/docutils-writer-manpage/docutils/writers/
=====

it does work. After 'dh_install' acts on the above specification,
'dh_pysupport' decides that the module should go in the place I want it
to go.

I would never have guessed this difference in behaviour from reading the
'dh_pysupport(1)' manpage, and I'm not at all clear on it even now.

Thanks for the continuing assistance on this.

--
\ "[W]hoever is able to make you absurd is able to make you |
`\ unjust." —Voltaire |
_o__) |
Ben Finney

--
To UNSUBSCRIBE, email to debian-python-REQUEST.RemoveThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.RemoveThis@lists.debian.org

Piotr OÅ¼arowski · External Since: Apr 15, 2009 Posts: 12

[Josselin Mouette, 2009-04-15]
> Le mercredi 15 avril 2009 à 14:19 +0200, Piotr OÅ¼arowski a écrit :
> > oh, you can't - dh_pysupport doesn't pick up files from /usr/share/pyshared
> > (Joss: could you add this path to movemodules?)
>
> It's not that simple, since that would prevent dh_pysupport from being
> idempotent. I guess I could just do nothing of there are already .public
> or .private files, though.

how about:
* moving all files to usr/share/pyshared (just like you do now)
* registering files basing on usr/share/pyshared content (ignore
old .public/.private files)

this way dh_pysupport will still be idempotent, unless I'm missing
something (all binary packages will use different pyshared dir, so there
shouldn't be any conflicts)

--
To UNSUBSCRIBE, email to debian-python-REQUEST RemoveThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster RemoveThis @lists.debian.org

Piotr OÅ¼arowski · External Since: Apr 15, 2009 Posts: 12

[Josselin Mouette, 2009-04-15]
> Le mercredi 15 avril 2009 à 15:17 +0200, Piotr OÅ¼arowski a écrit :
> > how about:
> > * moving all files to usr/share/pyshared (just like you do now)
> > * registering files basing on usr/share/pyshared content (ignore
> > old .public/.private files)
>
> I'm afraid this would lose some information on the supported versions.

can't you parse debian/{pyversions,control} again?
(note that files that differ per Python version are already in
usr/share/pymodules)

--
To UNSUBSCRIBE, email to debian-python-REQUEST.RemoveThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.RemoveThis@lists.debian.org