Debian bug report logs - #636
upgrading source

Message received at debian-bugs-done:

From debian.org!imurdock Thu Jun 22 22:19:19 1995
Return-Path: <imurdock@debian.org>
Received: from pixar.com by mongo.pixar.com with smtp
	(Smail3.1.28.1 #15) id m0sP19D-0006CNC; Thu, 22 Jun 95 22:19 PDT
Received: from debian.org (debra.debian.org) by pixar.com with SMTP id AA12273
  (5.67b/IDA-1.5 for debian-bugs-done-pipe@mongo.pixar.com); Thu, 22 Jun 1995 22:17:49 -0700
Received: by debian.org
	id m0sP1Az-0001cfC
	(Debian /\oo/\ Smail3.1.29.1 #29.32); Fri, 23 Jun 95 00:21 EST
Message-Id: <m0sP1Az-0001cfC@debian.org>
Date: Fri, 23 Jun 95 00:21 EST
From: imurdock@debian.org (Ian Murdock)
To: debian-bugs-done@pixar.com
Subject: Bug#636:

This bug was fixed in a recent upload of the package.  Please
check ftp.debian.org:/pub/debian/binary for the fixed version.

Message received at debian-bugs:

From cus.cam.ac.uk!iwj10 Tue Mar 28 04:36:07 1995
Return-Path: <iwj10@cus.cam.ac.uk>
Received: from pixar.com by mongo.pixar.com with smtp
	(Smail3.1.28.1 #15) id m0rtaVC-0005qbC; Tue, 28 Mar 95 04:36 PST
Received: from bootes.cus.cam.ac.uk by pixar.com with SMTP id AA28812
  (5.65c/IDA-1.4.4 for <debian-bugs@pixar.com>); Tue, 28 Mar 1995 04:35:52 -0800
Received: by bootes.cus.cam.ac.uk 
	(Smail-3.1.29.0 #30) id m0rtaUw-000BzjC; Tue, 28 Mar 95 13:35 BST
Received: by chiark (Smail3.1.28.1 #3)
	id m0rtO23-0002gOZ; Tue, 28 Mar 95 00:17 BST
Message-Id: <m0rtO23-0002gOZ.ijackson@nyx.cs.du.edu>
Date: Tue, 28 Mar 95 00:17 BST
From: iwj10@cus.cam.ac.uk (Ian Jackson)
To: debian-bugs@pixar.com
Subject: Re: Bug#636: upgrading source
In-Reply-To: <m0rsu5H-0004iTC@[192.77.155.4]>
References: <m0rseNY-0002gOZ.ijackson@nyx.cs.du.edu>
	<m0rsu5H-0004iTC@[192.77.155.4]>

Raul Miller writes ("Re: Bug#636: upgrading source"):
> [...]
>    > Last time I checked, I couldn't call dpkg recursively (to get a
>    > listing of the el files).
> 
>    That's also true.  Making dpkg behave at all sensibly when called
>    recursively would be a nightmare.  There would have to be a way to
>    tell it to unload all of the stuff it has loaded into core, and
>    then read it back in again.
> 
> Um... this is a classic archival readers/writer problem -- the system
> *should* be designed so that there's only one writer, but ideally,
> that shouldn't interfere with having multiple readers.

Ah, I think I misunderstood you.  Yes, reading will be possible, and
will almost certainly generate a consistent picture (if not always a
completely current one, obviously).

This will be in the C version of dpkg.  (Won't everything?)

> I suppose there's a few race conditions that can occur if a reader
> hits the system just as a change is being written, [...]

These aren't a problem, because dpkg has to be careful to make sure
that every intermediate state the database goes through is consistent
(internally, if not always in agreement with the actual state of the
system).

Otherwise if something terrible happened during dpkg processing (say,
the system runs out of swap and dpkg gets a SIGKILL) the system would
be completely broken.

> There's about a megabyte of elisp source, and the default is to leave
> them there.  However, for people that care to sit around an answer
> prompts (on a small machine), I'd like to be friendly...

This brings me to another question.  Should we have a `safe' vs.
`quick' installation option ?  If so, it would probably best be done
via an environment variable.

If the user selects `quick' a package should only prompt if there is
no default configuration that can be generated automatically.  Perhaps
a `conservative quick' vs. `risky quick' would be useful, where
`conservative quick' answers `no' to questions like `do you want to
send external mail' and `how about an anon-ftp area'.

So how about DPKGCONFIGMODE=risky or DPKGCONFIGMODE=conservative.  Any
other value (eg DPKGCONFIGMODE=safe) or unset means a normal (safe)
installation.

Ian M: any opinions ?  Do we need (want) any more than those three ?

> I guess the underlying issue I'm worrying about is that we don't have
> any tools for sysadmins to make changes to package configuration in
> any sort of sensible fashion (rename a file and dpkg looses track of
> it), so many sysadmins are going to be reluctant to mess with
> dpkg-installed packages, even when it's obviously in their best interest.

Removing files is OK.  They'll be installed again during upgrades, of
course, but dpkg will not complain when it finds them missing.

So, for your package, in the interim (ie, before you can invoke dpkg
recursively to give you a list of the files) I'd embed a list of the
files you may want to remove in the package.

Ian.

Message received at debian-bugs:

From legislate.com!rdr Sun Mar 26 07:17:54 1995
Return-Path: <rdr@legislate.com>
Received: from pixar.com by mongo.pixar.com with smtp
	(Smail3.1.28.1 #15) id m0rsu4d-0004uVC; Sun, 26 Mar 95 07:17 PST
Received: from [192.77.155.4] by pixar.com with SMTP id AA06155
  (5.65c/IDA-1.4.4 for <debian-bugs@pixar.com>); Sun, 26 Mar 1995 07:17:35 -0800
Received: by [192.77.155.4]
	id m0rsu5H-0004iTC
	(Debian /\oo/\ Smail3.1.29.1 #29.27); Sun, 26 Mar 95 10:18 EST
Message-Id: <m0rsu5H-0004iTC@[192.77.155.4]>
Date: Sun, 26 Mar 95 10:18 EST
From: rdr@legislate.com (Raul Miller)
To: iwj10@cus.cam.ac.uk, debian-bugs@pixar.com
In-Reply-To: <m0rseNY-0002gOZ.ijackson@nyx.cs.du.edu> (iwj10@cus.cam.ac.uk)
Subject: Re: Bug#636: upgrading source

Me:
   > (2) I need to address this issue in a different fashion.

You (Ian J.):
   Also true.  How about using two different packages ?

Too complicated for the general case user.

   > Last time I checked, I couldn't call dpkg recursively (to get a
   > listing of the el files).

   That's also true.  Making dpkg behave at all sensibly when called
   recursively would be a nightmare.  There would have to be a way to
   tell it to unload all of the stuff it has loaded into core, and
   then read it back in again.

Um... this is a classic archival readers/writer problem -- the system
*should* be designed so that there's only one writer, but ideally,
that shouldn't interfere with having multiple readers.

I suppose there's a few race conditions that can occur if a reader
hits the system just as a change is being written, but those that
can't be eliminated (say, because of disk space constraints) can
still be minimized (with a brief note in the docs that this can
happen).  In any event, typical recursive usage doesn't have to worry
that much about race conditions -- the parent typically suspends while
waiting for the child to finish.

   This is fraught with problems: how do you tell it ?  What if something
   goes wrong during this process ?

What do you have in mind that can go wrong during a read-only
invocation of dpkg?

   >   I suppose I could automatically include in postinst an explicit
   > list of such files in the package, but it seems like there ought
   > to be an easier way.

   I think this is probably the best thing to do if you really need to
   make changes to installed files after installation.

   However, I'd be more inclined to ask whether it's really necessary.
   How big are the source elisp files ?  Is it really worth prompting
   the user about them ?  Remember that prompting should be avoided if
   possible - packages should install with sensible defaults.  Perhaps
   if the files are that large you could provide a separate package
   with them.

There's about a megabyte of elisp source, and the default is to leave
them there.  However, for people that care to sit around an answer
prompts (on a small machine), I'd like to be friendly...

I guess the underlying issue I'm worrying about is that we don't have
any tools for sysadmins to make changes to package configuration in
any sort of sensible fashion (rename a file and dpkg looses track of
it), so many sysadmins are going to be reluctant to mess with
dpkg-installed packages, even when it's obviously in their best interest.

   > Also, ideally, when changes are made to a packages contents,
   > there should be a way of registering it with dpkg.  Perhaps a
   > "look at this directory and reflect all changes back to the named
   > package" option?

   Yes, that could be done in some way, and I've been thinking about
   it, but it would be very hard to arrange to be able to invoke it
   from within a postinst script or some such.

That's fine -- if there's going to be such a provision, I'm going to
rip this code out of my postinst.

Raul Miller

Message received at debian-bugs:

From cus.cam.ac.uk!iwj10 Fri Mar 24 15:53:37 1995
Return-Path: <iwj10@cus.cam.ac.uk>
Received: from pixar.com by mongo.pixar.com with smtp
	(Smail3.1.28.1 #15) id m0rsJAe-0007M3A; Fri, 24 Mar 95 15:53 PST
Received: from bootes.cus.cam.ac.uk by pixar.com with SMTP id AA10234
  (5.65c/IDA-1.4.4 for <debian-bugs@pixar.com>); Fri, 24 Mar 1995 15:53:23 -0800
Received: by bootes.cus.cam.ac.uk 
	(Smail-3.1.29.0 #30) id m0rsJAR-000BzoA; Fri, 24 Mar 95 23:53 GMT
Received: by chiark (Smail3.1.28.1 #3)
	id m0rsI8Z-0002gOA; Fri, 24 Mar 95 22:47 GMT
Message-Id: <m0rsI8Z-0002gOA.ijackson@nyx.cs.du.edu>
Date: Fri, 24 Mar 95 22:47 GMT
From: iwj10@cus.cam.ac.uk (Ian Jackson)
To: debian-bugs@pixar.com
Subject: Re: Bug#636: upgrading source
Precedence: air-mail

Raul Miller writes ("Bug#636: upgrading source"):
> Package: source
> 
> It's difficult to upgrade the 'source' package.  The current scheme of
> things requires the admin muck around with dpkg-specific knowledge to
> upgrade.  This is because the prerm script is:
> [....]

Umm, the `source' package is inherently un-package-like, so whatever
we do will be a kludge.

There is a strong argument to say that we shouldn't support
`upgrading' it - the supported way to upgrade it would be to download
a new source tree as a tarfile and rm -r the old one manually.
Certainly this should be *a* supported way.

If we support `upgrading' it at all IMO it should replace the source
tree that was originally from the earlier source package, and all the
compiled objects, with the new one.

I think the way to do this is to have the prerm script check its
argument: if the package is being upgraded then "$1" will be "upgrade"
or "failed-upgrade" (the latter if the script is being called because
the to-be-upgraded package's prerm failed and dpkg is trying the new
package's version instead).

In this case the script should probably allow the upgrade to go ahead.
When dpkg goes to remove the package it will leave all the directories
behind, because of the .o files and so forth.

Thus, the postrm will have to check its argument in the same way as
the prerm, and rm -r the remaining files.

One problem with this approach is that it will fall over horribly if
the sysadmin does the first thing one time and the second the next.
By that time the old tree will be gone, and dpkg will complain
bitterly that it can't back up the old version of the `package' before
installing the new one.

> I'm not sure what should be done about this..  One possibility that
> comes to mind is replace the `exit 1` line with:
> 
> cp /var/lib/dpkg/info/source.list /usr/src/linux &&
> : >/var/lib/dpkg/info/source.list

This is *definitely* completely out.

Packages should UNDER NO CIRCUMSTANCES mess with the /var/lib/dpkg
area.  In particular, undocumented changes to the structure may occur
at any point, and in any case dpkg may have large amounts of it loaded
into core during package processing so that any changes wouldn't take.

(The dpkg package itself is of course an exception to this rule, as it
already knows what's going on.)

Ian.

Message received at debian-bugs:

From legislate.com!rdr Fri Mar 24 11:31:05 1995
Return-Path: <rdr@legislate.com>
Received: from pixar.com by mongo.pixar.com with smtp
	(Smail3.1.28.1 #15) id m0rsF4Z-0005O9C; Fri, 24 Mar 95 11:31 PST
Received: from [192.77.155.4] by pixar.com with SMTP id AA01213
  (5.65c/IDA-1.4.4 for <debian-bugs@pixar.com>); Fri, 24 Mar 1995 11:30:57 -0800
Received: by [192.77.155.4]
	id m0rsF4q-0004iOC
	(Debian /\oo/\ Smail3.1.29.1 #29.27); Fri, 24 Mar 95 14:31 EST
Message-Id: <m0rsF4q-0004iOC@[192.77.155.4]>
Date: Fri, 24 Mar 95 14:31 EST
From: rdr@legislate.com (Raul Miller)
To: debian-bugs@pixar.com
Subject: upgrading source

Package: source

It's difficult to upgrade the 'source' package.  The current scheme of
things requires the admin muck around with dpkg-specific knowledge to
upgrade.  This is because the prerm script is:

......................................................................
#! /bin/sh

cat << EOF

The kernel source can only be removed manually.  Sorry.

If you *really* want to remove the kernel source, remove the directory
/usr/src/linux-<version>, and make the /usr/src/linux symbolic link
point to another directory containing the kernel include files.  Without
them, you will be unable to compile any programs on your system.

EOF

exit 1
......................................................................

I'm not sure what should be done about this..  One possibility that
comes to mind is replace the `exit 1` line with:

cp /var/lib/dpkg/info/source.list /usr/src/linux &&
: >/var/lib/dpkg/info/source.list

Perhaps there's a better solution?

Is this related to the R5/R6 problem with X?

-- 
Raul Miller