Ticket #334 (assigned defect)

Opened 15 years ago

Last modified 11 years ago

aclocal.m4 is Wrong

Reported by: andersk Owned by: achernya
Priority: low Milestone: The Distant Future
Component: development Keywords:
Cc: Fixed in version:
Upstream bug:

Description

aclocal.m4 is supposed to be generated by aclocal, not shipped as a holy blob that gets copied into every package. Furthermore, most of the macros inside it should probably be upstreamed, at which point it can disappear.

Change History

comment:1 Changed 15 years ago by ghudson

The first claim is only true if you're using automake. (See the aclocal man page and the package membership of /usr/bin/aclocal-1.10.)

comment:2 Changed 15 years ago by jdreed

  • Milestone set to The Distant Future

comment:3 Changed 15 years ago by jdreed

  • Component changed from -- to development

comment:4 Changed 14 years ago by broder

I don't think that simply upstreaming macros is going to get us out of this mess. For instance, krb5 has the AC_CHECK_KRB5 macro, but it has different semantics from both the ATHENA_KRB5 and ATHENA_KRB5_REQUIRED macros (both of our macros will error if you pass --with-krb5 but configure can't find the library; AC_CHECK_KRB5 simply warns in this case).

So even if we tried to drop our krb5-related macros in favor of the upstream macro, we'd still need a chunk of autoconf to wrap around AC_CHECK_KRB5 to get the semantics we want. And since ATHENA_KRB5{,_REQUIRED} are used in several of our packages, we'd want to pull that chunk of autoconf into *some* kind of common file so we weren't copying code around.

And we're back to square one.

As an alternative solution, what if we ship our aclocal.m4 as /usr/share/aclocal/athena.m4 in a package that any of our autoconf-using packages build-dep on, and then we set DEB_AUTO_UPDATE_ACLOCAL.

Of course, except for the part where DEB_AUTO_UPDATE_ACLOCAL is worthless because it only runs aclocal if there's already an aclocal.m4 file. Damn you,  DebianBug:461513

comment:5 Changed 14 years ago by broder

So I created the debathena-aclocal package, which contains our aclocal.m4 split up into a bunch of smaller files.

We can start using that along with a debian/rules snippet from libnss-nonlocal:

DEB_AUTO_UPDATE_ACLOCAL = 1.9
[...]
debian/stamp-autotools-files: aclocal.m4
aclocal.m4:
        touch $@

But this doesn't solve all of our problems. In particular, we still need to get install-sh from somewhere (it's distributed with automake, but needed for autoconf), and automake won't install it for us without actually using automake in our build system.

comment:6 Changed 14 years ago by andersk

I have an answer for install-sh, but you aren’t going to like it.

 http://web.mit.edu/andersk/Public/debathena-delete-unconfiscate.patch

comment:7 follow-up: ↓ 10 Changed 14 years ago by geofft

I have another answer for install-sh, which you might ... differently dislike. Keep in mind that I don't actually know autoconf in the slightest.

 AC_PROG_CC
+AC_CONFIG_AUX_DIRS("/usr/share/automake")
 AC_PROG_INSTALL

comment:8 Changed 14 years ago by geofft

Oh, I should mention that that of course necessitates adding automake as a build dependency, even though we're not using it in any real sense. (I'd likely feel better if we were doing this as a Debian patch, so that the non-Debianized part of the source doesn't upset people.)

The purpose of the error is so that you can take the package and hope to install it on a system without a BSD-compatible install program. Since we, in fact, hope that nobody will ever do so, I don't feel bad about brutally shutting up the error. I think it would also be fine to put a dummy install-sh in a directory provided by debathena-aclocal, although I'd have to look harder to see if $ac_aux_dir can ever be relevant.

comment:9 follow-up: ↓ 11 Changed 14 years ago by ghudson

If you're going to defeat the AC_PROG_INSTALL portability mechanism, it's probably simpler to just not use AC_PROG_INSTALL. I verified (on Hardy) that configure doesn't complain about a missing install-sh if you don't use that macro.

comment:10 in reply to: ↑ 7 Changed 14 years ago by andersk

Replying to geofft:

+AC_CONFIG_AUX_DIRS("/usr/share/automake")

/usr/share/automake is owned by automake1.4, so this would tie our packages to a particular version of automake (and an ancient one at that).

What about this variant of my answer? CDBS won’t call automake if there’s no Makefile.am, so we do it ourselves instead of tricking CDBS into doing it:

DEB_AUTO_UPDATE_AUTOMAKE = 1.9
DEB_AUTO_UPDATE_ACLOCAL = 1.9

debian/stamp-autotools-files: aclocal.m4 install-sh missing
aclocal.m4:
	touch $@
install-sh missing:
	-automake-$(DEB_AUTO_UPDATE_AUTOMAKE) --foreign --add-missing
clean::
	rm -f aclocal.m4 install-sh missing

comment:11 in reply to: ↑ 9 Changed 14 years ago by geofft

Replying to ghudson:

If you're going to defeat the AC_PROG_INSTALL portability mechanism, it's probably simpler to just not use AC_PROG_INSTALL. I verified (on Hardy) that configure doesn't complain about a missing install-sh if you don't use that macro.

I'm not defeating the entire mechanism, just the fallback if there is no working install binary anywhere on disk. As I understand it, with this in place AC_PROG_INSTALL will still figure out whether to use /usr/bin/install or another install in your path. But, again, chances are high that I'm completely misunderstanding what's going on here.

Ideally I'd like an AC_PROG_INSTALL that looks for install-sh only if there is no usable install program on the machine you're configuring for.

comment:12 follow-up: ↓ 13 Changed 14 years ago by broder

My standard debian/rules snippet for aclocal.m4-less builds is currently this:

debian/stamp-autotools-files: aclocal.m4 config.guess config.sub install-sh missing
aclocal.m4:
        aclocal-1.9
config.guess config.sub:
        cp /usr/share/misc/$@ $@
install-sh missing:
        -automake-1.9 --foreign --add-missing

and additionally adding automake1.9, autotools-dev, and possibly debathena-aclocal to the build-dependencies.

Using

aclocal.m4:
	touch $@

seemed to create a race condition between aclocal and autoconf where autoconf sometimes sees the empty aclocal.m4 we just touched instead of the one with the new macros in it.

So far I've applied this configuration to debathena-libathdir and debathena-athdir.

comment:13 in reply to: ↑ 12 Changed 14 years ago by andersk

Replying to broder:

Using
aclocal.m4:

touch $@

seemed to create a race condition between aclocal and autoconf where autoconf sometimes sees the empty aclocal.m4 we just touched instead of the one with the new macros in it.

That makes no sense, though. You said on Zephyr that you were able to reproduce this manually; can you explain how and where?

comment:14 Changed 14 years ago by broder

I agree it makes no sense, but what I've been seeing matches up with it.

I observed the bug during a package build both if I was running it out of AFS and if I was running it in our build chroots (ext3 on top of ext3 using aufs; both ext3 partitions are on LVM). In all cases, the bug only surfaced intermittently.

I was able to reproduce the bug by running debian/rules build out of my homedir. I was also able to reproduce the bug by running touch aclocal.m4; cp /usr/share/misc/config.sub .; cp /usr/share/misc/config.guess .; automake --add-missing --foreign; aclocal-1.9; autoconf; ./configure.

I was not able to reproduce the bug if I left off the touch aclocal.m4 at the beginning. I was not able to reproduce the bug if I typed the commands in sequence (i.e. I had to have them all in a single shell line). I was not able to reproduce the bug if I disabled autom4te.cache (using the instructions on  http://www.gnu.org/software/hello/manual/autoconf/Customizing-autom4te.html)

comment:15 Changed 14 years ago by andersk

Putting ‘sleep 1’ after ‘automake --add-missing --foreign’ seems to work around the problem. The contents of the directory before autoconf are identical whether or not the sleep is there, so the timestamps are important.

comment:16 follow-up: ↓ 17 Changed 13 years ago by geofft

For the record, it appears intended that automake errors out with

configure.ac: no proper invocation of AM_INIT_AUTOMAKE was found.
configure.ac: You should verify that configure.in invokes AM_INIT_AUTOMAKE,
configure.ac: that aclocal.m4 is present in the top-level directory,
configure.ac: and that aclocal.m4 was recently regenerated (using aclocal).

and the error is ignored by the - in the current Makefile snippet (and the || : in the additional automake arguments in Anders' original "you aren't going to like this".

Anyway, here's a list of packages that currently have a configure.in, which is basically the same as the packages that use daconfiscate:

athena/bin/athinfo/configure.in
athena/bin/athrun/configure.in
athena/bin/authwatch/configure.in
athena/bin/cxref/configure.in
athena/bin/dash/configure.in
athena/bin/dent/configure.in
athena/bin/desync/configure.in
athena/bin/finger/configure.in
athena/bin/gathlogout/configure.in
athena/bin/gathrun/configure.in
athena/bin/get_message/configure.in
athena/bin/getcluster/configure.in
athena/bin/grenew/configure.in
athena/bin/hesinfo/configure.in
athena/bin/hostinfo/configure.in
athena/bin/just/configure.in
athena/bin/lert/configure.in
athena/bin/machtype/configure.in
athena/bin/mitmailutils/configure.in
athena/bin/newpag/configure.in
athena/bin/quota/configure.in
athena/bin/saferm/configure.in
athena/bin/sendbug/configure.in
athena/bin/syncupdate/configure.in
athena/bin/xcluster/configure.in
athena/bin/xdsc/configure.in
athena/etc/athinfod/configure.in
athena/etc/busyd/configure.in
athena/etc/cleanup/configure.in
athena/etc/cviewd/configure.in
athena/etc/gettime/configure.in
athena/etc/ktconvert/configure.in
athena/etc/larvnetd/configure.in
athena/etc/listsuidcells/configure.in
athena/etc/messaged/configure.in
athena/lib/Xj/configure.in
athena/lib/ares/configure.in
athena/lib/firefox-extension/configure.in
athena/lib/gms/configure.in
athena/lib/hesiod/configure.in
athena/lib/larv/configure.in
third/moira/configure.in
third/moira/util/et/configure.in

A couple of these (listsuidcells? cxref?) never got packaged, and probably want to move to attic.

Last edited 12 years ago by andersk (previous) (diff)

comment:17 in reply to: ↑ 16 Changed 13 years ago by andersk

Replying to geofft:

For the record, it appears intended that automake errors out with
[…]
and the error is ignored by the - in the current Makefile snippet (and the || : in the additional automake arguments in Anders' original "you aren't going to like this".

Yes, see  546.

Last edited 12 years ago by andersk (previous) (diff)

comment:18 Changed 12 years ago by jdreed

I've lost track of where we are with this. I don't immediately see what's wrong (or unlikeable) about Anders' initial proposed patch. Is that approach still feasible?

comment:19 Changed 12 years ago by andersk

It looks like we already committed some form of this approach to a handful of packages, including delete (r25119 and r25120).

comment:20 Changed 11 years ago by jdreed

athdir 10.1 is in production.
athrun 10.1 is in -dev, except for raring (prod).

comment:21 Changed 11 years ago by jdreed

  • Status changed from new to assigned
  • Owner set to achernya

So, the current plan is to convert everything to automake and to use rra's krb5 macros (and everything else now uses pkg-config) in debathena-aclocal. So achernya will commit those changes to aclocal (but not yet rip out the old macros), push that through our workflow. Once that's on magrathea, we can rebuild the world, and then push out yet another new aclocal that kills the old macros.

Note: See TracTickets for help on using tickets.