Enumerate tests with special rules in tests-special variable.

This patch is a revised and updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00196.html>.

In order to generate overall summaries of the results of all tests in
the glibc testsuite, we need to identify and concatenate the files
with the results of individual tests.

Tomas Dohnalek's patch used $(common-objpfx)*/*.test-result for this.
However, the normal glibc approach is explicit enumeration of the
expected set of files with a given property, rather than all files
matching some pattern like that.  Furthermore, we would like to be
able to mark tests as UNRESOLVED if the file with their results is for
some reason missing, and in future we would like to be able to mark
tests as UNSUPPORTED if they are disabled for a particular
configuration (rather than simply having them missing from the list of
tests as at present).  Such handling of tests that were not run or did
not record results requires an explicit enumeration of tests.

For the tests following the default makefile rules, $(tests) (and
$(xtests)) provides such an enumeration.  Others, however, are added
directly as dependencies of the "tests" and "xtests" makefile
targets.  This patch changes the makefiles to put them in variables
tests-special and xtests-special, with appropriate dependencies on the
tests listed there then being added centrally.

Those variables are used in Rules and so need to be set before Rules
is included in a subdirectory makefile, which is often earlier in the
makefile than the dependencies were present before.  We previously
discussed the question of where to include Rules; see the question at
<https://sourceware.org/ml/libc-alpha/2012-11/msg00798.html>, and a
discussion in
<https://sourceware.org/ml/libc-alpha/2013-01/msg00337.html> of why
Rules is included early rather than late in subdirectory makefiles.

It was necessary to avoid an indirection through the check-abi target
and get the check-abi-* targets for individual libraries into the
tests-special variable.  The intl/ test $(objpfx)tst-gettext.out,
previously built only because of dependencies from other tests, was
also added to tests-special for the same reason.

The entries in tests-special are the full makefile targets, complete
with $(objpfx) and .out.  If a future change causes tests to be named
consistently with a .out suffix, this can be changed to include just
the path relative to $(objpfx), without .out.

Tested x86_64, including that the same set of files is generated in
the build directory by a build and testsuite run both before and after
the patch (except for changes to the
elf/tst-null-argv.debug.out.<number> file name), and a build with
run-built-tests=no to verify there aren't any more obvious instances
of the issue Marcus Shawcroft reported with a previous version in
<https://sourceware.org/ml/libc-alpha/2014-01/msg00462.html>.

	* Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	(tests): Depend on $(tests-special).
	* Makerules (check-abi-list): New variable.
	(check-abi): Depend on $(check-abi-list).
	[$(subdir) = elf] (tests-special): Add
	$(objpfx)check-abi-libc.out.
	[$(build-shared) = yes && subdir] (tests-special): Add
	$(check-abi-list).
	[$(build-shared) = yes && subdir] (tests): Do not depend on
	check-abi.
	* Rules (tests): Depend on $(tests-special).
	(xtests): Depend on $(xtests-special).
	* catgets/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	* conform/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	* elf/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	* grp/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	* iconv/Makefile (xtests): Change dependencies to ....
	(xtests-special): ... additions to this variable.
	* iconvdata/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	* intl/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.  Also add
	$(objpfx)tst-gettext.out.
	* io/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	* libio/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	* malloc/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	* misc/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	* nptl/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	* nptl_db/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	* posix/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	(xtests): Change dependencies to ....
	(xtests-special): ... additions to this variable.
	* resolv/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	(xtests): Change dependencies to ....
	(xtests-special): ... additions to this variable.
	* stdio-common/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	(do-tst-unbputc): Remove target.
	(do-tst-printf): Likewise.
	* stdlib/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	* string/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.
	* sysdeps/x86/Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.

localedata:
	* Makefile (tests): Change dependencies to ....
	(tests-special): ... additions to this variable.

25 files changed, 207 insertions, 130 deletions

diff --git a/ChangeLog b/ChangeLog
index 89b40c8c88..89e8d03509 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,64 @@
 2014-03-06  Joseph Myers  <joseph@codesourcery.com>
 
+	* Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	(tests): Depend on $(tests-special).
+	* Makerules (check-abi-list): New variable.
+	(check-abi): Depend on $(check-abi-list).
+	[$(subdir) = elf] (tests-special): Add
+	$(objpfx)check-abi-libc.out.
+	[$(build-shared) = yes && subdir] (tests-special): Add
+	$(check-abi-list).
+	[$(build-shared) = yes && subdir] (tests): Do not depend on
+	check-abi.
+	* Rules (tests): Depend on $(tests-special).
+	(xtests): Depend on $(xtests-special).
+	* catgets/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	* conform/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	* elf/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	* grp/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	* iconv/Makefile (xtests): Change dependencies to ....
+	(xtests-special): ... additions to this variable.
+	* iconvdata/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	* intl/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.  Also add
+	$(objpfx)tst-gettext.out.
+	* io/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	* libio/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	* malloc/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	* misc/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	* nptl/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	* nptl_db/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	* posix/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	(xtests): Change dependencies to ....
+	(xtests-special): ... additions to this variable.
+	* resolv/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	(xtests): Change dependencies to ....
+	(xtests-special): ... additions to this variable.
+	* stdio-common/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	(do-tst-unbputc): Remove target.
+	(do-tst-printf): Likewise.
+	* stdlib/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	* string/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+	* sysdeps/x86/Makefile (tests): Change dependencies to ....
+	(tests-special): ... additions to this variable.
+
 	* conform/data/netinet/tcp.h-data [POSIX2008 || XOPEN2K8]: Enable
 	whole file.
 	* conform/data/sys/timeb.h-data [POSIX2008 || XOPEN2K8]: Disable
diff --git a/Makefile b/Makefile
index a9b559716a..aee88e1e3e 100644
--- a/Makefile
+++ b/Makefile
@@ -250,7 +250,7 @@ mostlyclean: parent-mostlyclean
 tests-clean:
 	@$(MAKE) subdir_testclean no_deps=t
 
-tests: $(objpfx)c++-types-check.out $(objpfx)check-local-headers.out
+tests-special += $(objpfx)c++-types-check.out $(objpfx)check-local-headers.out
 ifneq ($(CXX),no)
 
 vpath c++-types.data $(+sysdep_dirs)
@@ -310,12 +310,14 @@ installed-headers = argp/argp.h assert/assert.h catgets/nl_types.h \
 		    time/sys/time.h time/sys/timeb.h wcsmbs/wchar.h \
 		    wctype/wctype.h
 
-tests: $(objpfx)begin-end-check.out
+tests-special += $(objpfx)begin-end-check.out
 $(objpfx)begin-end-check.out: scripts/begin-end-check.pl
 	$(PERL) scripts/begin-end-check.pl $(installed-headers) > $@; \
 	$(evaluate-test)
 endif
 
+tests: $(tests-special)
+
 # The realclean target is just like distclean for the parent, but we want
 # the subdirs to know the difference in case they care.
 realclean distclean: parent-clean
diff --git a/Makerules b/Makerules
index c2324366df..ad81ea9f76 100644
--- a/Makerules
+++ b/Makerules
@@ -1246,7 +1246,9 @@ endef
 
 .PHONY: update-abi check-abi
 update-abi: $(patsubst %.so,update-abi-%,$(install-lib.so-versioned))
-check-abi: $(patsubst %.so,$(objpfx)check-abi-%.out,$(install-lib.so-versioned))
+check-abi-list = $(patsubst %.so,$(objpfx)check-abi-%.out, \
+				 $(install-lib.so-versioned))
+check-abi: $(check-abi-list)
 ifdef subdir
 subdir_check-abi: check-abi
 subdir_update-abi: update-abi
@@ -1257,13 +1259,14 @@ endif
 
 ifeq ($(subdir),elf)
 check-abi: $(objpfx)check-abi-libc.out
+tests-special += $(objpfx)check-abi-libc.out
 update-abi: update-abi-libc
 common-generated += libc.symlist
 endif
 
 ifeq ($(build-shared),yes)
 ifdef subdir
-tests: check-abi
+tests-special += $(check-abi-list)
 endif
 endif
 
diff --git a/Rules b/Rules
index 3e305ddaa5..d37ae52b6c 100644
--- a/Rules
+++ b/Rules
@@ -95,11 +95,11 @@ else
 others: $(addprefix $(objpfx),$(extra-objs))
 endif
 ifeq ($(run-built-tests),no)
-tests: $(addprefix $(objpfx),$(tests) $(test-srcs))
-xtests: tests
+tests: $(addprefix $(objpfx),$(tests) $(test-srcs)) $(tests-special)
+xtests: tests $(xtests-special)
 else
-tests: $(tests:%=$(objpfx)%.out)
-xtests: tests $(xtests:%=$(objpfx)%.out)
+tests: $(tests:%=$(objpfx)%.out) $(tests-special)
+xtests: tests $(xtests:%=$(objpfx)%.out) $(xtests-special)
 endif
 
 ifeq ($(build-programs),yes)
diff --git a/catgets/Makefile b/catgets/Makefile
index 5eba3b5605..02ca263c4e 100644
--- a/catgets/Makefile
+++ b/catgets/Makefile
@@ -31,6 +31,11 @@ extra-objs	= $(gencat-modules:=.o)
 tests = tst-catgets
 test-srcs = test-gencat
 
+ifeq ($(run-built-tests),yes)
+tests-special += $(objpfx)de/libc.cat $(objpfx)test1.cat $(objpfx)test2.cat \
+		 $(objpfx)test-gencat.out
+endif
+
 gencat-modules	= xmalloc
 
 # To find xmalloc.c
@@ -53,8 +58,6 @@ generated-dirs += de
 tst-catgets-ENV = NLSPATH="$(objpfx)%l/%N.cat" LANG=de
 
 ifeq ($(run-built-tests),yes)
-tests: $(objpfx)de/libc.cat $(objpfx)test1.cat $(objpfx)test2.cat \
-       $(objpfx)test-gencat.out
 # This test just checks whether the program produces any error or not.
 # The result is not tested.
 $(objpfx)test1.cat: test1.msg $(objpfx)gencat
diff --git a/conform/Makefile b/conform/Makefile
index 40081f304b..3f76773758 100644
--- a/conform/Makefile
+++ b/conform/Makefile
@@ -22,14 +22,14 @@ subdir	:= conform
 
 include ../Makeconfig
 
-include ../Rules
-
 ifneq (yes,$(fast-check))
 ifeq (no,$(cross-compiling))
-tests: $(objpfx)run-conformtest.out
+tests-special += $(objpfx)run-conformtest.out
 endif
 endif
 
+include ../Rules
+
 test-xfail-run-conformtest = yes
 $(objpfx)run-conformtest.out: run-conformtest.sh conformtest.pl \
 			      $(wildcard data/*.h-data) \
diff --git a/elf/Makefile b/elf/Makefile
index 5637b4899c..1007a63c6c 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -157,8 +157,8 @@ tests-execstack-yes = tst-execstack tst-execstack-needed tst-execstack-prog
 endif
 endif
 ifeq ($(run-built-tests),yes)
-tests: $(objpfx)tst-leaks1-mem $(objpfx)tst-leaks1-static-mem \
-       $(objpfx)noload-mem
+tests-special += $(objpfx)tst-leaks1-mem $(objpfx)tst-leaks1-static-mem \
+		 $(objpfx)noload-mem
 endif
 tlsmod17a-suffixes = 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
 tlsmod18a-suffixes = 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
@@ -253,12 +253,31 @@ modules-names += ifuncmod1 ifuncmod3 ifuncmod5 ifuncmod6
 endif
 endif
 
+ifeq (yes,$(build-shared))
+ifeq ($(run-built-tests),yes)
+tests-special += $(objpfx)tst-pathopt.out $(objpfx)tst-rtld-load-self.out
+endif
+tests-special += $(objpfx)check-textrel.out $(objpfx)check-execstack.out \
+		 $(objpfx)check-localplt.out
+endif
 
-include ../Rules
+ifeq ($(run-built-tests),yes)
+tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \
+		 $(objpfx)tst-array1-static-cmp.out \
+		 $(objpfx)tst-array2-cmp.out $(objpfx)tst-array3-cmp.out \
+		 $(objpfx)tst-array4-cmp.out $(objpfx)tst-array5-cmp.out \
+		 $(objpfx)tst-array5-static-cmp.out $(objpfx)order2-cmp.out \
+		 $(objpfx)tst-initorder-cmp.out \
+		 $(objpfx)tst-initorder2-cmp.out $(objpfx)tst-unused-dep.out \
+		 $(objpfx)tst-unused-dep-cmp.out
+endif
 
 check-abi: $(objpfx)check-abi-ld.out
+tests-special += $(objpfx)check-abi-ld.out
 update-abi: update-abi-ld
 
+include ../Rules
+
 ifeq (yes,$(build-shared))
 # Make sure these things are built in the `make lib' pass so they can be used
 # to run programs during the `make others' pass.
@@ -430,12 +449,6 @@ CPPFLAGS-.os += $(if $(filter $(@F),$(patsubst %,%.os,$(all-rtld-routines))),\
 test-modules = $(addprefix $(objpfx),$(addsuffix .so,$(strip $(modules-names))))
 generated += $(addsuffix .so,$(strip $(modules-names)))
 
-ifeq (yes,$(build-shared))
-ifeq ($(run-built-tests),yes)
-tests: $(objpfx)tst-pathopt.out $(objpfx)tst-rtld-load-self.out
-endif
-endif
-
 $(objpfx)testobj1.so: $(libdl)
 $(objpfx)testobj1_1.so: $(objpfx)testobj1.so $(libdl)
 $(objpfx)testobj2.so: $(objpfx)testobj1.so $(libdl)
@@ -640,9 +653,6 @@ $(objpfx)circlemod2a.so: $(objpfx)circlemod3a.so
 
 $(objpfx)order: $(addprefix $(objpfx),dep4.so dep3.so dep2.so dep1.so)
 
-ifeq ($(run-built-tests),yes)
-tests: $(objpfx)order-cmp.out
-endif
 $(objpfx)order-cmp.out: $(objpfx)order.out
 	(echo "0123456789" | cmp $< -) > $@; \
 	$(evaluate-test)
@@ -856,13 +866,6 @@ endif
 LDFLAGS-tst-array2 = $(no-as-needed)
 LDFLAGS-tst-array5 = $(no-as-needed)
 
-ifeq ($(run-built-tests),yes)
-tests: $(objpfx)tst-array1-cmp.out $(objpfx)tst-array1-static-cmp.out \
-       $(objpfx)tst-array2-cmp.out $(objpfx)tst-array3-cmp.out \
-       $(objpfx)tst-array4-cmp.out $(objpfx)tst-array5-cmp.out \
-       $(objpfx)tst-array5-static-cmp.out
-endif
-
 $(objpfx)tst-array1-cmp.out: tst-array1.exp $(objpfx)tst-array1.out
 	cmp $^ > $@; \
 	$(evaluate-test)
@@ -902,8 +905,6 @@ CFLAGS-tst-pie1.c += $(pie-ccflag)
 $(objpfx)tst-pie1: $(objpfx)tst-piemod1.so
 
 ifeq (yes,$(build-shared))
-tests: $(objpfx)check-textrel.out $(objpfx)check-execstack.out
-
 all-built-dso := $(common-objpfx)libc.so \
 		 $(filter-out $(common-objpfx)linkobj/libc.so, \
 			      $(sort $(wildcard $(addprefix $(common-objpfx), \
@@ -961,8 +962,6 @@ $(all-built-dso:=.jmprel): %.jmprel: % Makefile
 	mv -f $@T $@
 common-generated += $(all-built-dso:$(common-objpfx)%=%.jmprel)
 
-tests: $(objpfx)check-localplt.out
-
 localplt-built-dso := $(addprefix $(common-objpfx),\
 				  libc.so \
 				  math/libm.so \
@@ -1026,9 +1025,6 @@ tst-audit8-ENV = LD_AUDIT=$(objpfx)tst-auditmod1.so
 $(objpfx)tst-global1: $(libdl)
 $(objpfx)tst-global1.out: $(objpfx)testobj6.so $(objpfx)testobj2.so
 
-ifeq ($(run-built-tests),yes)
-tests: $(objpfx)order2-cmp.out
-endif
 $(objpfx)order2: $(libdl)
 $(objpfx)order2.out: $(objpfx)order2mod1.so $(objpfx)order2mod2.so
 $(objpfx)order2-cmp.out: $(objpfx)order2.out
@@ -1131,9 +1127,6 @@ $(objpfx)tst-unique3.out: $(objpfx)tst-unique3lib2.so
 
 $(objpfx)tst-unique4: $(objpfx)tst-unique4lib.so
 
-ifeq ($(run-built-tests),yes)
-tests: $(objpfx)tst-initorder-cmp.out
-endif
 $(objpfx)tst-initorder-cmp.out: tst-initorder.exp $(objpfx)tst-initorder.out
 	cmp $^ > $@; \
 	$(evaluate-test)
@@ -1153,9 +1146,6 @@ endef
 object-suffixes-left := a b c d
 include $(o-iterator)
 
-ifeq ($(run-built-tests),yes)
-tests: $(objpfx)tst-initorder2-cmp.out
-endif
 $(objpfx)tst-initorder2-cmp.out: tst-initorder2.exp $(objpfx)tst-initorder2.out
 	cmp $^ > $@; \
 	$(evaluate-test)
@@ -1166,10 +1156,6 @@ $(objpfx)tst-relsort1mod2.so: $(libm)
 $(objpfx)tst-relsort1.out: $(objpfx)tst-relsort1mod1.so \
 			   $(objpfx)tst-relsort1mod2.so
 
-ifeq ($(run-built-tests),yes)
-tests: $(objpfx)tst-unused-dep.out $(objpfx)tst-unused-dep-cmp.out
-endif
-
 $(objpfx)tst-unused-dep.out: $(objpfx)testobj1.so
 	$(test-wrapper-env) \
 	LD_TRACE_LOADED_OBJECTS=1 \
diff --git a/grp/Makefile b/grp/Makefile
index 3ed8471fdd..a60d2f95ab 100644
--- a/grp/Makefile
+++ b/grp/Makefile
@@ -32,6 +32,9 @@ tests := testgrp
 
 ifeq (yes,$(build-shared))
 test-srcs :=  tst_fgetgrent
+ifeq ($(run-built-tests),yes)
+tests-special += $(objpfx)tst_fgetgrent.out
+endif
 endif
 
 
@@ -54,7 +57,6 @@ endif
 ifeq ($(run-built-tests),yes)
 # tst_fgetgrent currently only works with shared libraries
 ifeq (yes,$(build-shared))
-tests: $(objpfx)tst_fgetgrent.out
 $(objpfx)tst_fgetgrent.out: tst_fgetgrent.sh $(objpfx)tst_fgetgrent
 	$(SHELL) $< $(common-objpfx) '$(test-program-prefix)'; \
 	$(evaluate-test)
diff --git a/iconv/Makefile b/iconv/Makefile
index ec3535412e..8728fefc13 100644
--- a/iconv/Makefile
+++ b/iconv/Makefile
@@ -60,6 +60,10 @@ CPPFLAGS-strtab = -DNOT_IN_libc
 CPPFLAGS-charmap = -DNOT_IN_libc
 CPPFLAGS-charmap-dir = -DNOT_IN_libc
 
+ifeq ($(run-built-tests),yes)
+xtests-special += test-iconvconfig
+endif
+
 include ../Rules
 
 $(inst_bindir)/iconv: $(objpfx)iconv_prog $(+force)
@@ -68,10 +72,6 @@ $(inst_bindir)/iconv: $(objpfx)iconv_prog $(+force)
 $(objpfx)iconv_prog: $(iconv_prog-modules:%=$(objpfx)%.o)
 $(objpfx)iconvconfig: $(iconvconfig-modules:%=$(objpfx)%.o)
 
-ifeq ($(run-built-tests),yes)
-xtests: test-iconvconfig
-endif
-
 .PHONY: test-iconvconfig
 test-iconvconfig: /dev/null $(objpfx)iconvconfig
 	tmp=$(objpfx)gconv-modules.cache.$$$$; \
diff --git a/iconvdata/Makefile b/iconvdata/Makefile
index 8fabe52718..f0d832af6d 100644
--- a/iconvdata/Makefile
+++ b/iconvdata/Makefile
@@ -265,17 +265,17 @@ endif
 
 endif # build-shared = yes
 
-include ../Rules
-
 ifeq ($(run-built-tests),yes)
 ifeq (yes,$(build-shared))
-tests: $(objpfx)iconv-test.out $(objpfx)tst-tables.out
+tests-special += $(objpfx)iconv-test.out $(objpfx)tst-tables.out
 ifneq (no,$(PERL))
-tests: $(objpfx)mtrace-tst-loading
+tests-special += $(objpfx)mtrace-tst-loading
 endif
 endif
 endif
 
+include ../Rules
+
 tst-loading-ENV = MALLOC_TRACE=$