Enhance the tracer with new data and fixes.

* Increase trace entry to 64-bytes.

The following patch increases the trace entry to 64-bytes, still a
proper multiple of the shared memory window size. While we have doubled
the entry size the on-disk format is still smaller than the ASCII
version. In the future we may wish to add variable sized records, but
for now the simplicity of this method works well.

With the extra bytes we are going to:
- Record internal size information for incoming (free) and outgoing
  chunks (malloc, calloc, realloc, etc).
  - Simplifies accounting of RSS usage and provides an extra cross check
    between malloc<->free based on internal chunk sizes.
- Record alignment information for memalign, and posix_memalign.
  - Continues to extend the tracer to the full API.
- Leave 128-bits of padding for future path uses.
  - Useful for more path information.

Additionally __MTB_TYPE_POSIX_MEMALIGN is added for the sole purpose of
recording the trace only so that we can hard-fail in the workload
converter when we see such an entry.

Lastly C_MEMALIGN, C_VALLOC, C_PVALLOC, and C_POSIX_MEMALIGN are added
for workload entries for the sake of completeness.

Builds on x86_64, capture looks good and it works.

* Teach trace_dump about the new entries.

The following patch teaches trace_dump about the new posix_memalign
entry. It also teaches trace_dump about the new size2 and size3 fields.
Tested by tracing a program that uses malloc, free, and memalign and
verifying that the extra fields show the expected chunk sizes, and
alignments dumped with trace_dump.

Tested on x86_64 with no apparently problems.

* Teach trace2wl and trace_run about new entries

(a) trace2wl changes:

The following patch teaches trace2wl how to output entries for valloc
and pvalloc, it does so exactly the same way it does for malloc, since
from the perspective of the API they are identical.

Additionally trace2wl is taught how to output an event for memalign,
storing alignment and size in the event record.

Lastly posix_memalign is detected and the converter aborted if it's
seen.  It is my opinion that we should not ignore this data during
conversion.  If we see a need for it we should implement it later.

(b) trace_run changes:

Some cosmetic cleanup in printing 'pthread_t' which is always an address
of the struct pthread structure in memory, so to make debugging easier
we should print the value as a hex pointer.

Teach the simulator how to run memalign. With the newly recorded
alignment information we double check that the resulting memory is
correctly aligned.

We do not implement valloc and pvalloc, they will abort the simulator.
This is incremental progress.

Tested on x86_64 by converting and running a multithreaded test
application that calls calloc, malloc, free, and memalign.

* Disable recursive traces and save new data.

(a) Adds support for disabling recurisvely recorded traces e.g. realloc
calling malloc no longer produces a realloc and malloc trace event. We
solve this by using a per-thread variable to disable new trace creation,
but allow path bits to be set.  This lets us record the code paths
taken, but only record one public API event.

(b) Save internal chunk size information into trace events for all APIs.
The most important is free where we record the free size, this allows
easier tooling to compute running idea RSS values.

Tested on x86_64 with some small applications and test programs.

1 files changed, 17 insertions, 11 deletions

diff --git a/malloc/trace_dump.c b/malloc/trace_dump.c
index d3a72f248c..7b691d9ed0 100644
--- a/malloc/trace_dump.c
+++ b/malloc/trace_dump.c
@@ -42,14 +42,15 @@ data_looks_like_raw_trace (unsigned char *data, long n_data)
 }
 
 const char * const typenames[] = {
-  "unused  ",
-  "malloc  ",
-  "calloc  ",
-  "free    ",
-  "realloc ",
+  "unused",
+  "malloc",
+  "calloc",
+  "free",
+  "realloc",
   "memalign",
-  "valloc  ",
-  "pvalloc  ",
+  "valloc",
+  "pvalloc",
+  "posix_memalign",
 };
 
 void
@@ -62,7 +63,8 @@ dump_raw_trace (unsigned char *data, long n_data)
 
   printf ("%ld out of %ld events captured (I think)\n", head, head);
 
-  printf ("threadid type     path     ptr1             size             ptr2\n");
+  printf ("%8s %8s %8s %16s %16s %16s %16s %16s\n",
+	  "threadid", "type", "path", "ptr1", "size", "ptr2", "size2", "size3");
 
   while (data <= edata - sizeof (struct __malloc_trace_buffer_s))
     {
@@ -73,9 +75,11 @@ dump_raw_trace (unsigned char *data, long n_data)
 	case __MTB_TYPE_UNUSED:
 	  break;
 	default:
-	  printf ("%08x %s %c%c%c%c%c%c%c%c %016llx %016llx %016llx\n",
+	  /* Consider 'memalign' to be the largest API word we want to align
+	     on so make the name 8 chars wide at a minimum.  */
+	  printf ("%08x %8s %c%c%c%c%c%c%c%c %016llx %016llx %016llx %016llx %016llx\n",
 		  t->thread,
-		  t->type == __MTB_TYPE_MAGIC ? "magic   " : typenames[t->type],
+		  t->type == __MTB_TYPE_MAGIC ? "magic" : typenames[t->type],
 		  t->path_thread_cache ? 'T' : '-',
 		  t->path_cpu_cache ? 'c' : '-',
 		  t->path_cpu_cache2 ? 'C' : '-',
@@ -86,7 +90,9 @@ dump_raw_trace (unsigned char *data, long n_data)
 		  t->path_hook ? 'H' : '-',
 		  (long long unsigned int) (size_t) t->ptr1,
 		  (long long unsigned int) t->size,
-		  (long long unsigned int) (size_t) t->ptr2);
+		  (long long unsigned int) (size_t) t->ptr2,
+		  (long long unsigned int) t->size2,
+		  (long long unsigned int) t->size3);
 	  break;
 	}