diff options
| author | Paul Eggert <eggert@cs.ucla.edu> | 2015-12-04 08:27:14 -0800 |
|---|---|---|
| committer | Paul Eggert <eggert@cs.ucla.edu> | 2015-12-04 08:28:34 -0800 |
| commit | 2cc4b9cce51643ec299e97450ccde4deeecfb083 (patch) | |
| tree | b3a0b638c939c9db32d5ae755489da00ff2661db /manual/string.texi | |
| parent | e59c94fa0e2871bdfcc363899e3be376c0def770 (diff) | |
| download | glibc-2cc4b9cce51643ec299e97450ccde4deeecfb083.tar.xz glibc-2cc4b9cce51643ec299e97450ccde4deeecfb083.zip | |
Consistency about byte vs character in string.texi
* manual/string.texi (String and Array Utilities):
Distinguish more carefully among bytes, multibyte characters,
and wide characters. Use "byte" when talking about C 'char',
to distinguish it more clearly from multibyte characters.
Say "wide character" or "multibyte character" instead of
"character", when a wide or multibyte character is intended.
Similarly for "multibyte string" versus "string".
Define these terms more carefully.
Diffstat (limited to 'manual/string.texi')
| -rw-r--r-- | manual/string.texi | 457 |
1 files changed, 236 insertions, 221 deletions
diff --git a/manual/string.texi b/manual/string.texi index 5f8a17ec48..4f276a95ae 100644 --- a/manual/string.texi +++ b/manual/string.texi @@ -2,7 +2,7 @@ @c %MENU% Utilities for copying and comparing strings and arrays @chapter String and Array Utilities -Operations on strings (or arrays of characters) are an important part of +Operations on strings (null-terminated byte sequences) are an important part of many programs. @Theglibc{} provides an extensive set of string utility functions, including functions for copying, concatenating, comparing, and searching strings. Many of these functions can also @@ -44,13 +44,13 @@ too. @cindex string, representation of This section is a quick summary of string concepts for beginning C -programmers. It describes how character strings are represented in C +programmers. It describes how strings are represented in C and some common pitfalls. If you are already familiar with this material, you can skip this section. @cindex string -@cindex multibyte character string -A @dfn{string} is an array of @code{char} objects. But string-valued +A @dfn{string} is a null-terminated array of bytes of type @code{char}, +including the terminating null byte. String-valued variables are usually declared to be pointers of type @code{char *}. Such variables do not include space for the text of a string; that has to be stored somewhere else---in an array variable, a string constant, @@ -60,66 +60,74 @@ variable. Alternatively you can store a @dfn{null pointer} in the pointer variable. The null pointer does not point anywhere, so attempting to reference the string it points to gets an error. -@cindex wide character string -``string'' normally refers to multibyte character strings as opposed to -wide character strings. Wide character strings are arrays of type -@code{wchar_t} and as for multibyte character strings usually pointers -of type @code{wchar_t *} are used. - -@cindex null character +@cindex multibyte character +@cindex multibyte string +@cindex wide string +A @dfn{multibyte character} is a sequence of one or more bytes that +represents a single character using the locale's encoding scheme; a +null byte always represents the null character. A @dfn{multibyte +string} is a string that consists entirely of multibyte +characters. In contrast, a @dfn{wide string} is a null-terminated +sequence of @code{wchar_t} objects. A wide-string variable is usually +declared to be a pointer of type @code{wchar_t *}, by analogy with +string variables and @code{char *}. @xref{Extended Char Intro}. + +@cindex null byte @cindex null wide character -By convention, a @dfn{null character}, @code{'\0'}, marks the end of a -multibyte character string and the @dfn{null wide character}, -@code{L'\0'}, marks the end of a wide character string. For example, in +By convention, the @dfn{null byte}, @code{'\0'}, +marks the end of a string and the @dfn{null wide character}, +@code{L'\0'}, marks the end of a wide string. For example, in testing to see whether the @code{char *} variable @var{p} points to a -null character marking the end of a string, you can write +null byte marking the end of a string, you can write @code{!*@var{p}} or @code{*@var{p} == '\0'}. -A null character is quite different conceptually from a null pointer, -although both are represented by the integer @code{0}. +A null byte is quite different conceptually from a null pointer, +although both are represented by the integer constant @code{0}. @cindex string literal -@dfn{String literals} appear in C program source as strings of -characters between double-quote characters (@samp{"}) where the initial -double-quote character is immediately preceded by a capital @samp{L} -(ell) character (as in @code{L"foo"}). In @w{ISO C}, string literals -can also be formed by @dfn{string concatenation}: @code{"a" "b"} is the -same as @code{"ab"}. For wide character strings one can either use +A @dfn{string literal} appears in C program source as a multibyte +string between double-quote characters (@samp{"}). If the +initial double-quote character is immediately preceded by a capital +@samp{L} (ell) character (as in @code{L"foo"}), it is a wide string +literal. String literals can also contribute to @dfn{string +concatenation}: @code{"a" "b"} is the same as @code{"ab"}. +For wide strings one can use either @code{L"a" L"b"} or @code{L"a" "b"}. Modification of string literals is not allowed by the GNU C compiler, because literals are placed in read-only storage. -Character arrays that are declared @code{const} cannot be modified +Arrays that are declared @code{const} cannot be modified either. It's generally good style to declare non-modifiable string pointers to be of type @code{const char *}, since this often allows the C compiler to detect accidental modifications as well as providing some amount of documentation about what your program intends to do with the string. -The amount of memory allocated for the character array may extend past -the null character that normally marks the end of the string. In this +The amount of memory allocated for a byte array may extend past the null byte +that marks the end of the string that the array contains. In this document, the term @dfn{allocated size} is always used to refer to the -total amount of memory allocated for the string, while the term -@dfn{length} refers to the number of characters up to (but not -including) the terminating null character. +total amount of memory allocated for an array, while the term +@dfn{length} refers to the number of bytes up to (but not including) +the terminating null byte. Wide strings are similar, except their +sizes and lengths count wide characters, not bytes. @cindex length of string @cindex allocation size of string @cindex size of string @cindex string length @cindex string allocation -A notorious source of program bugs is trying to put more characters in a +A notorious source of program bugs is trying to put more bytes into a string than fit in its allocated size. When writing code that extends -strings or moves characters into a pre-allocated array, you should be +strings or moves bytes into a pre-allocated array, you should be very careful to keep track of the length of the text and make explicit checks for overflowing the array. Many of the library functions @emph{do not} do this for you! Remember also that you need to allocate -an extra byte to hold the null character that marks the end of the +an extra byte to hold the null byte that marks the end of the string. @cindex single-byte string @cindex multibyte string -Originally strings were sequences of bytes where each byte represents a +Originally strings were sequences of bytes where each byte represented a single character. This is still true today if the strings are encoded using a single-byte character encoding. Things are different if the strings are encoded using a multibyte encoding (for more information on @@ -130,37 +138,37 @@ has to be aware of this and interpret the byte sequences accordingly. But since there is no separate interface taking care of these differences the byte-based string functions are sometimes hard to use. Since the count parameters of these functions specify bytes a call to -@code{strncpy} could cut a multibyte character in the middle and put an +@code{memcpy} could cut a multibyte character in the middle and put an incomplete (and therefore unusable) byte sequence in the target buffer. -@cindex wide character string +@cindex wide string To avoid these problems later versions of the @w{ISO C} standard introduce a second set of functions which are operating on @dfn{wide characters} (@pxref{Extended Char Intro}). These functions don't have the problems the single-byte versions have since every wide character is a legal, interpretable value. This does not mean that cutting wide -character strings at arbitrary points is without problems. It normally +strings at arbitrary points is without problems. It normally is for alphabet-based languages (except for non-normalized text) but languages based on syllables still have the problem that more than one wide character is necessary to complete a logical unit. This is a higher level problem which the @w{C library} functions are not designed to solve. But it is at least good that no invalid byte sequences can be -created. Also, the higher level functions can also much easier operate -on wide character than on multibyte characters so that a general advise +created. Also, the higher level functions can also much more easily operate +on wide characters than on multibyte characters so that a common strategy is to use wide characters internally whenever text is more than simply copied. The remaining of this chapter will discuss the functions for handling -wide character strings in parallel with the discussion of the multibyte -character strings since there is almost always an exact equivalent +wide strings in parallel with the discussion of +strings since there is almost always an exact equivalent available. @node String/Array Conventions @section String and Array Conventions This chapter describes both functions that work on arbitrary arrays or -blocks of memory, and functions that are specific to null-terminated -arrays of characters and wide characters. +blocks of memory, and functions that are specific to strings and wide +strings. Functions that operate on arbitrary blocks of memory have names beginning with @samp{mem} and @samp{wmem} (such as @code{memcpy} and @@ -176,21 +184,21 @@ size argument. Parameters to the @samp{wmem} functions must be of type but arrays of this type. In contrast, functions that operate specifically on strings and wide -character strings have names beginning with @samp{str} and @samp{wcs} +strings have names beginning with @samp{str} and @samp{wcs} respectively (such as @code{strcpy} and @code{wcscpy}) and look for a -null character to terminate the string instead of requiring an explicit +terminating null byte or null wide character instead of requiring an explicit size argument to be passed. (Some of these functions accept a specified -maximum length, but they also check for premature termination with a -null character.) The array arguments and return values for these +maximum length, but they also check for premature termination.) +The array arguments and return values for these functions have type @code{char *} and @code{wchar_t *} respectively, and -the array elements are referred to as ``characters'' and ``wide +the array elements are referred to as ``bytes'' and ``wide characters''. In many cases, there are both @samp{mem} and @samp{str}/@samp{wcs} versions of a function. The one that is more appropriate to use depends on the exact situation. When your program is manipulating arbitrary arrays or blocks of storage, then you should always use the @samp{mem} -functions. On the other hand, when you are manipulating null-terminated +functions. On the other hand, when you are manipulating strings it is usually more convenient to use the @samp{str}/@samp{wcs} functions, unless you already know the length of the string in advance. The @samp{wmem} functions should be used for wide character arrays with @@ -202,10 +210,10 @@ Some of the memory and string functions take single characters as arguments. Since a value of type @code{char} is automatically promoted into a value of type @code{int} when used as a parameter, the functions are declared with @code{int} as the type of the parameter in question. -In case of the wide character function the situation is similarly: the +In case of the wide character functions the situation is similar: the parameter type for a single wide character is @code{wint_t} and not @code{wchar_t}. This would for many implementations not be necessary -since the @code{wchar_t} is large enough to not be automatically +since @code{wchar_t} is large enough to not be automatically promoted, but since the @w{ISO C} standard does not require such a choice of types the @code{wint_t} type is used. @@ -220,9 +228,9 @@ This function is declared in the header file @file{string.h}. @comment ISO @deftypefun size_t strlen (const char *@var{s}) @safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}} -The @code{strlen} function returns the length of the null-terminated +The @code{strlen} function returns the length of the string @var{s} in bytes. (In other words, it returns the offset of the -terminating null character within the array.) +terminating null byte within the array.) For example, @smallexample @@ -230,9 +238,9 @@ strlen ("hello, world") @result{} 12 @end smallexample -When applied to a character array, the @code{strlen} function returns +When applied to an array, the @code{strlen} function returns the length of the string stored there, not its allocated size. You can -get the allocated size of the character array that holds a string using +get the allocated size of the array that holds a string using the @code{sizeof} operator: @smallexample @@ -243,7 +251,7 @@ strlen (string) @result{} 12 @end smallexample -But beware, this will not work unless @var{string} is the character +But beware, this will not work unless @var{string} is the array itself, not a pointer to it. For example: @smallexample @@ -289,10 +297,10 @@ The wide character equivalent is declared in @file{wchar.h}. @safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}} The @code{wcslen} function is the wide character equivalent to @code{strlen}. The return value is the number of wide characters in the -wide character string pointed to by @var{ws} (this is also the offset of +wide string pointed to by @var{ws} (this is also the offset of the terminating null wide character of @var{ws}). -Since there are no multi wide character sequences making up one +Since there are no multi wide character sequences making up one wide character the return value is not only the offset in the array, it is also the number of wide characters. @@ -303,13 +311,14 @@ This function was introduced in @w{Amendment 1} to @w{ISO C90}. @comment GNU @deftypefun size_t strnlen (const char *@var{s}, size_t @var{maxlen}) @safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}} -The @code{strnlen} function returns the length of the string @var{s} in -bytes if this length is smaller than @var{maxlen} bytes. Otherwise it +If the array @var{s} of size @var{maxlen} contains a null byte, +the @code{strnlen} function returns the length of the string @var{s} in +bytes. Otherwise it returns @var{maxlen}. Therefore this function is equivalent to @code{(strlen (@var{s}) < @var{maxlen} ? strlen (@var{s}) : @var{maxlen})} but it -is more efficient and works even if the string @var{s} is not -null-terminated. +is more efficient and works even if @var{s} is not null-terminated so +long as @var{maxlen} does not exceed the size of @var{s}'s array. @smallexample char string[32] = "hello, world"; @@ -358,7 +367,7 @@ destination arrays overlap. For example, if the beginning of the destination array overlaps the end of the source array, the original contents of that part of the source array may get overwritten before it is copied. Even worse, in the case of the string functions, the null -character marking the end of the string may be lost, and the copy +byte marking the end of the string may be lost, and the copy function might get stuck in a loop trashing all the memory allocated to your program. @@ -547,8 +556,8 @@ returns the value of @var{block}. @comment ISO @deftypefun {char *} strcpy (char *restrict @var{to}, const char *restrict @var{from}) @safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}} -This copies characters from the string @var{from} (up to and including -the terminating null character) into the string @var{to}. Like +This copies bytes from the string @var{from} (up to and including +the terminating null byte) into the string @var{to}. Like @code{memcpy}, this function has undefined results if the strings overlap. The return value is the value of @var{to}. @end deftypefun @@ -557,7 +566,7 @@ overlap. The return value is the value of @var{to}. @comment ISO @deftypefun {wchar_t *} wcscpy (wchar_t *restrict @var{wto}, const wchar_t *restrict @var{wfrom}) @safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}} -This copies wide characters from the string @var{wfrom} (up to and +This copies wide characters from the wide string @var{wfrom} (up to and including the terminating null wide character) into the string @var{wto}. Like @code{wmemcpy}, this function has undefined results if the strings overlap. The return value is the value of @var{wto}. @@ -568,15 +577,16 @@ the strings overlap. The return value is the value of @var{wto}. @deftypefun {char *} strncpy (char *restrict @var{to}, const char *restrict @var{from}, size_t @var{size}) @safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}} This function is similar to @code{strcpy} but always copies exactly -@var{size} characters into @var{to}. +@var{size} bytes into @var{to}. -If the length of @var{from} is more than @var{size}, then @code{strncpy} -copies just the first @var{size} characters. Note that in this case -there is no null terminator written into @var{to}. +If @var{from} does not contain a null byte in its first @var{size} +bytes, @code{strncpy} copies just the first @var{size} bytes. In this +case no null terminator is written into @var{to}. -If the length of @var{from} is less than @var{size}, then @code{strncpy} -copies all of @var{from}, followed by enough null characters to add up -to @var{size} characters in all. This behavior is rarely useful, but it +Otherwise @var{from} must be a string with length less than +@var{size}. In this case @code{strncpy} copies all of @var{from}, +followed by enough null bytes to add up to @var{size} bytes in all. +This behavior is rarely useful, but it is specified by the @w{ISO C} standard. The behavior of @code{strncpy} is undefined if the strings overlap. @@ -586,7 +596,7 @@ relating to writing past the end of the allocated space for @var{to}. However, it can also make your program much slower in one common case: copying a string which is probably small into a potentially large buffer. In this case, @var{size} may be large, and when it is, @code{strncpy} will -waste a considerable amount of time copying null characters. +waste a considerable amount of time copying null bytes. @end deftypefun @comment wchar.h @@ -596,12 +606,14 @@ waste a considerable amount of time copying null characters. This function is similar to @code{wcscpy} but always copies exactly @var{size} wide characters into @var{wto}. -If the length of @var{wfrom} is more than @var{size}, then -@code{wcsncpy} copies just the first @var{size} wide characters. Note -that in this case there is no null terminator written into @var{wto}. +If @var{wfrom} does not contain a null wide character in its first +@var{size} wide characters, then @code{wcsncpy} copies just the first +@var{size} wide characters. In this case no null terminator is +written into @var{wto}. -If the length of @var{wfrom} is less than @var{size}, then -@code{wcsncpy} copies all of @var{wfrom}, followed by enough null wide +Otherwise @var{wfrom} must be a wide string with length less than +@var{size}. In this case @code{wcsncpy} copies all of @var{wfrom}, +followed by enough null wide characters to add up to @var{size} wide characters in all. This behavior is rarely useful, but it is specified by the @w{ISO C} standard. @@ -620,7 +632,7 @@ waste a considerable amount of time copying null wide characters. @comment SVID @deftypefun {char *} strdup (const char *@var{s}) @safety{@prelim{}@mtsafe{}@asunsafe{@ascuheap{}}@acunsafe{@acsmem{}}} -This function copies the null-terminated string @var{s} into a newly +This function copies the string @var{s} into a newly allocated string. The string is allocated using @code{malloc}; see @ref{Unconstrained Allocation}. If @code{malloc} cannot allocate space for the new string, @code{strdup} returns a null pointer. Otherwise it @@ -631,12 +643,11 @@ returns a pointer to the new string. @comment GNU @deftypefun {wchar_t *} wcsdup (const wchar_t *@var{ws}) @safety{@prelim{}@mtsafe{}@asunsafe{@ascuheap{}}@acunsafe{@acsmem{}}} -This function copies the null-terminated wide character string @var{ws} +This function copies the wide string @var{ws} into a newly allocated string. The string is allocated using @code{malloc}; see @ref{Unconstrained Allocation}. If @code{malloc} cannot allocate space for the new string, @code{wcsdup} returns a null -pointer. Otherwise it returns a pointer to the new wide character -string. +pointer. Otherwise it returns a pointer to the new wide string. This function is a GNU extension. @end deftypefun @@ -646,11 +657,11 @@ This function is a GNU extension. @deftypefun {char *} strndup (const char *@var{s}, size_t @var{size}) @safety{@prelim{}@mtsafe{}@asunsafe{@ascuheap{}}@acunsafe{@acsmem{}}} This function is similar to @code{strdup} but always copies at most -@var{size} characters into the newly allocated string. +@var{size} bytes into the newly allocated string. If the length of @var{s} is more than @var{size}, then @code{strndup} -copies just the first @var{size} characters and adds a closing null -terminator. Otherwise all characters are copied and the string is +copies just the first @var{size} bytes and adds a closing null +byte. Otherwise all bytes are copied and the string is terminated. This function is different to @code{strncpy} in that it always @@ -665,7 +676,7 @@ terminates the destination string. @safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}} This function is like @code{strcpy}, except that it returns a pointer to the end of the string @var{to} (that is, the address of the terminating -null character @code{to + strlen (from)}) rather than the beginning. +null byte @code{to + strlen (from)}) rather than the beginning. For example, this program uses @code{stpcpy} to concatenate @samp{foo} and @samp{bar} to produce @samp{foobar}, which it then prints. @@ -688,7 +699,7 @@ declared in @file{string.h}. @safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}} This function is like @code{wcscpy}, except that it returns a pointer to the end of the string @var{wto} (that is, the address of the terminating -null character @code{wto + strlen (wfrom)}) rather than the beginning. +null wide character @code{wto + wcslen (wfrom)}) rather than the beginning. This function is not part of ISO or POSIX but was found useful while developing @theglibc{} itself. @@ -703,19 +714,19 @@ The behavior of @code{wcpcpy} is undefined if the strings overlap. @deftypefun {char *} stpncpy (char *restrict @var{to}, const char *restrict @var{from}, size_t @var{size}) @safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}} This function is similar to @code{stpcpy} but copies always exactly -@var{size} characters into @var{to}. +@var{size} bytes into @var{to}. If the length of @var{from} is more than @var{size}, then @code{stpncpy} -copies just the first @var{size} characters and returns a pointer to the -character directly following the one which was copied last. Note that in +copies just the first @var{size} bytes and returns a pointer to the +byte directly following the one which was copied last. Note that in this case there is no null terminator written into @var{to}. If the length of @var{from} is less than @var{size}, then @code{stpncpy} -copies all of @var{from}, followed by enough null characters to add up -to @var{size} characters in all. This behavior is rarely useful, but it +copies all of @var{from}, followed by enough null bytes to add up +to @var{size} bytes in all. This behavior is rarely useful, but it is implemented to be useful in contexts where this behavior of the @code{strncpy} is used. @code{stpncpy} returns a pointer to the -@emph{first} written null character. +@emph{first} written null byte. This function is not part of ISO or POSIX but was found useful while developing @theglibc{} itself. @@ -729,7 +740,7 @@ declared in @file{string.h}. @deftypefun {wchar_t *} wcpncpy (wchar_t *restrict @var{wto}, const wchar_t *restrict @var{wfrom}, size_t @var{size}) @safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}} This function is similar to @code{wcpcpy} but copies always exactly -@var{wsize} characters into @var{wto}. +@var{wsize} wide characters into @var{wto}. If the length of @var{wfrom} is more than @var{size}, then @code{wcpncpy} copies just the first @var{size} wide characters and @@ -738,11 +749,11 @@ non-null wide character which was copied last. Note that in this case there is no null terminator written into @var{wto}. If the length of @var{wfrom} is less than @var{size}, then @code{wcpncpy} -copies all of @var{wfrom}, followed by enough null characters to add up -to @var{size} characters in all. This behavior is rarely useful, but it +copies all of @var{wfrom}, followed by enough null wide characters to add up +to @var{size} wide characters in all. This behavior is rarely useful, but it is implemented to be useful in contexts where this behavior of the @code{wcsncpy} is used. @code{wcpncpy} returns a pointer to the -@emph{first} written null character. +@emph{first} written null wide character. This function is not part of ISO or POSIX but was found useful while developing @theglibc{} itself. @@ -800,9 +811,9 @@ parameter list in a function call. @deftypefun {char *} strcat (char *restrict @var{to}, const char *restrict @var{from}) @safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}} The @code{strcat} function is similar to @code{strcpy}, except that the -characters from @var{from} are concatenated or appended to the end of -@var{to}, instead of overwriting it. That is, the first character from -@var{from} overwrites the null character marking the end of @var{to}. +bytes from @var{from} are concatenated or appended to the end of +@var{to}, instead of overwriting it. That is, the first byte from +@var{from} overwrites the null byte marking the end of @var{to}. An equivalent definition for @code{strcat} would be: @@ -823,9 +834,9 @@ This function has undefined results if the strings overlap. @deftypefun {wchar_t *} wcscat (wchar_t *restrict @var{wto}, const wchar_t *restrict @var{wfrom}) @safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}} The @code{wcscat} function is similar to @code{wcscpy}, except that the -characters from @var{wfrom} are concatenated or appended to the end of -@var{wto}, instead of overwriting it. That is, the first character from -@var{wfrom} overwrites the null character marking the end of @var{wto}. +wide characters from @var{wfrom} are concatenated or appended to the end of +@var{wto}, instead of overwriting it. That is, the first wide character from |
