C string handling
C standard library |
---|
General topics |
Miscellaneous headers |
The C programming language has a set of functions implementing operations on strings (character strings and byte strings) in its standard library. Various operations, such as copying, concatenation, tokenization and searching are supported. For character strings, the standard library uses the convention that strings are null-terminated: a string of n characters is represented as an array of n + 1 elements, the last of which is a "NUL" character.
The only support for strings in the programming language proper is that the compiler translates quoted string constants into null-terminated strings.
Definitions
A string is a contiguous sequence of code units terminated by the first zero code (\0
, corresponding to the null character). In C, there are two types of strings: string, which is sometimes called byte string which uses the type char
s as code units (one char
is at least 8 bits), and wide string[1] which uses the type wchar_t
as code units.
A common misconception is that all char
arrays are strings, because string literals are converted to arrays during the compilation (or translation) phase.[2] It is important to remember that a string ends at the first zero code unit. An array or string literal that contains a zero before the last byte therefore contains a string, or possibly several strings, but is not itself a string.[3] Conversely, it is possible to create a char
array that is not null-terminated and is thus not a string: char
is often used as a small integer when needing to save memory.
The term pointer to a string is used in C to describe a pointer to the initial (lowest-addressed) byte of a string.[1] In C, pointers are used to pass strings to functions. Documentation (including this page) will often use the term string to mean pointer to a string.
The term length of a string is used in C to describe the number of bytes preceding the zero byte.[1] strlen
is a standardised function commonly used to determine the length of a string. A common mistake is to not realize that a string uses one more unit of memory than this length, in order to store the zero that ends the string.
Character encodings
Each string ends at the first occurrence of the zero code unit of the appropriate kind (char
or wchar_t
). Consequently, a byte string can contain non-NUL characters in ASCII or any ASCII extension, but not characters in encodings such as UTF-16 (even though a 16-bit code unit might be nonzero, its high or low byte might be zero). The encodings that can be stored in wide strings are defined by the width of wchar_t
. In most implementations, wchar_t
is at least 16 bits, and so all 16-bit encodings, such as UCS-2, can be stored. If wchar_t
is 32-bits, then 32-bit encodings, such as UTF-32, can be stored.
Variable-width encodings can be used in both byte strings and wide strings. String length and offsets are measured in bytes or wchar_t
, not in "characters", which can be confusing to beginning programmers. UTF-8 and Shift JIS are often used in C byte strings, while UTF-16 is often used in C wide strings when wchar_t
is 16 bits. Truncating strings with variable length characters using functions like strncpy
can produce invalid sequences at the end of the string. This can be unsafe if the truncated parts are interpreted by code that assumes the input is valid.
Support for Unicode literals such as char foo[512] = "φωωβαρ";
(UTF-8) or wchar_t foo[512] = L"φωωβαρ";
(UTF-16 or UTF-32) is implementation defined,[4] and may require that the source code be in the same encoding. Some compilers or editors will require entering all non-ASCII characters as \xNN
sequences for each byte of UTF-8, and/or \uNNNN
for each word of UTF-16.
Overview of functions
Most of the functions that operate on C strings are declared in the string.h
header (cstring
in C++), while functions that operate on C wide strings are declared in the wchar.h
header (cwchar
in C++). These headers also contain declarations of functions used for handling memory buffers; the name is thus something of a misnomer.
Functions declared in string.h
are extremely popular since, as a part of the C standard library, they are guaranteed to work on any platform which supports C. However, some security issues exist with these functions, such as potential buffer overflows when not used carefully and properly, causing the programmers to prefer safer and possibly less portable variants, out of which some popular ones are listed below. Some of these functions also violate const-correctness by accepting a const
string pointer and returning a non-const
pointer within the string. To correct this, some have been separated into two overloaded functions in the C++ version of the standard library.
In historical documentation the term "character" was often used instead of "byte" for C strings, which leads many to believe that these functions somehow do not work for UTF-8. In fact all lengths are defined as being in bytes and this is true in all implementations, and these functions work as well with UTF-8 as with single-byte encodings. The BSD documentation has been fixed to make this clear, but POSIX, Linux, and Windows documentation still uses "character" in many places where "byte" or "wchar_t" is the correct term.
Functions for handling memory buffers can process sequences of bytes that include null-byte as part of the data. Names of these functions typically start with mem
, as opposite to the str
prefix.
Constants and types
Name | Notes |
---|---|
NULL | Macro expanding to the null pointer constant; that is, a constant representing a pointer value which is guaranteed not to be a valid address of an object in memory. |
wchar_t | Type used for a code unit in wide strings, usually either 16 or 32 bits. No specific interpretation is specified for these code units; the C standard requires of a wchar_t only that it be at least as large as a char, not that it can actually store Unicode code points or UTF-16 code units.[5] |
wint_t | Integer type that can hold any value of a wchar_t as well as the value of the macro WEOF. This type is unchanged by integral promotions. Usually a 32 bit signed value. |
mbstate_t | Contains all the information about the conversion state required from one call to a function to the other. |
Functions
Byte string |
Wide string |
Description[note 1] | |
---|---|---|---|
String manipulation |
strcpy [6] |
wcscpy [7] |
Copies one string to another |
strncpy [8] |
wcsncpy [9] |
Writes exactly n bytes, copying from source or adding nulls | |
strcat [10] |
wcscat [11] |
Appends one string to another | |
strncat [12] |
wcsncat [13] |
Appends no more than n bytes from one string to another | |
strxfrm [14] |
wcsxfrm [15] |
Transforms a string according to the current locale | |
String examination |
strlen [16] |
wcslen [17] |
Returns the length of the string |
strcmp [18] |
wcscmp [19] |
Compares two strings (three-way comparison) | |
strncmp [20] |
wcsncmp [21] |
Compares a specific number of bytes in two strings | |
strcoll [22] |
wcscoll [23] |
Compares two strings according to the current locale | |
strchr [24] |
wcschr [25] |
Finds the first occurrence of a byte in a string | |
strrchr [26] |
wcsrchr [27] |
Finds the last occurrence of a byte in a string | |
strspn [28] |
wcsspn [29] |
Returns the number of initial bytes in a string that are in a second string | |
strcspn [30] |
wcscspn [31] |
Returns the number of initial bytes in a string that are not in a second string | |
strpbrk [32] |
wcspbrk [33] |
Finds in a string the first occurrence of a byte in a set | |
strstr [34] |
wcsstr [35] |
Finds the first occurrence of a substring in a string | |
strtok [36] |
wcstok [37] |
Splits a string into tokens | |
Miscellaneous | strerror [38] |
N/A | Returns a string containing a message derived from an error code |
Memory manipulation |
memset [39] |
wmemset [40] |
Fills a buffer with a repeated byte |
memcpy [41] |
wmemcpy [42] |
Copies one buffer to another | |
memmove [43] |
wmemmove [44] |
Copies one buffer to another, possibly overlapping, buffer | |
memcmp [45] |
wmemcmp [46] |
Compares two buffers (three-way comparison) | |
memchr [47] |
wmemchr [48] |
Finds the first occurrence of a byte in a buffer | |
|
Multibyte functions
Name | Description |
---|---|
mblen [49] |
Returns the number of bytes in the next multibyte character |
mbtowc [50] |
Converts the next multibyte character to a wide character |
wctomb [51] |
Converts a wide character to its multibyte representation |
mbstowcs [52] |
Converts a multibyte string to a wide string |
wcstombs [53] |
Converts a wide string to a multibyte string |
btowc [54] |
Convert a single-byte character to wide character, if possible |
wctob [55] |
Convert a wide character to a single-byte character, if possible |
mbsinit [56] |
Checks if a state object represents initial state |
mbrlen [57] |
Returns the number of bytes in the next multibyte character, given state |
mbrtowc [58] |
Converts the next multibyte character to a wide character, given state |
wcrtomb [59] |
Converts a wide character to its multibyte representation, given state |
mbsrtowcs [60] |
Converts a multibyte string to a wide string, given state |
wcsrtombs [61] |
Converts a wide string to a multibyte string, given state |
"state" is used by encodings that rely on history such as shift states. This is not needed by UTF-8 or UTF-32. UTF-16 uses them to keep track of surrogate pairs and to hide the fact that it actually is a multi-word encoding.
Numeric conversions
Byte string |
Wide string |
Description[note 1] |
---|---|---|
atof [62] |
N/A | converts a string to a floating-point value |
atoi atol atoll [63] |
N/A | converts a string to an integer (C99) |
strtof (C99)[64]strtod [65]strtold (C99)[66] |
wcstof (C99)[67]wcstod [68]wcstold (C99)[69] |
converts a string to a floating-point value |
strtol strtoll [70] |
wcstol wcstoll [71] |
converts a string to a signed integer |
strtoul strtoull [72] |
wcstoul wcstoull [73] |
converts a string to an unsigned integer |
|
The C standard library contains several functions for numeric conversions. The functions that deal with byte strings are defined in the stdlib.h
header (cstdlib
header in C++). The functions that deal with wide strings are defined in the wchar.h
header (cwchar
header in C++).
The strtoxxx
functions are not const-correct, since they accept a const
string pointer and return a non-const
pointer within the string. Also, since the Normative Amendment 1 (C95), atoxx
functions are considered subsumed by strtoxxx
functions, for which reason neither C95 nor any later standard provides wide-character versions of these functions.[74]
Popular extensions
Name | Platform | Description |
---|---|---|
memccpy [75] | SVID, POSIX | copies up to specified number of bytes between two memory areas, which must not overlap, stopping when a given byte is found. |
mempcpy [76] | GNU | a variant of memcpy returning a pointer to the byte following the last written byte |
strcasecmp [77] | POSIX, BSD | case-insensitive versions of strcmp |
strcat_s [78] | Windows | a variant of strcat that checks the destination buffer size before copying |
strcpy_s [79] | Windows | a variant of strcpy that checks the destination buffer size before copying |
strdup [80] | POSIX | allocates and duplicates a string |
strerror_r [81] | POSIX 1, GNU | a variant of strerror that is thread-safe. The GNU version is incompatible with the POSIX one. |
stricmp [82] | Windows | case-insensitive versions of strcmp |
strlcpy [83] | BSD, Solaris | a variant of strcpy that truncates the result to fit in the destination buffer[84] |
strlcat [83] | BSD, Solaris | a variant of strcat that truncates the result to fit in the destination buffer[84] |
strsignal [85] | POSIX:2008 | returns string representation of a signal code. Not thread safe. |
strtok_r [86] | POSIX | a variant of strtok that is thread-safe |
Replacements
Despite the well-established need to replace strcat
[10] and strcpy
[6] with functions that do not allow buffer overflows, no accepted standard has arisen. This is partly due to the mistaken belief by many C programmers that strncat
and strncpy
have the desired behavior; however, neither function was designed for this (they were intended to manipulate null-padded fixed-size string buffers, a data format less commonly used in modern software), and the behavior and arguments are non-intuitive and often written incorrectly even by expert programmers.[84]
The most popular[lower-alpha 1] replacement are the strlcat
and strlcpy
functions, which appeared in OpenBSD 2.4 in December, 1998.[84] These functions always write one NUL to the destination buffer, truncating the result if necessary, and return the size of buffer that would be needed, which allows detection of the truncation and provides a size for creating a new buffer that will not truncate. They have been criticized on the basis of allegedly being inefficient[87] and encouraging the use of C strings (instead of some superior alternative form of string).[88][89] Consequently, they have not been included in the GNU C library (used by software on Linux), although they are implemented in the C libraries for OpenBSD, FreeBSD, NetBSD, Solaris, OS X, and QNX. The lack of GNU C library support has not stopped various software authors from using it and bundling a replacement, among other SDL, GLib, ffmpeg, rsync, and even internally in the Linux kernel. Open source implementations for these functions are available.[90][91]
As part of its 2004 Security Development Lifecycle, Microsoft introduced a family of "secure" functions including strcpy_s
and strcat_s
(along with many others).[92] These functions were standardized with some minor changes as part of the optional C11 (Annex K) proposed by ISO/IEC WDTR 24731. Experience with these functions has shown significant problems with their adoption and errors in usage, so the removal of Annex K is proposed for the next revision of the C standard.[93] These functions perform various checks including whether the string is too long to fit in the buffer. If the checks fail, a user-specified "runtime-constraint handler" function is called,[94] which usually aborts the program.[95][96] Some functions perform destructive operations before calling the runtime-constraint handler; for example, strcat_s
sets the destination to the empty string,[97] which can make it difficult to recover from error conditions or debug them. These functions attracted considerable criticism because initially they were implemented only on Windows and at the same time warning messages started to be produced by Microsoft Visual C++ suggesting the programmers to use these functions instead of standard ones. This has been speculated by some to be an attempt by Microsoft to lock developers into its platform.[98] Although open-source implementations of these functions are available,[99] these functions are not present in common Unix C libraries.
If the string length is known, then memcpy
[41] or memmove
[43] can be more efficient than strcpy
, so some programs use them to optimize C string manipulation. They accept a buffer length as a parameter, so they can be employed to prevent buffer overflows in a manner similar to the aforementioned functions.
See also
- C syntax § Strings – source code syntax, including backslash escape sequences
- String functions
- Null-terminated string
Notes
- ↑ On GitHub, there are 7,813,206 uses of
strlcpy
, versus 38,644 uses ofstrcpy_s
(and 15,286,150 uses ofstrcpy
).
References
- 1 2 3 "The C99 standard draft + TC3" (PDF). §7.1.1p1. Retrieved 7 January 2011.
- ↑ "The C99 standard draft + TC3" (PDF). §6.4.5p7. Retrieved 7 January 2011.
- ↑ "The C99 standard draft + TC3" (PDF). Section 6.4.5 footnote 66. Retrieved 7 January 2011.
- ↑ "The C99 standard draft + TC3" (PDF). §5.1.1.2 Translation phases, p1. Retrieved 23 December 2011.
- ↑ Gillam, Richard (2003). Unicode Demystified: A Practical Programmer's Guide to the Encoding Standard. Addison-Wesley Professional. p. 714.
- 1 2 "strcpy - cppreference.com". En.cppreference.com. 2014-01-02. Retrieved 2014-03-06.
- ↑ "wcscpy - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strncpy - cppreference.com". En.cppreference.com. 2013-10-04. Retrieved 2014-03-06.
- ↑ "wcsncpy - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- 1 2 "strcat - cppreference.com". En.cppreference.com. 2013-10-08. Retrieved 2014-03-06.
- ↑ "wcscat - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strncat - cppreference.com". En.cppreference.com. 2013-07-01. Retrieved 2014-03-06.
- ↑ "wcsncat - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strxfrm - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wcsxfrm - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strlen - cppreference.com". En.cppreference.com. 2013-12-27. Retrieved 2014-03-06.
- ↑ "wcslen - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strcmp - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wcscmp - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strncmp - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wcsncmp - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strcoll - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wcscoll - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strchr - cppreference.com". En.cppreference.com. 2014-02-23. Retrieved 2014-03-06.
- ↑ "wcschr - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strrchr - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wcsrchr - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strspn - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wcsspn - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strcspn - cppreference.com". En.cppreference.com. 2013-05-31. Retrieved 2014-03-06.
- ↑ "wcscspn - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strpbrk - cppreference.com". En.cppreference.com. 2013-05-31. Retrieved 2014-03-06.
- ↑ "wcspbrk - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strstr - cppreference.com". En.cppreference.com. 2013-10-16. Retrieved 2014-03-06.
- ↑ "wcsstr - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strtok - cppreference.com". En.cppreference.com. 2013-09-03. Retrieved 2014-03-06.
- ↑ "wcstok - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strerror - cppreference.com". En.cppreference.com. 2013-05-31. Retrieved 2014-03-06.
- ↑ "memset - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wmemset - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- 1 2 "memcpy - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wmemcpy - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- 1 2 "memmove - cppreference.com". En.cppreference.com. 2014-01-25. Retrieved 2014-03-06.
- ↑ "wmemmove - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "memcmp - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wmemcmp - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "memchr - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wmemchr - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "mblen - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "mbtowc - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wctomb - cppreference.com". En.cppreference.com. 2014-02-04. Retrieved 2014-03-06.
- ↑ "mbstowcs - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wcstombs - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "btowc - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wctob - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "mbsinit - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "mbrlen - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "mbrtowc - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wcrtomb - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "mbsrtowcs - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wcsrtombs - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "atof - cppreference.com". En.cppreference.com. 2013-05-31. Retrieved 2014-03-06.
- ↑ "atoi, atol, atoll - cppreference.com". En.cppreference.com. 2014-01-18. Retrieved 2014-03-06.
- ↑ "strtof, strtod, strtold - cppreference.com". En.cppreference.com. 2014-02-04. Retrieved 2014-03-06.
- ↑ "strtof, strtod, strtold - cppreference.com". En.cppreference.com. 2014-02-04. Retrieved 2014-03-06.
- ↑ "strtof, strtod, strtold - cppreference.com". En.cppreference.com. 2014-02-04. Retrieved 2014-03-06.
- ↑ "wcstof, wcstod, wcstold - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wcstof, wcstod, wcstold - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "wcstof, wcstod, wcstold - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strtol, strtoll - cppreference.com". En.cppreference.com. 2014-02-04. Retrieved 2014-03-06.
- ↑ "wcstol, wcstoll - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ "strtoul, strtoull - cppreference.com". En.cppreference.com. 2014-02-04. Retrieved 2014-03-06.
- ↑ "wcstoul, wcstoull - cppreference.com". En.cppreference.com. Retrieved 2014-03-06.
- ↑ C99 Rationale, 7.20.1.1
- ↑ "memccpy". Pubs.opengroup.org. Retrieved 2014-03-06.
- ↑ "mempcpy(3) - Linux manual page". Kernel.org. Retrieved 2014-03-06.
- ↑ "strcasecmp(3) - Linux manual page". Kernel.org. Retrieved 2014-03-06.
- ↑ "strcat_s, wcscat_s, _mbscat_s". Msdn.microsoft.com. Retrieved 2014-03-06.
- ↑ "strcpy_s, wcscpy_s, _mbscpy_s". Msdn.microsoft.com. Retrieved 2014-03-06.
- ↑ "strdup". Pubs.opengroup.org. Retrieved 2014-03-06.
- ↑ "strerror(3) - Linux manual page". Kernel.org. Retrieved 2014-03-06.
- ↑ "String | stricmp()". C Programming Expert.com. Retrieved 2014-03-06.
- 1 2 "strlcpy, strlcat — size-bounded string copying and concatenation". OpenBSD. Retrieved 2016-05-26.
- 1 2 3 4 Todd C. Miller; Theo de Raadt (1999). "strlcpy and strlcat – consistent, safe, string copy and concatenation.". USENIX '99.
- ↑ "strsignal". Pubs.opengroup.org. Retrieved 2014-03-06.
- ↑ "strtok". Pubs.opengroup.org. Retrieved 2014-03-06.
- ↑ Miller, Damien (October 2005). "Secure Portability" (PDF). Retrieved 26 June 2016.
This [strlcpy and strlcat] API has been adopted by most modern operating systems and many standalone software packages [...]. The notable exception is the GNU standard C library, glibc, whose maintainer steadfastly refuses to include these improved APIs, labelling them “horribly inefficient BSD crap”, despite prior evidence that they are faster is most cases than the APIs they replace.
- ↑ libc-alpha mailing list, selected messages from 8 August 2000 thread: 53, 60, 61
- ↑ The ups and downs of strlcpy(); LWN.net
- ↑ Todd C. Miller. "strlcpy.c". BSD Cross Reference.
- ↑ Todd C. Miller. "strlcat.c". BSD Cross Reference.
- ↑ Lovell, Martyn. "Repel Attacks on Your Code with the Visual Studio 2005 Safe C and C++ Libraries". Retrieved 13 February 2015.
- ↑ "Field Experience With Annex K — Bounds Checking Interfaces". Retrieved 5 November 2015.
- ↑ "The C11 standard draft" (PDF). §K.3.1.4p2. Retrieved 13 February 2013.
- ↑ "The C11 standard draft" (PDF). §K.3.6.1.1p4. Retrieved 13 February 2013.
- ↑ "Parameter Validation".
- ↑ "The C11 standard draft" (PDF). §K.3.7.2.1p4. Retrieved 13 February 2013.
- ↑ Danny Kalev. "They're at it again". InformIT. Retrieved 10 November 2011.
- ↑ Safe C Library. "The Safe C Library provides bound checking memory and string functions per ISO/IEC TR24731". Sourceforge. Retrieved 6 March 2013.
External links
The Wikibook C Programming has a page on the topic of: C Programming/Strings |
- Fast memcpy in C, multiple C coding examples to target different types of CPU instruction architectures