This is The GNU C Library Reference Manual, for version 2.40.9000.
Copyright © 1993–2024 Free Software Foundation, Inc.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with the Invariant Sections being “Free Software Needs Free Documentation” and “GNU Lesser General Public License”, the Front-Cover texts being “A GNU Manual”, and with the Back-Cover Texts as in (a) below. A copy of the license is included in the section entitled "GNU Free Documentation License".
(a) The FSF’s Back-Cover Text is: “You have the freedom to copy and modify this GNU manual. Buying copies from the FSF supports it in developing GNU and promoting software freedom.”
malloc
malloc
malloc
malloc
-Related Functionsmalloc
gettext
family of functions
gettext
usesgettext
in GUI programsgettext
gettext
printf
inetd
Daemon
getopt
argp_parse
Functionargp_parse
argp_help
Functionargp_help
Functionsysconf
pathconf
The C language provides no built-in facilities for performing such common operations as input/output, memory management, string manipulation, and the like. Instead, these facilities are defined in a standard library, which you compile and link with your programs.
The GNU C Library, described in this document, defines all of the library functions that are specified by the ISO C standard, as well as additional features specific to POSIX and other derivatives of the Unix operating system, and extensions specific to GNU systems.
The purpose of this manual is to tell you how to use the facilities of the GNU C Library. We have mentioned which features belong to which standards to help you identify things that are potentially non-portable to other systems. But the emphasis in this manual is not on strict portability.
This manual is written with the assumption that you are at least somewhat familiar with the C programming language and basic programming concepts. Specifically, familiarity with ISO standard C (see ISO C), rather than “traditional” pre-ISO C dialects, is assumed.
The GNU C Library includes several header files, each of which provides definitions and declarations for a group of related facilities; this information is used by the C compiler when processing your program. For example, the header file stdio.h declares facilities for performing input and output, and the header file string.h declares string processing utilities. The organization of this manual generally follows the same division as the header files.
If you are reading this manual for the first time, you should read all of the introductory material and skim the remaining chapters. There are a lot of functions in the GNU C Library and it’s not realistic to expect that you will be able to remember exactly how to use each and every one of them. It’s more important to become generally familiar with the kinds of facilities that the library provides, so that when you are writing your programs you can recognize when to make use of library functions, and where in this manual you can find more specific information about them.
This section discusses the various standards and other sources that the GNU C Library is based upon. These sources include the ISO C and POSIX standards, and the System V and Berkeley Unix implementations.
The primary focus of this manual is to tell you how to make effective use of the GNU C Library facilities. But if you are concerned about making your programs compatible with these standards, or portable to operating systems other than GNU, this can affect how you use the library. This section gives you an overview of these standards, so that you will know what they are when they are mentioned in other parts of the manual.
See Summary of Library Facilities, for an alphabetical list of the functions and other symbols provided by the library. This list also states which standards each function or symbol comes from.
The GNU C Library is compatible with the C standard adopted by the American National Standards Institute (ANSI): American National Standard X3.159-1989—“ANSI C” and later by the International Standardization Organization (ISO): ISO/IEC 9899:1990, “Programming languages—C”. We here refer to the standard as ISO C since this is the more general standard in respect of ratification. The header files and library facilities that make up the GNU C Library are a superset of those specified by the ISO C standard.
If you are concerned about strict adherence to the ISO C standard, you should use the ‘-ansi’ option when you compile your programs with the GNU C compiler. This tells the compiler to define only ISO standard features from the library header files, unless you explicitly ask for additional features. See Feature Test Macros, for information on how to do this.
Being able to restrict the library to include only ISO C features is important because ISO C puts limitations on what names can be defined by the library implementation, and the GNU extensions don’t fit these limitations. See Reserved Names, for more information about these restrictions.
This manual does not attempt to give you complete details on the differences between ISO C and older dialects. It gives advice on how to write programs to work portably under multiple C dialects, but does not aim for completeness.
The GNU C Library is also compatible with the ISO POSIX family of standards, known more formally as the Portable Operating System Interface for Computer Environments (ISO/IEC 9945). They were also published as ANSI/IEEE Std 1003. POSIX is derived mostly from various versions of the Unix operating system.
The library facilities specified by the POSIX standards are a superset of those required by ISO C; POSIX specifies additional features for ISO C functions, as well as specifying new additional functions. In general, the additional requirements and functionality defined by the POSIX standards are aimed at providing lower-level support for a particular kind of operating system environment, rather than general programming language support which can run in many diverse operating system environments.
The GNU C Library implements all of the functions specified in ISO/IEC 9945-1:1996, the POSIX System Application Program Interface, commonly referred to as POSIX.1. The primary extensions to the ISO C facilities specified by this standard include file system interface primitives (see File System Interface), device-specific terminal control functions (see Low-Level Terminal Interface), and process control functions (see Processes).
Some facilities from ISO/IEC 9945-2:1993, the POSIX Shell and Utilities standard (POSIX.2) are also implemented in the GNU C Library. These include utilities for dealing with regular expressions and other pattern matching facilities (see Pattern Matching).
This manual documents various safety properties of GNU C Library functions, in lines that follow their prototypes and look like:
Preliminary: | MT-Safe | AS-Safe | AC-Safe |
The properties are assessed according to the criteria set forth in the POSIX standard for such safety contexts as Thread-, Async-Signal- and Async-Cancel- -Safety. Intuitive definitions of these properties, attempting to capture the meaning of the standard definitions, follow.
MT-Safe
or Thread-Safe functions are safe to call in the presence
of other threads. MT, in MT-Safe, stands for Multi Thread.
Being MT-Safe does not imply a function is atomic, nor that it uses any of the memory synchronization mechanisms POSIX exposes to users. It is even possible that calling MT-Safe functions in sequence does not yield an MT-Safe combination. For example, having a thread call two MT-Safe functions one right after the other does not guarantee behavior equivalent to atomic execution of a combination of both functions, since concurrent calls in other threads may interfere in a destructive way.
Whole-program optimizations that could inline functions across library interfaces may expose unsafe reordering, and so performing inlining across the GNU C Library interface is not recommended. The documented MT-Safety status is not guaranteed under whole-program optimization. However, functions defined in user-visible headers are designed to be safe for inlining.
AS-Safe
or Async-Signal-Safe functions are safe to call from
asynchronous signal handlers. AS, in AS-Safe, stands for Asynchronous
Signal.
Many functions that are AS-Safe may set errno
, or modify the
floating-point environment, because their doing so does not make them
unsuitable for use in signal handlers. However, programs could
misbehave should asynchronous signal handlers modify this thread-local
state, and the signal handling machinery cannot be counted on to
preserve it. Therefore, signal handlers that call functions that may
set errno
or modify the floating-point environment must
save their original values, and restore them before returning.
AC-Safe
or Async-Cancel-Safe functions are safe to call when
asynchronous cancellation is enabled. AC in AC-Safe stands for
Asynchronous Cancellation.
The POSIX standard defines only three functions to be AC-Safe, namely
pthread_cancel
, pthread_setcancelstate
, and
pthread_setcanceltype
. At present the GNU C Library provides no
guarantees beyond these three functions, but does document which
functions are presently AC-Safe. This documentation is provided for use
by the GNU C Library developers.
Just like signal handlers, cancellation cleanup routines must configure the floating point environment they require. The routines cannot assume a floating point environment, particularly when asynchronous cancellation is enabled. If the configuration of the floating point environment cannot be performed atomically then it is also possible that the environment encountered is internally inconsistent.
MT-Unsafe
, AS-Unsafe
, AC-Unsafe
functions are not
safe to call within the safety contexts described above. Calling them
within such contexts invokes undefined behavior.
Functions not explicitly documented as safe in a safety context should be regarded as Unsafe.
Preliminary
safety properties are documented, indicating these
properties may not be counted on in future releases of
the GNU C Library.
Such preliminary properties are the result of an assessment of the properties of our current implementation, rather than of what is mandated and permitted by current and future standards.
Although we strive to abide by the standards, in some cases our
implementation is safe even when the standard does not demand safety,
and in other cases our implementation does not meet the standard safety
requirements. The latter are most likely bugs; the former, when marked
as Preliminary
, should not be counted on: future standards may
require changes that are not compatible with the additional safety
properties afforded by the current implementation.
Furthermore, the POSIX standard does not offer a detailed definition of safety. We assume that, by “safe to call”, POSIX means that, as long as the program does not invoke undefined behavior, the “safe to call” function behaves as specified, and does not cause other functions to deviate from their specified behavior. We have chosen to use its loose definitions of safety, not because they are the best definitions to use, but because choosing them harmonizes this manual with POSIX.
Please keep in mind that these are preliminary definitions and annotations, and certain aspects of the definitions are still under discussion and might be subject to clarification or change.
Over time, we envision evolving the preliminary safety notes into stable
commitments, as stable as those of our interfaces. As we do, we will
remove the Preliminary
keyword from safety notes. As long as the
keyword remains, however, they are not to be regarded as a promise of
future behavior.
Other keywords that appear in safety notes are defined in subsequent sections.
Functions that are unsafe to call in certain contexts are annotated with keywords that document their features that make them unsafe to call. AS-Unsafe features in this section indicate the functions are never safe to call when asynchronous signals are enabled. AC-Unsafe features indicate they are never safe to call when asynchronous cancellation is enabled. There are no MT-Unsafe marks in this section.
lock
Functions marked with lock
as an AS-Unsafe feature may be
interrupted by a signal while holding a non-recursive lock. If the
signal handler calls another such function that takes the same lock, the
result is a deadlock.
Functions annotated with lock
as an AC-Unsafe feature may, if
cancelled asynchronously, fail to release a lock that would have been
released if their execution had not been interrupted by asynchronous
thread cancellation. Once a lock is left taken, attempts to take that
lock will block indefinitely.
corrupt
Functions marked with corrupt
as an AS-Unsafe feature may corrupt
data structures and misbehave when they interrupt, or are interrupted
by, another such function. Unlike functions marked with lock
,
these take recursive locks to avoid MT-Safety problems, but this is not
enough to stop a signal handler from observing a partially-updated data
structure. Further corruption may arise from the interrupted function’s
failure to notice updates made by signal handlers.
Functions marked with corrupt
as an AC-Unsafe feature may leave
data structures in a corrupt, partially updated state. Subsequent uses
of the data structure may misbehave.
heap
Functions marked with heap
may call heap memory management
functions from the malloc
/free
family of functions and are
only as safe as those functions. This note is thus equivalent to:
| AS-Unsafe lock | AC-Unsafe lock fd mem |
dlopen
Functions marked with dlopen
use the dynamic loader to load
shared libraries into the current execution image. This involves
opening files, mapping them into memory, allocating additional memory,
resolving symbols, applying relocations and more, all of this while
holding internal dynamic loader locks.
The locks are enough for these functions to be AS- and AC-Unsafe, but
other issues may arise. At present this is a placeholder for all
potential safety issues raised by dlopen
.
plugin
Functions annotated with plugin
may run code from plugins that
may be external to the GNU C Library. Such plugin functions are assumed to be
MT-Safe, AS-Unsafe and AC-Unsafe. Examples of such plugins are stack
unwinding libraries, name service switch (NSS) and character set
conversion (iconv) back-ends.
Although the plugins mentioned as examples are all brought in by means
of dlopen, the plugin
keyword does not imply any direct
involvement of the dynamic loader or the libdl
interfaces, those
are covered by dlopen
. For example, if one function loads a
module and finds the addresses of some of its functions, while another
just calls those already-resolved functions, the former will be marked
with dlopen
, whereas the latter will get the plugin
. When
a single function takes all of these actions, then it gets both marks.
i18n
Functions marked with i18n
may call internationalization
functions of the gettext
family and will be only as safe as those
functions. This note is thus equivalent to:
| MT-Safe env | AS-Unsafe corrupt heap dlopen | AC-Unsafe corrupt |
timer
Functions marked with timer
use the alarm
function or
similar to set a time-out for a system call or a long-running operation.
In a multi-threaded program, there is a risk that the time-out signal
will be delivered to a different thread, thus failing to interrupt the
intended thread. Besides being MT-Unsafe, such functions are always
AS-Unsafe, because calling them in signal handlers may interfere with
timers set in the interrupted code, and AC-Unsafe, because there is no
safe way to guarantee an earlier timer will be reset in case of
asynchronous cancellation.
For some features that make functions unsafe to call in certain contexts, there are known ways to avoid the safety problem other than refraining from calling the function altogether. The keywords that follow refer to such features, and each of their definitions indicate how the whole program needs to be constrained in order to remove the safety problem indicated by the keyword. Only when all the reasons that make a function unsafe are observed and addressed, by applying the documented constraints, does the function become safe to call in a context.
init
Functions marked with init
as an MT-Unsafe feature perform
MT-Unsafe initialization when they are first called.
Calling such a function at least once in single-threaded mode removes this specific cause for the function to be regarded as MT-Unsafe. If no other cause for that remains, the function can then be safely called after other threads are started.
Functions marked with init
as an AS- or AC-Unsafe feature use the
internal libc_once
machinery or similar to initialize internal
data structures.
If a signal handler interrupts such an initializer, and calls any
function that also performs libc_once
initialization, it will
deadlock if the thread library has been loaded.
Furthermore, if an initializer is partially complete before it is canceled or interrupted by a signal whose handler requires the same initialization, some or all of the initialization may be performed more than once, leaking resources or even resulting in corrupt internal data.
Applications that need to call functions marked with init
as an
AS- or AC-Unsafe feature should ensure the initialization is performed
before configuring signal handlers or enabling cancellation, so that the
AS- and AC-Safety issues related with libc_once
do not arise.
race
Functions annotated with race
as an MT-Safety issue operate on
objects in ways that may cause data races or similar forms of
destructive interference out of concurrent execution. In some cases,
the objects are passed to the functions by users; in others, they are
used by the functions to return values to users; in others, they are not
even exposed to users.
We consider access to objects passed as (indirect) arguments to
functions to be data race free. The assurance of data race free objects
is the caller’s responsibility. We will not mark a function as
MT-Unsafe or AS-Unsafe if it misbehaves when users fail to take the
measures required by POSIX to avoid data races when dealing with such
objects. As a general rule, if a function is documented as reading from
an object passed (by reference) to it, or modifying it, users ought to
use memory synchronization primitives to avoid data races just as they
would should they perform the accesses themselves rather than by calling
the library function. FILE
streams are the exception to the
general rule, in that POSIX mandates the library to guard against data
races in many functions that manipulate objects of this specific opaque
type. We regard this as a convenience provided to users, rather than as
a general requirement whose expectations should extend to other types.
In order to remind users that guarding certain arguments is their
responsibility, we will annotate functions that take objects of certain
types as arguments. We draw the line for objects passed by users as
follows: objects whose types are exposed to users, and that users are
expected to access directly, such as memory buffers, strings, and
various user-visible struct
types, do not give reason for
functions to be annotated with race
. It would be noisy and
redundant with the general requirement, and not many would be surprised
by the library’s lack of internal guards when accessing objects that can
be accessed directly by users.
As for objects that are opaque or opaque-like, in that they are to be
manipulated only by passing them to library functions (e.g.,
FILE
, DIR
, obstack
, iconv_t
), there might be
additional expectations as to internal coordination of access by the
library. We will annotate, with race
followed by a colon and the
argument name, functions that take such objects but that do not take
care of synchronizing access to them by default. For example,
FILE
stream unlocked
functions will be annotated, but
those that perform implicit locking on FILE
streams by default
will not, even though the implicit locking may be disabled on a
per-stream basis.
In either case, we will not regard as MT-Unsafe functions that may access user-supplied objects in unsafe ways should users fail to ensure the accesses are well defined. The notion prevails that users are expected to safeguard against data races any user-supplied objects that the library accesses on their behalf.
This user responsibility does not apply, however, to objects controlled
by the library itself, such as internal objects and static buffers used
to return values from certain calls. When the library doesn’t guard
them against concurrent uses, these cases are regarded as MT-Unsafe and
AS-Unsafe (although the race
mark under AS-Unsafe will be omitted
as redundant with the one under MT-Unsafe). As in the case of
user-exposed objects, the mark may be followed by a colon and an
identifier. The identifier groups all functions that operate on a
certain unguarded object; users may avoid the MT-Safety issues related
with unguarded concurrent access to such internal objects by creating a
non-recursive mutex related with the identifier, and always holding the
mutex when calling any function marked as racy on that identifier, as
they would have to should the identifier be an object under user
control. The non-recursive mutex avoids the MT-Safety issue, but it
trades one AS-Safety issue for another, so use in asynchronous signals
remains undefined.
When the identifier relates to a static buffer used to hold return
values, the mutex must be held for as long as the buffer remains in use
by the caller. Many functions that return pointers to static buffers
offer reentrant variants that store return values in caller-supplied
buffers instead. In some cases, such as tmpname
, the variant is
chosen not by calling an alternate entry point, but by passing a
non-NULL
pointer to the buffer in which the returned values are
to be stored. These variants are generally preferable in multi-threaded
programs, although some of them are not MT-Safe because of other
internal buffers, also documented with race
notes.
const
Functions marked with const
as an MT-Safety issue non-atomically
modify internal objects that are better regarded as constant, because a
substantial portion of the GNU C Library accesses them without
synchronization. Unlike race
, that causes both readers and
writers of internal objects to be regarded as MT-Unsafe and AS-Unsafe,
this mark is applied to writers only. Writers remain equally MT- and
AS-Unsafe to call, but the then-mandatory constness of objects they
modify enables readers to be regarded as MT-Safe and AS-Safe (as long as
no other reasons for them to be unsafe remain), since the lack of
synchronization is not a problem when the objects are effectively
constant.
The identifier that follows the const
mark will appear by itself
as a safety note in readers. Programs that wish to work around this
safety issue, so as to call writers, may use a non-recursve
rwlock
associated with the identifier, and guard all calls
to functions marked with const
followed by the identifier with a
write lock, and all calls to functions marked with the identifier
by itself with a read lock. The non-recursive locking removes the
MT-Safety problem, but it trades one AS-Safety problem for another, so
use in asynchronous signals remains undefined.
sig
Functions marked with sig
as a MT-Safety issue (that implies an
identical AS-Safety issue, omitted for brevity) may temporarily install
a signal handler for internal purposes, which may interfere with other
uses of the signal, identified after a colon.
This safety problem can be worked around by ensuring that no other uses of the signal will take place for the duration of the call. Holding a non-recursive mutex while calling all functions that use the same temporary signal; blocking that signal before the call and resetting its handler afterwards is recommended.
There is no safe way to guarantee the original signal handler is restored in case of asynchronous cancellation, therefore so-marked functions are also AC-Unsafe.
Besides the measures recommended to work around the MT- and AS-Safety problem, in order to avert the cancellation problem, disabling asynchronous cancellation and installing a cleanup handler to restore the signal to the desired state and to release the mutex are recommended.
term
Functions marked with term
as an MT-Safety issue may change the
terminal settings in the recommended way, namely: call tcgetattr
,
modify some flags, and then call tcsetattr
; this creates a window
in which changes made by other threads are lost. Thus, functions marked
with term
are MT-Unsafe. The same window enables changes made by
asynchronous signals to be lost. These functions are also AS-Unsafe,
but the corresponding mark is omitted as redundant.
It is thus advisable for applications using the terminal to avoid
concurrent and reentrant interactions with it, by not using it in signal
handlers or blocking signals that might use it, and holding a lock while
calling these functions and interacting with the terminal. This lock
should also be used for mutual exclusion with functions marked with
race:tcattr(fd)
, where fd is a file descriptor for
the controlling terminal. The caller may use a single mutex for
simplicity, or use one mutex per terminal, even if referenced by
different file descriptors.
Functions marked with term
as an AC-Safety issue are supposed to
restore terminal settings to their original state, after temporarily
changing them, but they may fail to do so if cancelled.
Besides the measures recommended to work around the MT- and AS-Safety problem, in order to avert the cancellation problem, disabling asynchronous cancellation and installing a cleanup handler to restore the terminal settings to the original state and to release the mutex are recommended.
Additional keywords may be attached to functions, indicating features that do not make a function unsafe to call, but that may need to be taken into account in certain classes of programs:
locale
Functions annotated with locale
as an MT-Safety issue read from
the locale object without any form of synchronization. Functions
annotated with locale
called concurrently with locale changes may
behave in ways that do not correspond to any of the locales active
during their execution, but an unpredictable mix thereof.
We do not mark these functions as MT- or AS-Unsafe, however, because
functions that modify the locale object are marked with
const:locale
and regarded as unsafe. Being unsafe, the latter
are not to be called when multiple threads are running or asynchronous
signals are enabled, and so the locale can be considered effectively
constant in these contexts, which makes the former safe.
env
Functions marked with env
as an MT-Safety issue access the
environment with getenv
or similar, without any guards to ensure
safety in the presence of concurrent modifications.
We do not mark these functions as MT- or AS-Unsafe, however, because
functions that modify the environment are all marked with
const:env
and regarded as unsafe. Being unsafe, the latter are
not to be called when multiple threads are running or asynchronous
signals are enabled, and so the environment can be considered
effectively constant in these contexts, which makes the former safe.
hostid
The function marked with hostid
as an MT-Safety issue reads from
the system-wide data structures that hold the “host ID” of the
machine. These data structures cannot generally be modified atomically.
Since it is expected that the “host ID” will not normally change, the
function that reads from it (gethostid
) is regarded as safe,
whereas the function that modifies it (sethostid
) is marked with
const:hostid
, indicating it may require special
care if it is to be called. In this specific case, the special care
amounts to system-wide (not merely intra-process) coordination.
sigintr
Functions marked with sigintr
as an MT-Safety issue access the
_sigintr
internal data structure without any guards to ensure
safety in the presence of concurrent modifications.
We do not mark these functions as MT- or AS-Unsafe, however, because
functions that modify the this data structure are all marked with
const:sigintr
and regarded as unsafe. Being unsafe, the latter
are not to be called when multiple threads are running or asynchronous
signals are enabled, and so the data structure can be considered
effectively constant in these contexts, which makes the former safe.
fd
Functions annotated with fd
as an AC-Safety issue may leak file
descriptors if asynchronous thread cancellation interrupts their
execution.
Functions that allocate or deallocate file descriptors will generally be marked as such. Even if they attempted to protect the file descriptor allocation and deallocation with cleanup regions, allocating a new descriptor and storing its number where the cleanup region could release it cannot be performed as a single atomic operation. Similarly, releasing the descriptor and taking it out of the data structure normally responsible for releasing it cannot be performed atomically. There will always be a window in which the descriptor cannot be released because it was not stored in the cleanup handler argument yet, or it was already taken out before releasing it. It cannot be taken out after release: an open descriptor could mean either that the descriptor still has to be closed, or that it already did so but the descriptor was reallocated by another thread or signal handler.
Such leaks could be internally avoided, with some performance penalty, by temporarily disabling asynchronous thread cancellation. However, since callers of allocation or deallocation functions would have to do this themselves, to avoid the same sort of leak in their own layer, it makes more sense for the library to assume they are taking care of it than to impose a performance penalty that is redundant when the problem is solved in upper layers, and insufficient when it is not.
This remark by itself does not cause a function to be regarded as AC-Unsafe. However, cumulative effects of such leaks may pose a problem for some programs. If this is the case, suspending asynchronous cancellation for the duration of calls to such functions is recommended.
mem
Functions annotated with mem
as an AC-Safety issue may leak
memory if asynchronous thread cancellation interrupts their execution.
The problem is similar to that of file descriptors: there is no atomic interface to allocate memory and store its address in the argument to a cleanup handler, or to release it and remove its address from that argument, without at least temporarily disabling asynchronous cancellation, which these functions do not do.
This remark does not by itself cause a function to be regarded as generally AC-Unsafe. However, cumulative effects of such leaks may be severe enough for some programs that disabling asynchronous cancellation for the duration of calls to such functions may be required.
cwd
Functions marked with cwd
as an MT-Safety issue may temporarily
change the current working directory during their execution, which may
cause relative pathnames to be resolved in unexpected ways in other
threads or within asynchronous signal or cancellation handlers.
This is not enough of a reason to mark so-marked functions as MT- or
AS-Unsafe, but when this behavior is optional (e.g., nftw
with
FTW_CHDIR
), avoiding the option may be a good alternative to
using full pathnames or file descriptor-relative (e.g. openat
)
system calls.
!posix
This remark, as an MT-, AS- or AC-Safety note to a function, indicates the safety status of the function is known to differ from the specified status in the POSIX standard. For example, POSIX does not require a function to be Safe, but our implementation is, or vice-versa.
For the time being, the absence of this remark does not imply the safety properties we documented are identical to those mandated by POSIX for the corresponding functions.
:identifier
Annotations may sometimes be followed by identifiers, intended to group
several functions that e.g. access the data structures in an unsafe way,
as in race
and const
, or to provide more specific
information, such as naming a signal in a function marked with
sig
. It is envisioned that it may be applied to lock
and
corrupt
as well in the future.
In most cases, the identifier will name a set of functions, but it may
name global objects or function arguments, or identifiable properties or
logical components associated with them, with a notation such as
e.g. :buf(arg)
to denote a buffer associated with the argument
arg, or :tcattr(fd)
to denote the terminal attributes of a
file descriptor fd.
The most common use for identifiers is to provide logical groups of functions and arguments that need to be protected by the same synchronization primitive in order to ensure safe operation in a given context.
/condition
Some safety annotations may be conditional, in that they only apply if a
boolean expression involving arguments, global variables or even the
underlying kernel evaluates to true. Such conditions as
/hurd
or /!linux!bsd
indicate the preceding marker only
applies when the underlying kernel is the HURD, or when it is neither
Linux nor a BSD kernel, respectively. /!ps
and
/one_per_line
indicate the preceding marker only applies when
argument ps is NULL, or global variable one_per_line is
nonzero.
When all marks that render a function unsafe are adorned with such conditions, and none of the named conditions hold, then the function can be regarded as safe.
The GNU C Library defines facilities from some versions of Unix which are not formally standardized, specifically from the 4.2 BSD, 4.3 BSD, and 4.4 BSD Unix systems (also known as Berkeley Unix) and from SunOS (a popular 4.2 BSD derivative that includes some Unix System V functionality). These systems support most of the ISO C and POSIX facilities, and 4.4 BSD and newer releases of SunOS in fact support them all.
The BSD facilities include symbolic links (see Symbolic Links), the
select
function (see Waiting for Input or Output), the BSD signal
functions (see BSD Signal Handling), and sockets (see Sockets).
The System V Interface Description (SVID) is a document describing the AT&T Unix System V operating system. It is to some extent a superset of the POSIX standard (see POSIX (The Portable Operating System Interface)).
The GNU C Library defines most of the facilities required by the SVID that are not also required by the ISO C or POSIX standards, for compatibility with System V Unix and other Unix systems (such as SunOS) which include these facilities. However, many of the more obscure and less generally useful facilities required by the SVID are not included. (In fact, Unix System V itself does not provide them all.)
The supported facilities from System V include the methods for
inter-process communication and shared memory, the hsearch
and
drand48
families of functions, fmtmsg
and several of the
mathematical functions.
The X/Open Portability Guide, published by the X/Open Company, Ltd., is a more general standard than POSIX. X/Open owns the Unix copyright and the XPG specifies the requirements for systems which are intended to be a Unix system.
The GNU C Library complies to the X/Open Portability Guide, Issue 4.2, with all extensions common to XSI (X/Open System Interface) compliant systems and also all X/Open UNIX extensions.
The additions on top of POSIX are mainly derived from functionality available in System V and BSD systems. Some of the really bad mistakes in System V systems were corrected, though. Since fulfilling the XPG standard with the Unix extensions is a precondition for getting the Unix brand chances are good that the functionality is available on commercial systems.
The GNU C Library includes by reference the Linux man-pages 6.9.1 documentation to document the listed syscalls for the Linux kernel. For reference purposes only the latest Linux man-pages Project documentation can be accessed from the Linux kernel website. Where the syscall has more specific documentation in this manual that more specific documentation is considered authoritative.
Additional details on the Linux system call interface can be found in See System Calls.
This section describes some of the practical issues involved in using the GNU C Library.
Libraries for use by C programs really consist of two parts: header files that define types and macros and declare variables and functions; and the actual library or archive that contains the definitions of the variables and functions.
(Recall that in C, a declaration merely provides information that a function or variable exists and gives its type. For a function declaration, information about the types of its arguments might be provided as well. The purpose of declarations is to allow the compiler to correctly process references to the declared variables and functions. A definition, on the other hand, actually allocates storage for a variable or says what a function does.)
In order to use the facilities in the GNU C Library, you should be sure that your program source files include the appropriate header files. This is so that the compiler has declarations of these facilities available and can correctly process references to them. Once your program has been compiled, the linker resolves these references to the actual definitions provided in the archive file.
Header files are included into a program source file by the ‘#include’ preprocessor directive. The C language supports two forms of this directive; the first,
#include "header"
is typically used to include a header file header that you write yourself; this would contain definitions and declarations describing the interfaces between the different parts of your particular application. By contrast,
#include <file.h>
is typically used to include a header file file.h that contains definitions and declarations for a standard library. This file would normally be installed in a standard place by your system administrator. You should use this second form for the C library header files.
Typically, ‘#include’ directives are placed at the top of the C source file, before any other code. If you begin your source files with some comments explaining what the code in the file does (a good idea), put the ‘#include’ directives immediately afterwards, following the feature test macro definition (see Feature Test Macros).
For more information about the use of header files and ‘#include’ directives, see Header Files in The GNU C Preprocessor Manual.
The GNU C Library provides several header files, each of which contains the type and macro definitions and variable and function declarations for a group of related facilities. This means that your programs may need to include several header files, depending on exactly which facilities you are using.
Some library header files include other library header files automatically. However, as a matter of programming style, you should not rely on this; it is better to explicitly include all the header files required for the library facilities you are using. The GNU C Library header files have been written in such a way that it doesn’t matter if a header file is accidentally included more than once; including a header file a second time has no effect. Likewise, if your program needs to include multiple header files, the order in which they are included doesn’t matter.
Compatibility Note: Inclusion of standard header files in any order and any number of times works in any ISO C implementation. However, this has traditionally not been the case in many older C implementations.
Strictly speaking, you don’t have to include a header file to use a function it declares; you could declare the function explicitly yourself, according to the specifications in this manual. But it is usually better to include the header file because it may define types and macros that are not otherwise available and because it may define more efficient macro replacements for some functions. It is also a sure way to have the correct declaration.
If we describe something as a function in this manual, it may have a macro definition as well. This normally has no effect on how your program runs—the macro definition does the same thing as the function would. In particular, macro equivalents for library functions evaluate arguments exactly once, in the same way that a function call would. The main reason for these macro definitions is that sometimes they can produce an inline expansion that is considerably faster than an actual function call.
Taking the address of a library function works even if it is also defined as a macro. This is because, in this context, the name of the function isn’t followed by the left parenthesis that is syntactically necessary to recognize a macro call.
You might occasionally want to avoid using the macro definition of a function—perhaps to make your program easier to debug. There are two ways you can do this:
For example, suppose the header file stdlib.h declares a function
named abs
with
extern int abs (int);
and also provides a macro definition for abs
. Then, in:
#include <stdlib.h> int f (int *i) { return abs (++*i); }
the reference to abs
might refer to either a macro or a function.
On the other hand, in each of the following examples the reference is
to a function and not a macro.
#include <stdlib.h> int g (int *i) { return (abs) (++*i); } #undef abs int h (int *i) { return abs (++*i); }
Since macro definitions that double for a function behave in exactly the same way as the actual function version, there is usually no need for any of these methods. In fact, removing macro definitions usually just makes your program slower.
The names of all library types, macros, variables and functions that come from the ISO C standard are reserved unconditionally; your program may not redefine these names. All other library names are reserved if your program explicitly includes the header file that defines or declares them. There are several reasons for these restrictions:
exit
to do something completely different from
what the standard exit
function does, for example. Preventing
this situation helps to make your programs easier to understand and
contributes to modularity and maintainability.
In addition to the names documented in this manual, reserved names include all external identifiers (global functions and variables) that begin with an underscore (‘_’) and all identifiers regardless of use that begin with either two underscores or an underscore followed by a capital letter are reserved names. This is so that the library and header files can define functions, variables, and macros for internal purposes without risk of conflict with names in user programs.
Some additional classes of identifier names are reserved for future extensions to the C language or the POSIX.1 environment. While using these names for your own purposes right now might not cause a problem, they do raise the possibility of conflict with future versions of the C or POSIX standards, so you should avoid these names.
float
and long double
arguments,
respectively.
In addition, some individual header files reserve names beyond those that they actually define. You only need to worry about these restrictions if your program includes that particular header file.
The exact set of features available when you compile a source file is controlled by which feature test macros you define.
If you compile your programs using ‘gcc -ansi’, you get only the ISO C library features, unless you explicitly request additional features by defining one or more of the feature macros. See GNU CC Command Options in The GNU CC Manual, for more information about GCC options.
You should define these macros by using ‘#define’ preprocessor
directives at the top of your source code files. These directives
must come before any #include
of a system header file. It
is best to make them the very first thing in the file, preceded only by
comments. You could also use the ‘-D’ option to GCC, but it’s
better if you make the source files indicate their own meaning in a
self-contained way.
This system exists to allow the library to conform to multiple standards.
Although the different standards are often described as supersets of each
other, they are usually incompatible because larger standards require
functions with names that smaller ones reserve to the user program. This
is not mere pedantry — it has been a problem in practice. For instance,
some non-GNU programs define functions named getline
that have
nothing to do with this library’s getline
. They would not be
compilable if all features were enabled indiscriminately.
This should not be used to verify that a program conforms to a limited standard. It is insufficient for this purpose, as it will not protect you from including header files outside the standard, or relying on semantics undefined within the standard.
If you define this macro, then the functionality from the POSIX.1 standard (IEEE Standard 1003.1) is available, as well as all of the ISO C facilities.
The state of _POSIX_SOURCE
is irrelevant if you define the
macro _POSIX_C_SOURCE
to a positive integer.
Define this macro to a positive integer to control which POSIX functionality is made available. The greater the value of this macro, the more functionality is made available.
If you define this macro to a value greater than or equal to 1
,
then the functionality from the 1990 edition of the POSIX.1 standard
(IEEE Standard 1003.1-1990) is made available.
If you define this macro to a value greater than or equal to 2
,
then the functionality from the 1992 edition of the POSIX.2 standard
(IEEE Standard 1003.2-1992) is made available.
If you define this macro to a value greater than or equal to 199309L
,
then the functionality from the 1993 edition of the POSIX.1b standard
(IEEE Standard 1003.1b-1993) is made available.
If you define this macro to a value greater than or equal to
199506L
, then the functionality from the 1995 edition of the
POSIX.1c standard (IEEE Standard 1003.1c-1995) is made available.
If you define this macro to a value greater than or equal to
200112L
, then the functionality from the 2001 edition of the
POSIX standard (IEEE Standard 1003.1-2001) is made available.
If you define this macro to a value greater than or equal to
200809L
, then the functionality from the 2008 edition of the
POSIX standard (IEEE Standard 1003.1-2008) is made available.
Greater values for _POSIX_C_SOURCE
will enable future extensions.
The POSIX standards process will define these values as necessary, and
the GNU C Library should support them some time after they become standardized.
The 1996 edition of POSIX.1 (ISO/IEC 9945-1: 1996) states that
if you define _POSIX_C_SOURCE
to a value greater than
or equal to 199506L
, then the functionality from the 1996
edition is made available. In general, in the GNU C Library, bugfixes to
the standards are included when specifying the base version; e.g.,
POSIX.1-2004 will always be included with a value of 200112L
.
If you define this macro, functionality described in the X/Open
Portability Guide is included. This is a superset of the POSIX.1 and
POSIX.2 functionality and in fact _POSIX_SOURCE
and
_POSIX_C_SOURCE
are automatically defined.
As the unification of all Unices, functionality only available in BSD and SVID is also included.
If the macro _XOPEN_SOURCE_EXTENDED
is also defined, even more
functionality is available. The extra functions will make all functions
available which are necessary for the X/Open Unix brand.
If the macro _XOPEN_SOURCE
has the value 500 this includes
all functionality described so far plus some new definitions from the
Single Unix Specification, version 2. The value 600
(corresponding to the sixth revision) includes definitions from SUSv3,
and using 700 (the seventh revision) includes definitions from
SUSv4.
If this macro is defined some extra functions are available which
rectify a few shortcomings in all previous standards. Specifically,
the functions fseeko
and ftello
are available. Without
these functions the difference between the ISO C interface
(fseek
, ftell
) and the low-level POSIX interface
(lseek
) would lead to problems.
This macro was introduced as part of the Large File Support extension (LFS).
If you define this macro an additional set of functions is made available which enables 32 bit systems to use files of sizes beyond the usual limit of 2GB. This interface is not available if the system does not support files that large. On systems where the natural file size limit is greater than 2GB (i.e., on 64 bit systems) the new functions are identical to the replaced functions.
The new functionality is made available by a new set of types and
functions which replace the existing ones. The names of these new objects
contain 64
to indicate the intention, e.g., off_t
vs. off64_t
and fseeko
vs. fseeko64
.
This macro was introduced as part of the Large File Support extension
(LFS). It is a transition interface for the period when 64 bit
offsets are not generally used (see _FILE_OFFSET_BITS
).
This macro determines which file system interface shall be used, one
replacing the other. Whereas _LARGEFILE64_SOURCE
makes the 64 bit interface available as an additional interface,
_FILE_OFFSET_BITS
allows the 64 bit interface to
replace the old interface.
If _FILE_OFFSET_BITS
is defined to the
value 32
, the 32 bit interface is used and
types like off_t
have a size of 32 bits on 32 bit
systems.
If the macro is defined to the value 64
, the large file interface
replaces the old interface. I.e., the functions are not made available
under different names (as they are with _LARGEFILE64_SOURCE
).
Instead the old function names now reference the new functions, e.g., a
call to fseeko
now indeed calls fseeko64
.
If the macro is not defined it currently defaults to 32
, but
this default is planned to change due to a need to update
time_t
for Y2038 safety, and applications should not rely on
the default.
This macro should only be selected if the system provides mechanisms for
handling large files. On 64 bit systems this macro has no effect
since the *64
functions are identical to the normal functions.
This macro was introduced as part of the Large File Support extension (LFS).
Define this macro to control the bit size of time_t
, and therefore
the bit size of all time_t
-derived types and the prototypes of all
related functions.
_TIME_BITS
is undefined, the bit size of time_t
is
architecture dependent. Currently it defaults to 64 bits on most
architectures. Although it defaults to 32 bits on some traditional
architectures (i686, ARM), this is planned to change and applications
should not rely on this.
_TIME_BITS
is defined to be 64, time_t
is defined
to be a 64-bit integer. On platforms where time_t
was
traditionally 32 bits, calls to proper syscalls depend on the
Linux kernel version on which the system is running. For Linux kernel
version above 5.1 syscalls supporting 64-bit time are used. Otherwise,
a fallback code is used with legacy (i.e. 32-bit) syscalls.
On such platforms, the GNU C Library will also define __USE_TIME64_REDIRECTS
to indicate whether the declarations are expanded to different ones
(either by redefiniding the symbol name or using symbol aliais).
For instance, if the symbol clock_gettime
expands to
__glock_gettime64
.
_TIME_BITS
is defined to be 32, time_t
is defined to
be a 32-bit integer where that is supported. This is not recommended,
as 32-bit time_t
stops working in the year 2038.
_TIME_BITS=64
can be defined only when
_FILE_OFFSET_BITS=64
is also defined.
By using this macro certain ports gain support for 64-bit time and as a result become immune to the Y2038 problem.
If this macro is defined, features from ISO C99 are included. Since these features are included by default, this macro is mostly relevant when the compiler uses an earlier language version.
If this macro is defined, ISO C11 extensions to ISO C99 are included.
If this macro is defined, ISO C23 extensions to ISO C11 are included.
Only some features from this draft standard are supported by
the GNU C Library. The older name _ISOC2X_SOURCE
is also supported.
If this macro is defined, ISO C2Y extensions to ISO C23 are included. Only some features from this draft standard are supported by the GNU C Library.
If you define this macro to the value 1
, features from ISO/IEC
TR 24731-2:2010 (Dynamic Allocation Functions) are enabled. Only some
of the features from this TR are supported by the GNU C Library.
If you define this macro, features from ISO/IEC TS 18661-1:2014 (Floating-point extensions for C: Binary floating-point arithmetic) are enabled. Only some of the features from this TS are supported by the GNU C Library.
If you define this macro, features from ISO/IEC TS 18661-4:2015 (Floating-point extensions for C: Supplementary functions) are enabled. Only some of the features from this TS are supported by the GNU C Library.
If you define this macro, features from ISO/IEC TS 18661-3:2015 (Floating-point extensions for C: Interchange and extended types) are enabled. Only some of the features from this TS are supported by the GNU C Library.
If you define this macro, ISO C23 features defined in Annex F of that
standard are enabled. This affects declarations of the
totalorder
functions and functions related to NaN payloads.
If you define this macro, everything is included: ISO C89, ISO C99, POSIX.1, POSIX.2, BSD, SVID, X/Open, LFS, and GNU extensions. In the cases where POSIX.1 conflicts with BSD, the POSIX definitions take precedence.
If you define this macro, most features are included apart from X/Open, LFS and GNU extensions: the effect is to enable features from the 2008 edition of POSIX, as well as certain BSD and SVID features without a separate feature test macro to control them.
Be aware that compiler options also affect included features:
If this macro is defined, additional *at
interfaces are
included.
If this macro is defined to 1, security hardening is added to various library functions. If defined to 2, even stricter checks are applied. If defined to 3, the GNU C Library may also use checks that may have an additional performance overhead. See Fortification of function calls.
If this macro is defined, correct (but non compile-time constant) MINSIGSTKSZ, SIGSTKSZ and PTHREAD_STACK_MIN are defined.
These macros are obsolete. They have the same effect as defining
_POSIX_C_SOURCE
with the value 199506L
.
Some very old C libraries required one of these macros to be defined
for basic functionality (e.g. getchar
) to be thread-safe.
We recommend you use _GNU_SOURCE
in new programs. If you don’t
specify the ‘-ansi’ option to GCC, or other conformance options
such as -std=c99, and don’t define any of these macros
explicitly, the effect is the same as defining _DEFAULT_SOURCE
to 1.
When you define a feature test macro to request a larger class of features,
it is harmless to define in addition a feature test macro for a subset of
those features. For example, if you define _POSIX_C_SOURCE
, then
defining _POSIX_SOURCE
as well has no effect. Likewise, if you
define _GNU_SOURCE
, then defining either _POSIX_SOURCE
or
_POSIX_C_SOURCE
as well has no effect.
Here is an overview of the contents of the remaining chapters of this manual.
isspace
) and functions for
performing case conversion.
char
data type.
FILE *
objects). These are the normal C library functions
from stdio.h.
setjmp
and
longjmp
functions. These functions provide a facility for
goto
-like jumps which can jump from one function to another.
sizeof
operator and the symbolic constant NULL
, how to write functions
accepting variable numbers of arguments, and constants describing the
ranges and other properties of the numerical types. There is also a simple
debugging mechanism which allows you to put assertions in your code, and
have diagnostic messages printed if the tests fail.
If you already know the name of the facility you are interested in, you can look it up in Summary of Library Facilities. This gives you a summary of its syntax and a pointer to where you can find a more detailed description. This appendix is particularly useful if you just want to verify the order and type of arguments to a function, for example. It also tells you what standard or system each function, variable, or macro is derived from.
Many functions in the GNU C Library detect and report error conditions, and sometimes your programs need to check for these error conditions. For example, when you open an input file, you should verify that the file was actually opened correctly, and print an error message or take other appropriate action if the call to the library function failed.
This chapter describes how the error reporting facility works. Your program should include the header file errno.h to use this facility.
Most library functions return a special value to indicate that they have
failed. The special value is typically -1
, a null pointer, or a
constant such as EOF
that is defined for that purpose. But this
return value tells you only that an error has occurred. To find out
what kind of error it was, you need to look at the error code stored in the
variable errno
. This variable is declared in the header file
errno.h.
volatile int
errno ¶The variable errno
contains the system error number. You can
change the value of errno
.
Since errno
is declared volatile
, it might be changed
asynchronously by a signal handler; see Defining Signal Handlers.
However, a properly written signal handler saves and restores the value
of errno
, so you generally do not need to worry about this
possibility except when writing signal handlers.
The initial value of errno
at program startup is zero. In many
cases, when a library function encounters an error, it will set
errno
to a non-zero value to indicate what specific error
condition occurred. The documentation for each function lists the
error conditions that are possible for that function. Not all library
functions use this mechanism; some return an error code directly,
instead.
Warning: Many library functions may set errno
to some
meaningless non-zero value even if they did not encounter any errors,
and even if they return error codes directly. Therefore, it is
usually incorrect to check whether an error occurred by
inspecting the value of errno
. The proper way to check for
error is documented for each function.
Portability Note: ISO C specifies errno
as a
“modifiable lvalue” rather than as a variable, permitting it to be
implemented as a macro. For example, its expansion might involve a
function call, like *__errno_location ()
. In fact, that is
what it is
on GNU/Linux and GNU/Hurd systems. The GNU C Library, on each system, does
whatever is right for the particular system.
There are a few library functions, like sqrt
and atan
,
that return a perfectly legitimate value in case of an error, but also
set errno
. For these functions, if you want to check to see
whether an error occurred, the recommended method is to set errno
to zero before calling the function, and then check its value afterward.
All the error codes have symbolic names; they are macros defined in errno.h. The names start with ‘E’ and an upper-case letter or digit; you should consider names of this form to be reserved names. See Reserved Names.
The error code values are all positive integers and are all distinct,
with one exception: EWOULDBLOCK
and EAGAIN
are the same.
Since the values are distinct, you can use them as labels in a
switch
statement; just don’t use both EWOULDBLOCK
and
EAGAIN
. Your program should not make any other assumptions about
the specific values of these symbolic constants.
The value of errno
doesn’t necessarily have to correspond to any
of these macros, since some library functions might return other error
codes of their own for other situations. The only values that are
guaranteed to be meaningful for a particular library function are the
ones that this manual lists for that function.
Except on GNU/Hurd systems, almost any system call can return EFAULT
if
it is given an invalid pointer as an argument. Since this could only
happen as a result of a bug in your program, and since it will not
happen on GNU/Hurd systems, we have saved space by not mentioning
EFAULT
in the descriptions of individual functions.
In some Unix systems, many system calls can also return EFAULT
if
given as an argument a pointer into the stack, and the kernel for some
obscure reason fails in its attempt to extend the stack. If this ever
happens, you should probably try using statically or dynamically
allocated memory instead of stack memory on that system.
The error code macros are defined in the header file errno.h. All of them expand into integer constant values. Some of these error codes can’t occur on GNU systems, but they can occur using the GNU C Library on other systems.
int
EPERM ¶“Operation not permitted.” Only the owner of the file (or other resource) or processes with special privileges can perform the operation.
int
ENOENT ¶“No such file or directory.” This is a “file doesn’t exist” error for ordinary files that are referenced in contexts where they are expected to already exist.
int
ESRCH ¶“No such process.” No process matches the specified process ID.
int
EINTR ¶“Interrupted system call.” An asynchronous signal occurred and prevented completion of the call. When this happens, you should try the call again.
You can choose to have functions resume after a signal that is handled,
rather than failing with EINTR
; see Primitives Interrupted by Signals.
int
EIO ¶“Input/output error.” Usually used for physical read or write errors.
int
ENXIO ¶“No such device or address.” The system tried to use the device represented by a file you specified, and it couldn’t find the device. This can mean that the device file was installed incorrectly, or that the physical device is missing or not correctly attached to the computer.
int
E2BIG ¶“Argument list too long.”
Used when the arguments passed to a new program
being executed with one of the exec
functions (see Executing a File) occupy too much memory space. This condition never arises on
GNU/Hurd systems.
int
ENOEXEC ¶“Exec format error.”
Invalid executable file format. This condition is detected by the
exec
functions; see Executing a File.
int
EBADF ¶“Bad file descriptor.” For example, I/O on a descriptor that has been closed or reading from a descriptor open only for writing (or vice versa).
int
ECHILD ¶“No child processes.” This error happens on operations that are supposed to manipulate child processes, when there aren’t any processes to manipulate.
int
EDEADLK ¶“Resource deadlock avoided.” Allocating a system resource would have resulted in a deadlock situation. The system does not guarantee that it will notice all such situations. This error means you got lucky and the system noticed; it might just hang. See File Locks, for an example.
int
ENOMEM ¶“Cannot allocate memory.” The system cannot allocate more virtual memory because its capacity is full.
int
EACCES ¶“Permission denied.” The file permissions do not allow the attempted operation.
int
EFAULT ¶“Bad address.” An invalid pointer was detected. On GNU/Hurd systems, this error never happens; you get a signal instead.
int
ENOTBLK ¶“Block device required.” A file that isn’t a block special file was given in a situation that requires one. For example, trying to mount an ordinary file as a file system in Unix gives this error.
int
EBUSY ¶“Device or resource busy.” A system resource that can’t be shared is already in use. For example, if you try to delete a file that is the root of a currently mounted filesystem, you get this error.
int
EEXIST ¶“File exists.” An existing file was specified in a context where it only makes sense to specify a new file.
int
EXDEV ¶“Invalid cross-device link.”
An attempt to make an improper link across file systems was detected.
This happens not only when you use link
(see Hard Links) but
also when you rename a file with rename
(see Renaming Files).
int
ENODEV ¶“No such device.” The wrong type of device was given to a function that expects a particular sort of device.
int
ENOTDIR ¶“Not a directory.” A file that isn’t a directory was specified when a directory is required.
int
EISDIR ¶“Is a directory.” You cannot open a directory for writing, or create or remove hard links to it.
int
EINVAL ¶“Invalid argument.” This is used to indicate various kinds of problems with passing the wrong argument to a library function.
int
EMFILE ¶“Too many open files.” The current process has too many files open and can’t open any more. Duplicate descriptors do count toward this limit.
In BSD and GNU, the number of open files is controlled by a resource
limit that can usually be increased. If you get this error, you might
want to increase the RLIMIT_NOFILE
limit or make it unlimited;
see Limiting Resource Usage.
int
ENFILE ¶“Too many open files in system.” There are too many distinct file openings in the entire system. Note that any number of linked channels count as just one file opening; see Linked Channels. This error never occurs on GNU/Hurd systems.
int
ENOTTY ¶“Inappropriate ioctl for device.” Inappropriate I/O control operation, such as trying to set terminal modes on an ordinary file.
int
ETXTBSY ¶“Text file busy.” An attempt to execute a file that is currently open for writing, or write to a file that is currently being executed. Often using a debugger to run a program is considered having it open for writing and will cause this error. (The name stands for “text file busy”.) This is not an error on GNU/Hurd systems; the text is copied as necessary.
int
EFBIG ¶“File too large.” The size of a file would be larger than allowed by the system.
int
ENOSPC ¶“No space left on device.” Write operation on a file failed because the disk is full.
int
ESPIPE ¶“Illegal seek.” Invalid seek operation (such as on a pipe).
int
EROFS ¶“Read-only file system.” An attempt was made to modify something on a read-only file system.
int
EMLINK ¶“Too many links.”
The link count of a single file would become too large.
rename
can cause this error if the file being renamed already has
as many links as it can take (see Renaming Files).
int
EPIPE ¶“Broken pipe.”
There is no process reading from the other end of a pipe.
Every library function that returns this error code also generates a
SIGPIPE
signal; this signal terminates the program if not handled
or blocked. Thus, your program will never actually see EPIPE
unless it has handled or blocked SIGPIPE
.
int
EDOM ¶“Numerical argument out of domain.” Used by mathematical functions when an argument value does not fall into the domain over which the function is defined.
int
ERANGE ¶“Numerical result out of range.” Used by mathematical functions when the result value is not representable because of overflow or underflow.
int
EAGAIN ¶“Resource temporarily unavailable.”
The call might work if you try again
later. The macro EWOULDBLOCK
is another name for EAGAIN
;
they are always the same in the GNU C Library.
This error can happen in a few different situations:
select
to find out
when the operation will be possible; see Waiting for Input or Output.
Portability Note: In many older Unix systems, this condition
was indicated by EWOULDBLOCK
, which was a distinct error code
different from EAGAIN
. To make your program portable, you should
check for both codes and treat them the same.
fork
can return this error. It indicates that the shortage is expected to
pass, so your program can try the call again later and it may succeed.
It is probably a good idea to delay for a few seconds before trying it
again, to allow time for other processes to release scarce resources.
Such shortages are usually fairly serious and affect the whole system,
so usually an interactive program should report the error to the user
and return to its command loop.
int
EWOULDBLOCK ¶“Operation would block.”
In the GNU C Library, this is another name for EAGAIN
(above).
The values are always the same, on every operating system.
C libraries in many older Unix systems have EWOULDBLOCK
as a
separate error code.
int
EINPROGRESS ¶“Operation now in progress.”
An operation that cannot complete immediately was initiated on an object
that has non-blocking mode selected. Some functions that must always
block (such as connect
; see Making a Connection) never return
EAGAIN
. Instead, they return EINPROGRESS
to indicate that
the operation has begun and will take some time. Attempts to manipulate
the object before the call completes return EALREADY
. You can
use the select
function to find out when the pending operation
has completed; see Waiting for Input or Output.
int
EALREADY ¶“Operation already in progress.” An operation is already in progress on an object that has non-blocking mode selected.
int
ENOTSOCK ¶“Socket operation on non-socket.” A file that isn’t a socket was specified when a socket is required.
int
EMSGSIZE ¶“Message too long.” The size of a message sent on a socket was larger than the supported maximum size.
int
EPROTOTYPE ¶“Protocol wrong type for socket.” The socket type does not support the requested communications protocol.
int
ENOPROTOOPT ¶“Protocol not available.” You specified a socket option that doesn’t make sense for the particular protocol being used by the socket. See Socket Options.
int
EPROTONOSUPPORT ¶“Protocol not supported.” The socket domain does not support the requested communications protocol (perhaps because the requested protocol is completely invalid). See Creating a Socket.
int
ESOCKTNOSUPPORT ¶“Socket type not supported.” The socket type is not supported.
int
EOPNOTSUPP ¶“Operation not supported.” The operation you requested is not supported. Some socket functions don’t make sense for all types of sockets, and others may not be implemented for all communications protocols. On GNU/Hurd systems, this error can happen for many calls when the object does not support the particular operation; it is a generic indication that the server knows nothing to do for that call.
int
EPFNOSUPPORT ¶“Protocol family not supported.” The socket communications protocol family you requested is not supported.
int
EAFNOSUPPORT ¶“Address family not supported by protocol.” The address family specified for a socket is not supported; it is inconsistent with the protocol being used on the socket. See Sockets.
int
EADDRINUSE ¶“Address already in use.” The requested socket address is already in use. See Socket Addresses.
int
EADDRNOTAVAIL ¶“Cannot assign requested address.” The requested socket address is not available; for example, you tried to give a socket a name that doesn’t match the local host name. See Socket Addresses.
int
ENETDOWN ¶“Network is down.” A socket operation failed because the network was down.
int
ENETUNREACH ¶“Network is unreachable.” A socket operation failed because the subnet containing the remote host was unreachable.
int
ENETRESET ¶“Network dropped connection on reset.” A network connection was reset because the remote host crashed.
int
ECONNABORTED ¶“Software caused connection abort.” A network connection was aborted locally.
int
ECONNRESET ¶“Connection reset by peer.” A network connection was closed for reasons outside the control of the local host, such as by the remote machine rebooting or an unrecoverable protocol violation.
int
ENOBUFS ¶“No buffer space available.”
The kernel’s buffers for I/O operations are all in use. In GNU, this
error is always synonymous with ENOMEM
; you may get one or the
other from network operations.
int
EISCONN ¶“Transport endpoint is already connected.” You tried to connect a socket that is already connected. See Making a Connection.
int
ENOTCONN ¶“Transport endpoint is not connected.”
The socket is not connected to anything. You get this error when you
try to transmit data over a socket, without first specifying a
destination for the data. For a connectionless socket (for datagram
protocols, such as UDP), you get EDESTADDRREQ
instead.
int
EDESTADDRREQ ¶“Destination address required.”
No default destination address was set for the socket. You get this
error when you try to transmit data over a connectionless socket,
without first specifying a destination for the data with connect
.
int
ESHUTDOWN ¶“Cannot send after transport endpoint shutdown.” The socket has already been shut down.
int
ETOOMANYREFS ¶“Too many references: cannot splice.”
int
ETIMEDOUT ¶“Connection timed out.” A socket operation with a specified timeout received no response during the timeout period.
int
ECONNREFUSED ¶“Connection refused.” A remote host refused to allow the network connection (typically because it is not running the requested service).
int
ELOOP ¶“Too many levels of symbolic links.” Too many levels of symbolic links were encountered in looking up a file name. This often indicates a cycle of symbolic links.
int
ENAMETOOLONG ¶“File name too long.”
Filename too long (longer than PATH_MAX
; see Limits on File System Capacity) or host name too long (in gethostname
or
sethostname
; see Host Identification).
int
EHOSTDOWN ¶“Host is down.” The remote host for a requested network connection is down.
int
EHOSTUNREACH ¶“No route to host.” The remote host for a requested network connection is not reachable.
int
ENOTEMPTY ¶“Directory not empty.” Directory not empty, where an empty directory was expected. Typically, this error occurs when you are trying to delete a directory.
int
EPROCLIM ¶“Too many processes.”
This means that the per-user limit on new process would be exceeded by
an attempted fork
. See Limiting Resource Usage, for details on
the RLIMIT_NPROC
limit.
int
EUSERS ¶“Too many users.” The file quota system is confused because there are too many users.
int
EDQUOT ¶“Disk quota exceeded.” The user’s disk quota was exceeded.
int
ESTALE ¶“Stale file handle.” This indicates an internal confusion in the file system which is due to file system rearrangements on the server host for NFS file systems or corruption in other file systems. Repairing this condition usually requires unmounting, possibly repairing and remounting the file system.
int
EREMOTE ¶“Object is remote.” An attempt was made to NFS-mount a remote file system with a file name that already specifies an NFS-mounted file. (This is an error on some operating systems, but we expect it to work properly on GNU/Hurd systems, making this error code impossible.)
int
EBADRPC ¶“RPC struct is bad.”
int
ERPCMISMATCH ¶“RPC version wrong.”
int
EPROGUNAVAIL ¶“RPC program not available.”
int
EPROGMISMATCH ¶“RPC program version wrong.”
int
EPROCUNAVAIL ¶“RPC bad procedure for program.”
int
ENOLCK ¶“No locks available.” This is used by the file locking facilities; see File Locks. This error is never generated by GNU/Hurd systems, but it can result from an operation to an NFS server running another operating system.
int
EFTYPE ¶“Inappropriate file type or format.” The file was the wrong type for the operation, or a data file had the wrong format.
On some systems chmod
returns this error if you try to set the
sticky bit on a non-directory file; see Assigning File Permissions.
int
EAUTH ¶“Authentication error.”
int
ENEEDAUTH ¶“Need authenticator.”
int
ENOSYS ¶“Function not implemented.”
This indicates that the function called is
not implemented at all, either in the C library itself or in the
operating system. When you get this error, you can be sure that this
particular function will always fail with ENOSYS
unless you
install a new version of the C library or the operating system.
int
ELIBEXEC ¶“Cannot exec a shared library directly.”
int
ENOTSUP ¶“Not supported.” A function returns this error when certain parameter values are valid, but the functionality they request is not available. This can mean that the function does not implement a particular command or option value or flag bit at all. For functions that operate on some object given in a parameter, such as a file descriptor or a port, it might instead mean that only that specific object (file descriptor, port, etc.) is unable to support the other parameters given; different file descriptors might support different ranges of parameter values.
If the entire function is not available at all in the implementation,
it returns ENOSYS
instead.
int
EILSEQ ¶“Invalid or incomplete multibyte or wide character.” While decoding a multibyte character the function came along an invalid or an incomplete sequence of bytes or the given wide character is invalid.
int
EBACKGROUND ¶“Inappropriate operation for background process.”
On GNU/Hurd systems, servers supporting the term
protocol return
this error for certain operations when the caller is not in the
foreground process group of the terminal. Users do not usually see this
error because functions such as read
and write
translate
it into a SIGTTIN
or SIGTTOU
signal. See Job Control,
for information on process groups and these signals.
int
EDIED ¶“Translator died.” On GNU/Hurd systems, opening a file returns this error when the file is translated by a program and the translator program dies while starting up, before it has connected to the file.
int
ED ¶“?.” The experienced user will know what is wrong.
int
EGREGIOUS ¶“You really blew it this time.” You did what?
int
EIEIO ¶“Computer bought the farm.” Go home and have a glass of warm, dairy-fresh milk.
int
EGRATUITOUS ¶“Gratuitous error.” This error code has no purpose.
int
EBADMSG ¶“Bad message.”
int
EIDRM ¶“Identifier removed.”
int
EMULTIHOP ¶“Multihop attempted.”
int
ENODATA ¶“No data available.”
int
ENOLINK ¶“Link has been severed.”
int
ENOMSG ¶“No message of desired type.”
int
ENOSR ¶“Out of streams resources.”
int
ENOSTR ¶“Device not a stream.”
int
EOVERFLOW ¶“Value too large for defined data type.”
int
EPROTO ¶“Protocol error.”
int
ETIME ¶“Timer expired.”
int
ECANCELED ¶“Operation canceled.”
An asynchronous operation was canceled before it
completed. See Perform I/O Operations in Parallel. When you call aio_cancel
,
the normal result is for the operations affected to complete with this
error; see Cancellation of AIO Operations.
int
EOWNERDEAD ¶“Owner died.”
int
ENOTRECOVERABLE ¶“State not recoverable.”
The following error codes are defined by the Linux/i386 kernel. They are not yet documented.
int
ERESTART ¶“Interrupted system call should be restarted.”
int
ECHRNG ¶“Channel number out of range.”
int
EL2NSYNC ¶“Level 2 not synchronized.”
int
EL3HLT ¶“Level 3 halted.”
int
EL3RST ¶“Level 3 reset.”
int
ELNRNG ¶“Link number out of range.”
int
EUNATCH ¶“Protocol driver not attached.”
int
ENOCSI ¶“No CSI structure available.”
int
EL2HLT ¶“Level 2 halted.”
int
EBADE ¶“Invalid exchange.”
int
EBADR ¶“Invalid request descriptor.”
int
EXFULL ¶“Exchange full.”
int
ENOANO ¶“No anode.”
int
EBADRQC ¶“Invalid request code.”
int
EBADSLT ¶“Invalid slot.”
int
EDEADLOCK ¶“File locking deadlock error.”
int
EBFONT ¶“Bad font file format.”
int
ENONET ¶“Machine is not on the network.”
int
ENOPKG ¶“Package not installed.”
int
EADV ¶“Advertise error.”
int
ESRMNT ¶“Srmount error.”
int
ECOMM ¶“Communication error on send.”
int
EDOTDOT ¶“RFS specific error.”
int
ENOTUNIQ ¶“Name not unique on network.”
int
EBADFD ¶“File descriptor in bad state.”
int
EREMCHG ¶“Remote address changed.”
int
ELIBACC ¶“Can not access a needed shared library.”
int
ELIBBAD ¶“Accessing a corrupted shared library.”
int
ELIBSCN ¶“.lib section in a.out corrupted.”
int
ELIBMAX ¶“Attempting to link in too many shared libraries.”
int
ESTRPIPE ¶“Streams pipe error.”
int
EUCLEAN ¶“Structure needs cleaning.”
int
ENOTNAM ¶“Not a XENIX named type file.”
int
ENAVAIL ¶“No XENIX semaphores available.”
int
EISNAM ¶“Is a named type file.”
int
EREMOTEIO ¶“Remote I/O error.”
int
ENOMEDIUM ¶“No medium found.”
int
EMEDIUMTYPE ¶“Wrong medium type.”
int
ENOKEY ¶“Required key not available.”
int
EKEYEXPIRED ¶“Key has expired.”
int
EKEYREVOKED ¶“Key has been revoked.”
int
EKEYREJECTED ¶“Key was rejected by service.”
int
ERFKILL ¶“Operation not possible due to RF-kill.”
int
EHWPOISON ¶“Memory page has hardware error.”
The library has functions and variables designed to make it easy for
your program to report informative error messages in the customary
format about the failure of a library call. The functions
strerror
and perror
give you the standard error message
for a given error code; the variable
program_invocation_short_name
gives you convenient access to the
name of the program that encountered the error.
char *
strerror (int errnum)
¶Preliminary: | MT-Safe | AS-Unsafe heap i18n | AC-Unsafe mem | See POSIX Safety Concepts.
The strerror
function maps the error code (see Checking for Errors) specified by the errnum argument to a descriptive error
message string. The string is translated according to the current
locale. The return value is a pointer to this string.
The value errnum normally comes from the variable errno
.
You should not modify the string returned by strerror
. Also, if
you make subsequent calls to strerror
or strerror_l
, or
the thread that obtained the string exits, the returned pointer will be
invalidated.
As there is no way to restore the previous state after calling
strerror
, library code should not call this function because it
may interfere with application use of strerror
, invalidating the
string pointer before the application is done using it. Instead,
strerror_r
, snprintf
with the ‘%m’ or ‘%#m’
specifiers, strerrorname_np
, or strerrordesc_np
can be
used instead.
The strerror
function preserves the value of errno
and
cannot fail.
The function strerror
is declared in string.h.
char *
strerror_l (int errnum, locale_t locale)
¶Preliminary: | MT-Safe | AS-Unsafe heap i18n | AC-Unsafe mem | See POSIX Safety Concepts.
This function is like strerror
, except that the returned string
is translated according to locale (instead of the current locale
used by strerror
). Note that calling strerror_l
invalidates the pointer returned by strerror
and vice versa.
The function strerror_l
is defined by POSIX and is declared in
string.h.
char *
strerror_r (int errnum, char *buf, size_t n)
¶Preliminary: | MT-Safe | AS-Unsafe i18n | AC-Unsafe | See POSIX Safety Concepts.
The following description is for the GNU variant of the function,
used if _GNU_SOURCE
is defined. See Feature Test Macros.
The strerror_r
function works like strerror
but instead of
returning a pointer to a string that is managed by the GNU C Library, it can
use the user supplied buffer starting at buf for storing the
string.
At most n characters are written (including the NUL byte) to buf, so it is up to the user to select a buffer large enough. Whether returned pointer points to the buf array or not depends on the errnum argument. If the result string is not stored in buf, the string will not change for the remaining execution of the program.
The function strerror_r
as described above is a GNU extension and
it is declared in string.h. There is a POSIX variant of this
function, described next.
int
strerror_r (int errnum, char *buf, size_t n)
¶Preliminary: | MT-Safe | AS-Unsafe i18n | AC-Unsafe | See POSIX Safety Concepts.
This variant of the strerror_r
function is used if a standard is
selected that includes strerror_r
, but _GNU_SOURCE
is not
defined. This POSIX variant of the function always writes the error
message to the specified buffer buf of size n bytes.
Upon success, strerror_r
returns 0. Two more return values are
used to indicate failure.
EINVAL
¶The errnum argument does not correspond to a known error constant.
ERANGE
¶The buffer size n is not large enough to store the entire error message.
Even if an error is reported, strerror_r
still writes as much of
the error message to the output buffer as possible. After a call to
strerror_r
, the value of errno
is unspecified.
If you want to use the always-copying POSIX semantics of
strerror_r
in a program that is potentially compiled with
_GNU_SOURCE
defined, you can use snprintf
with the
‘%m’ conversion specifier, like this:
int saved_errno = errno; errno = errnum; int ret = snprintf (buf, n, "%m"); errno = saved_errno; if (strerrorname_np (errnum) == NULL) return EINVAL; if (ret >= n) return ERANGE: return 0;
This function is declared in string.h if it is declared at all. It is a POSIX extension.
void
perror (const char *message)
¶Preliminary: | MT-Safe race:stderr | AS-Unsafe corrupt i18n heap lock | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
This function prints an error message to the stream stderr
;
see Standard Streams. The orientation of stderr
is not
changed.
If you call perror
with a message that is either a null
pointer or an empty string, perror
just prints the error message
corresponding to errno
, adding a trailing newline.
If you supply a non-null message argument, then perror
prefixes its output with this string. It adds a colon and a space
character to separate the message from the error string corresponding
to errno
.
The function perror
is declared in stdio.h.
const char *
strerrorname_np (int errnum)
¶| MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function returns the name describing the error errnum or
NULL
if there is no known constant with this value (e.g "EINVAL"
for EINVAL
). The returned string does not change for the
remaining execution of the program.
This function is a GNU extension, declared in the header file string.h.
const char *
strerrordesc_np (int errnum)
¶| MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function returns the message describing the error errnum or
NULL
if there is no known constant with this value (e.g "Invalid
argument" for EINVAL
). Different than strerror
the
returned description is not translated, and the returned string does not
change for the remaining execution of the program.
This function is a GNU extension, declared in the header file string.h.
strerror
and perror
produce the exact same message for any
given error code under the same locale; the precise text varies from
system to system. With the GNU C Library, the messages are fairly short;
there are no multi-line messages or embedded newlines. Each error
message begins with a capital letter and does not include any
terminating punctuation.
Many programs that don’t read input from the terminal are designed to
exit if any system call fails. By convention, the error message from
such a program should start with the program’s name, sans directories.
You can find that name in the variable
program_invocation_short_name
; the full file name is stored the
variable program_invocation_name
.
char *
program_invocation_name ¶This variable’s value is the name that was used to invoke the program
running in the current process. It is the same as argv[0]
. Note
that this is not necessarily a useful file name; often it contains no
directory names. See Program Arguments.
This variable is a GNU extension and is declared in errno.h.
char *
program_invocation_short_name ¶This variable’s value is the name that was used to invoke the program
running in the current process, with directory names removed. (That is
to say, it is the same as program_invocation_name
minus
everything up to the last slash, if any.)
This variable is a GNU extension and is declared in errno.h.
The library initialization code sets up both of these variables before
calling main
.
Portability Note: If you want your program to work with
non-GNU libraries, you must save the value of argv[0]
in
main
, and then strip off the directory names yourself. We
added these extensions to make it possible to write self-contained
error-reporting subroutines that require no explicit cooperation from
main
.
Here is an example showing how to handle failure to open a file
correctly. The function open_sesame
tries to open the named file
for reading and returns a stream if successful. The fopen
library function returns a null pointer if it couldn’t open the file for
some reason. In that situation, open_sesame
constructs an
appropriate error message using the strerror
function, and
terminates the program. If we were going to make some other library
calls before passing the error code to strerror
, we’d have to
save it in a local variable instead, because those other library
functions might overwrite errno
in the meantime.
#define _GNU_SOURCE #include <errno.h> #include <stdio.h> #include <stdlib.h> #include <string.h> FILE * open_sesame (char *name) { FILE *stream; errno = 0; stream = fopen (name, "r"); if (stream == NULL) { fprintf (stderr, "%s: Couldn't open file %s; %s\n", program_invocation_short_name, name, strerror (errno)); exit (EXIT_FAILURE); } else return stream; }
Using perror
has the advantage that the function is portable and
available on all systems implementing ISO C. But often the text
perror
generates is not what is wanted and there is no way to
extend or change what perror
does. The GNU coding standard, for
instance, requires error messages to be preceded by the program name and
programs which read some input files should provide information
about the input file name and the line number in case an error is
encountered while reading the file. For these occasions there are two
functions available which are widely used throughout the GNU project.
These functions are declared in error.h.
void
error (int status, int errnum, const char *format, …)
¶Preliminary: | MT-Safe locale | AS-Unsafe corrupt heap i18n | AC-Safe | See POSIX Safety Concepts.
The error
function can be used to report general problems during
program execution. The format argument is a format string just
like those given to the printf
family of functions. The
arguments required for the format can follow the format parameter.
Just like perror
, error
also can report an error code in
textual form. But unlike perror
the error value is explicitly
passed to the function in the errnum parameter. This eliminates
the problem mentioned above that the error reporting function must be
called immediately after the function causing the error since otherwise
errno
might have a different value.
error
prints first the program name. If the application
defined a global variable error_print_progname
and points it to a
function this function will be called to print the program name.
Otherwise the string from the global variable program_name
is
used. The program name is followed by a colon and a space which in turn
is followed by the output produced by the format string. If the
errnum parameter is non-zero the format string output is followed
by a colon and a space, followed by the error message for the error code
errnum. In any case is the output terminated with a newline.
The output is directed to the stderr
stream. If the
stderr
wasn’t oriented before the call it will be narrow-oriented
afterwards.
The function will return unless the status parameter has a
non-zero value. In this case the function will call exit
with
the status value for its parameter and therefore never return. If
error
returns, the global variable error_message_count
is
incremented by one to keep track of the number of errors reported.
void
error_at_line (int status, int errnum, const char *fname, unsigned int lineno, const char *format, …)
¶Preliminary: | MT-Unsafe race:error_at_line/error_one_per_line locale | AS-Unsafe corrupt heap i18n | AC-Unsafe corrupt/error_one_per_line | See POSIX Safety Concepts.
The error_at_line
function is very similar to the error
function. The only differences are the additional parameters fname
and lineno. The handling of the other parameters is identical to
that of error
except that between the program name and the string
generated by the format string additional text is inserted.
Directly following the program name a colon, followed by the file name pointed to by fname, another colon, and the value of lineno is printed.
This additional output of course is meant to be used to locate an error in an input file (like a programming language source code file etc).
If the global variable error_one_per_line
is set to a non-zero
value error_at_line
will avoid printing consecutive messages for
the same file and line. Repetition which are not directly following
each other are not caught.
Just like error
this function only returns if status is
zero. Otherwise exit
is called with the non-zero value. If
error
returns, the global variable error_message_count
is
incremented by one to keep track of the number of errors reported.
As mentioned above, the error
and error_at_line
functions
can be customized by defining a variable named
error_print_progname
.
void (*error_print_progname)
(void) ¶If the error_print_progname
variable is defined to a non-zero
value the function pointed to is called by error
or
error_at_line
. It is expected to print the program name or do
something similarly useful.
The function is expected to print to the stderr
stream and
must be able to handle whatever orientation the stream has.
The variable is global and shared by all threads.
unsigned int
error_message_count ¶The error_message_count
variable is incremented whenever one of
the functions error
or error_at_line
returns. The
variable is global and shared by all threads.
int
error_one_per_line ¶The error_one_per_line
variable influences only
error_at_line
. Normally the error_at_line
function
creates output for every invocation. If error_one_per_line
is
set to a non-zero value error_at_line
keeps track of the last
file name and line number for which an error was reported and avoids
directly following messages for the same file and line. This variable
is global and shared by all threads.
A program which read some input file and reports errors in it could look like this:
{ char *line = NULL; size_t len = 0; unsigned int lineno = 0; error_message_count = 0; while (! feof_unlocked (fp)) { ssize_t n = getline (&line, &len, fp); if (n <= 0) /* End of file or error. */ break; ++lineno; /* Process the line. */ ... if (Detect error in line) error_at_line (0, errval, filename, lineno, "some error text %s", some_variable); } if (error_message_count != 0) error (EXIT_FAILURE, 0, "%u errors found", error_message_count); }
error
and error_at_line
are clearly the functions of
choice and enable the programmer to write applications which follow the
GNU coding standard. The GNU C Library additionally contains functions which
are used in BSD for the same purpose. These functions are declared in
err.h. It is generally advised to not use these functions. They
are included only for compatibility.
void
warn (const char *format, …)
¶Preliminary: | MT-Safe locale | AS-Unsafe corrupt heap i18n | AC-Unsafe corrupt lock mem | See POSIX Safety Concepts.
The warn
function is roughly equivalent to a call like
error (0, errno, format, the parameters)
except that the global variables error
respects and modifies
are not used.
void
vwarn (const char *format, va_list ap)
¶Preliminary: | MT-Safe locale | AS-Unsafe corrupt heap i18n | AC-Unsafe corrupt lock mem | See POSIX Safety Concepts.
The vwarn
function is just like warn
except that the
parameters for the handling of the format string format are passed
in as a value of type va_list
.
void
warnx (const char *format, …)
¶Preliminary: | MT-Safe locale | AS-Unsafe corrupt heap | AC-Unsafe corrupt lock mem | See POSIX Safety Concepts.
The warnx
function is roughly equivalent to a call like
error (0, 0, format, the parameters)
except that the global variables error
respects and modifies
are not used. The difference to warn
is that no error number
string is printed.
void
vwarnx (const char *format, va_list ap)
¶Preliminary: | MT-Safe locale | AS-Unsafe corrupt heap | AC-Unsafe corrupt lock mem | See POSIX Safety Concepts.
The vwarnx
function is just like warnx
except that the
parameters for the handling of the format string format are passed
in as a value of type va_list
.
void
err (int status, const char *format, …)
¶Preliminary: | MT-Safe locale | AS-Unsafe corrupt heap i18n | AC-Unsafe corrupt lock mem | See POSIX Safety Concepts.
The err
function is roughly equivalent to a call like
error (status, errno, format, the parameters)
except that the global variables error
respects and modifies
are not used and that the program is exited even if status is zero.
void
verr (int status, const char *format, va_list ap)
¶Preliminary: | MT-Safe locale | AS-Unsafe corrupt heap i18n | AC-Unsafe corrupt lock mem | See POSIX Safety Concepts.
The verr
function is just like err
except that the
parameters for the handling of the format string format are passed
in as a value of type va_list
.
void
errx (int status, const char *format, …)
¶Preliminary: | MT-Safe locale | AS-Unsafe corrupt heap | AC-Unsafe corrupt lock mem | See POSIX Safety Concepts.
The errx
function is roughly equivalent to a call like
error (status, 0, format, the parameters)
except that the global variables error
respects and modifies
are not used and that the program is exited even if status
is zero. The difference to err
is that no error number
string is printed.
void
verrx (int status, const char *format, va_list ap)
¶Preliminary: | MT-Safe locale | AS-Unsafe corrupt heap | AC-Unsafe corrupt lock mem | See POSIX Safety Concepts.
The verrx
function is just like errx
except that the
parameters for the handling of the format string format are passed
in as a value of type va_list
.
This chapter describes how processes manage and use memory in a system that uses the GNU C Library.
The GNU C Library has several functions for dynamically allocating virtual memory in various ways. They vary in generality and in efficiency. The library also provides functions for controlling paging and allocation of real memory.
Memory mapped I/O is not discussed in this chapter. See Memory-mapped I/O.
One of the most basic resources a process has available to it is memory. There are a lot of different ways systems organize memory, but in a typical one, each process has one linear virtual address space, with addresses running from zero to some huge maximum. It need not be contiguous; i.e., not all of these addresses actually can be used to store data.
The virtual memory is divided into pages (4 kilobytes is typical). Backing each page of virtual memory is a page of real memory (called a frame) or some secondary storage, usually disk space. The disk space might be swap space or just some ordinary disk file. Actually, a page of all zeroes sometimes has nothing at all backing it – there’s just a flag saying it is all zeroes.
The same frame of real memory or backing store can back multiple virtual
pages belonging to multiple processes. This is normally the case, for
example, with virtual memory occupied by GNU C Library code. The same
real memory frame containing the printf
function backs a virtual
memory page in each of the existing processes that has a printf
call in its program.
In order for a program to access any part of a virtual page, the page must at that moment be backed by (“connected to”) a real frame. But because there is usually a lot more virtual memory than real memory, the pages must move back and forth between real memory and backing store regularly, coming into real memory when a process needs to access them and then retreating to backing store when not needed anymore. This movement is called paging.
When a program attempts to access a page which is not at that moment backed by real memory, this is known as a page fault. When a page fault occurs, the kernel suspends the process, places the page into a real page frame (this is called “paging in” or “faulting in”), then resumes the process so that from the process’ point of view, the page was in real memory all along. In fact, to the process, all pages always seem to be in real memory. Except for one thing: the elapsed execution time of an instruction that would normally be a few nanoseconds is suddenly much, much, longer (because the kernel normally has to do I/O to complete the page-in). For programs sensitive to that, the functions described in Locking Pages can control it.
Within each virtual address space, a process has to keep track of what is at which addresses, and that process is called memory allocation. Allocation usually brings to mind meting out scarce resources, but in the case of virtual memory, that’s not a major goal, because there is generally much more of it than anyone needs. Memory allocation within a process is mainly just a matter of making sure that the same byte of memory isn’t used to store two different things.
Processes allocate memory in two major ways: by exec and programmatically. Actually, forking is a third way, but it’s not very interesting. See Creating a Process.
Exec is the operation of creating a virtual address space for a process,
loading its basic program into it, and executing the program. It is
done by the “exec” family of functions (e.g. execl
). The
operation takes a program file (an executable), it allocates space to
load all the data in the executable, loads it, and transfers control to
it. That data is most notably the instructions of the program (the
text), but also literals and constants in the program and even
some variables: C variables with the static storage class (see Memory Allocation in C Programs).
Once that program begins to execute, it uses programmatic allocation to gain additional memory. In a C program with the GNU C Library, there are two kinds of programmatic allocation: automatic and dynamic. See Memory Allocation in C Programs.
Memory-mapped I/O is another form of dynamic virtual memory allocation. Mapping memory to a file means declaring that the contents of certain range of a process’ addresses shall be identical to the contents of a specified regular file. The system makes the virtual memory initially contain the contents of the file, and if you modify the memory, the system writes the same modification to the file. Note that due to the magic of virtual memory and page faults, there is no reason for the system to do I/O to read the file, or allocate real memory for its contents, until the program accesses the virtual memory. See Memory-mapped I/O.
Just as it programmatically allocates memory, the program can programmatically deallocate (free) it. You can’t free the memory that was allocated by exec. When the program exits or execs, you might say that all its memory gets freed, but since in both cases the address space ceases to exist, the point is really moot. See Program Termination.
A process’ virtual address space is divided into segments. A segment is a contiguous range of virtual addresses. Three important segments are:
This section covers how ordinary programs manage storage for their data,
including the famous malloc
function and some fancier facilities
special to the GNU C Library and GNU Compiler.
malloc
The C language supports two kinds of memory allocation through the variables in C programs:
In GNU C, the size of the automatic storage can be an expression that varies. In other C implementations, it must be a constant.
A third important kind of memory allocation, dynamic allocation, is not supported by C variables but is available via GNU C Library functions.
Dynamic memory allocation is a technique in which programs determine as they are running where to store some information. You need dynamic allocation when the amount of memory you need, or how long you continue to need it, depends on factors that are not known before the program runs.
For example, you may need a block to store a line read from an input file; since there is no limit to how long a line can be, you must allocate the memory dynamically and make it dynamically larger as you read more of the line.
Or, you may need a block for each record or each definition in the input data; since you can’t know in advance how many there will be, you must allocate a new block for each record or definition as you read it.
When you use dynamic allocation, the allocation of a block of memory is an action that the program requests explicitly. You call a function or macro when you want to allocate space, and specify the size with an argument. If you want to free the space, you do so by calling another function or macro. You can do these things whenever you want, as often as you want.
Dynamic allocation is not supported by C variables; there is no storage class “dynamic”, and there can never be a C variable whose value is stored in dynamically allocated space. The only way to get dynamically allocated memory is via a system call (which is generally via a GNU C Library function call), and the only way to refer to dynamically allocated space is through a pointer. Because it is less convenient, and because the actual process of dynamic allocation requires more computation time, programmers generally use dynamic allocation only when neither static nor automatic allocation will serve.
For example, if you want to allocate dynamically some space to hold a
struct foobar
, you cannot declare a variable of type struct
foobar
whose contents are the dynamically allocated space. But you can
declare a variable of pointer type struct foobar *
and assign it the
address of the space. Then you can use the operators ‘*’ and
‘->’ on this pointer variable to refer to the contents of the space:
{ struct foobar *ptr = malloc (sizeof *ptr); ptr->name = x; ptr->next = current_foobar; current_foobar = ptr; }
The malloc
implementation in the GNU C Library is derived from ptmalloc
(pthreads malloc), which in turn is derived from dlmalloc (Doug Lea malloc).
This malloc
may allocate memory
in two different ways depending on their size
and certain parameters that may be controlled by users. The most common way is
to allocate portions of memory (called chunks) from a large contiguous area of
memory and manage these areas to optimize their use and reduce wastage in the
form of unusable chunks. Traditionally the system heap was set up to be the one
large memory area but the GNU C Library malloc
implementation maintains
multiple such areas to optimize their use in multi-threaded applications. Each
such area is internally referred to as an arena.
As opposed to other versions, the malloc
in the GNU C Library does not round
up chunk sizes to powers of two, neither for large nor for small sizes.
Neighboring chunks can be coalesced on a free
no matter what their size
is. This makes the implementation suitable for all kinds of allocation
patterns without generally incurring high memory waste through fragmentation.
The presence of multiple arenas allows multiple threads to allocate
memory simultaneously in separate arenas, thus improving performance.
The other way of memory allocation is for very large blocks, i.e. much larger
than a page. These requests are allocated with mmap
(anonymous or via
/dev/zero; see Memory-mapped I/O)). This has the great advantage
that these chunks are returned to the system immediately when they are freed.
Therefore, it cannot happen that a large chunk becomes “locked” in between
smaller ones and even after calling free
wastes memory. The size
threshold for mmap
to be used is dynamic and gets adjusted according to
allocation patterns of the program. mallopt
can be used to statically
adjust the threshold using M_MMAP_THRESHOLD
and the use of mmap
can be disabled completely with M_MMAP_MAX
;
see Malloc Tunable Parameters.
A more detailed technical description of the GNU Allocator is maintained in the GNU C Library wiki. See https://sourceware.org/glibc/wiki/MallocInternals.
It is possible to use your own custom malloc
instead of the
built-in allocator provided by the GNU C Library. See Replacing malloc
.
The most general dynamic allocation facility is malloc
. It
allows you to allocate blocks of memory of any size at any time, make
them bigger or smaller at any time, and free the blocks individually at
any time (or never).
malloc
malloc
malloc
malloc
-Related FunctionsTo allocate a block of memory, call malloc
. The prototype for
this function is in stdlib.h.
void *
malloc (size_t size)
¶Preliminary: | MT-Safe | AS-Unsafe lock | AC-Unsafe lock fd mem | See POSIX Safety Concepts.
This function returns a pointer to a newly allocated block size
bytes long, or a null pointer (setting errno
)
if the block could not be allocated.
The contents of the block are undefined; you must initialize it yourself
(or use calloc
instead; see Allocating Cleared Space).
Normally you would convert the value to a pointer to the kind of object
that you want to store in the block. Here we show an example of doing
so, and of initializing the space with zeros using the library function
memset
(see Copying Strings and Arrays):
struct foo *ptr = malloc (sizeof *ptr); if (ptr == 0) abort (); memset (ptr, 0, sizeof (struct foo));
You can store the result of malloc
into any pointer variable
without a cast, because ISO C automatically converts the type
void *
to another type of pointer when necessary. However, a cast
is necessary if the type is needed but not specified by context.
Remember that when allocating space for a string, the argument to
malloc
must be one plus the length of the string. This is
because a string is terminated with a null character that doesn’t count
in the “length” of the string but does need space. For example:
char *ptr = malloc (length + 1);
See Representation of Strings, for more information about this.
malloc
¶If no more space is available, malloc
returns a null pointer.
You should check the value of every call to malloc
. It is
useful to write a subroutine that calls malloc
and reports an
error if the value is a null pointer, returning only if the value is
nonzero. This function is conventionally called xmalloc
. Here
it is:
void * xmalloc (size_t size) { void *value = malloc (size); if (value == 0) fatal ("virtual memory exhausted"); return value; }
Here is a real example of using malloc
(by way of xmalloc
).
The function savestring
will copy a sequence of characters into
a newly allocated null-terminated string:
char * savestring (const char *ptr, size_t len) { char *value = xmalloc (len + 1); value[len] = '\0'; return memcpy (value, ptr, len); }
The block that malloc
gives you is guaranteed to be aligned so
that it can hold any type of data. On GNU systems, the address is
always a multiple of eight on 32-bit systems, and a multiple of 16 on
64-bit systems. Only rarely is any higher boundary (such as a page
boundary) necessary; for those cases, use aligned_alloc
or
posix_memalign
(see Allocating Aligned Memory Blocks).
Note that the memory located after the end of the block is likely to be
in use for something else; perhaps a block already allocated by another
call to malloc
. If you attempt to treat the block as longer than
you asked for it to be, you are liable to destroy the data that
malloc
uses to keep track of its blocks, or you may destroy the
contents of another block. If you have already allocated a block and
discover you want it to be bigger, use realloc
(see Changing the Size of a Block).
Portability Notes:
malloc (0)
returns a non-null pointer to a newly allocated size-zero block;
other implementations may return NULL
instead.
POSIX and the ISO C standard allow both behaviors.
malloc
call sets errno
,
but ISO C does not require this and non-POSIX implementations
need not set errno
when failing.
malloc
always fails when size exceeds
PTRDIFF_MAX
, to avoid problems with programs that subtract
pointers or use signed indexes. Other implementations may succeed in
this case, leading to undefined behavior later.
malloc
¶When you no longer need a block that you got with malloc
, use the
function free
to make the block available to be allocated again.
The prototype for this function is in stdlib.h.
void
free (void *ptr)
¶Preliminary: | MT-Safe | AS-Unsafe lock | AC-Unsafe lock fd mem | See POSIX Safety Concepts.
The free
function deallocates the block of memory pointed at
by ptr.
Freeing a block alters the contents of the block. Do not expect to find any data (such as a pointer to the next block in a chain of blocks) in the block after freeing it. Copy whatever you need out of the block before freeing it! Here is an example of the proper way to free all the blocks in a chain, and the strings that they point to:
struct chain { struct chain *next; char *name; } void free_chain (struct chain *chain) { while (chain != 0) { struct chain *next = chain->next; free (chain->name); free (chain); chain = next; } }
Occasionally, free
can actually return memory to the operating
system and make the process smaller. Usually, all it can do is allow a
later call to malloc
to reuse the space. In the meantime, the
space remains in your program as part of a free-list used internally by
malloc
.
The free
function preserves the value of errno
, so that
cleanup code need not worry about saving and restoring errno
around a call to free
. Although neither ISO C nor
POSIX.1-2017 requires free
to preserve errno
, a future
version of POSIX is planned to require it.
There is no point in freeing blocks at the end of a program, because all of the program’s space is given back to the system when the process terminates.
Often you do not know for certain how big a block you will ultimately need at the time you must begin to use the block. For example, the block might be a buffer that you use to hold a line being read from a file; no matter how long you make the buffer initially, you may encounter a line that is longer.
You can make the block longer by calling realloc
or
reallocarray
. These functions are declared in stdlib.h.
void *
realloc (void *ptr, size_t newsize)
¶Preliminary: | MT-Safe | AS-Unsafe lock | AC-Unsafe lock fd mem | See POSIX Safety Concepts.
The realloc
function changes the size of the block whose address is
ptr to be newsize.
Since the space after the end of the block may be in use, realloc
may find it necessary to copy the block to a new address where more free
space is available. The value of realloc
is the new address of the
block. If the block needs to be moved, realloc
copies the old
contents.
If you pass a null pointer for ptr, realloc
behaves just
like ‘malloc (newsize)’.
Otherwise, if newsize is zero
realloc
frees the block and returns NULL
.
Otherwise, if realloc
cannot reallocate the requested size
it returns NULL
and sets errno
; the original block
is left undisturbed.
void *
reallocarray (void *ptr, size_t nmemb, size_t size)
¶Preliminary: | MT-Safe | AS-Unsafe lock | AC-Unsafe lock fd mem | See POSIX Safety Concepts.
The reallocarray
function changes the size of the block whose address
is ptr to be long enough to contain a vector of nmemb elements,
each of size size. It is equivalent to ‘realloc (ptr,
nmemb * size)’, except that reallocarray
fails safely if
the multiplication overflows, by setting errno
to ENOMEM
,
returning a null pointer, and leaving the original block unchanged.
reallocarray
should be used instead of realloc
when the new size
of the allocated block is the result of a multiplication that might overflow.
Portability Note: This function is not part of any standard. It was first introduced in OpenBSD 5.6.
Like malloc
, realloc
and reallocarray
may return a null
pointer if no memory space is available to make the block bigger. When this
happens, the original block is untouched; it has not been modified or
relocated.
In most cases it makes no difference what happens to the original block
when realloc
fails, because the application program cannot continue
when it is out of memory, and the only thing to do is to give a fatal error
message. Often it is convenient to write and use subroutines,
conventionally called xrealloc
and xreallocarray
,
that take care of the error message
as xmalloc
does for malloc
:
void * xreallocarray (void *ptr, size_t nmemb, size_t size) { void *value = reallocarray (ptr, nmemb, size); if (value == 0) fatal ("Virtual memory exhausted"); return value; } void * xrealloc (void *ptr, size_t size) { return xreallocarray (ptr, 1, size); }
You can also use realloc
or reallocarray
to make a block
smaller. The reason you would do this is to avoid tying up a lot of memory
space when only a little is needed.
In several allocation implementations, making a block smaller sometimes
necessitates copying it, so it can fail if no other space is available.
Portability Notes:
realloc (ptr, 0)
might free the block and return a non-null pointer to a size-zero
object, or it might fail and return NULL
without freeing the block.
The ISO C17 standard allows these variations.
PTRDIFF_MAX
in size, to avoid problems with programs
that subtract pointers or use signed indexes. Other implementations may
succeed, leading to undefined behavior later.
realloc
and
reallocarray
are guaranteed to change nothing and return the same
address that you gave. However, POSIX and ISO C allow the functions
to relocate the object or fail in this situation.
The function calloc
allocates memory and clears it to zero. It
is declared in stdlib.h.
void *
calloc (size_t count, size_t eltsize)
¶Preliminary: | MT-Safe | AS-Unsafe lock | AC-Unsafe lock fd mem | See POSIX Safety Concepts.
This function allocates a block long enough to contain a vector of
count elements, each of size eltsize. Its contents are
cleared to zero before calloc
returns.
You could define calloc
as follows:
void * calloc (size_t count, size_t eltsize) { void *value = reallocarray (0, count, eltsize); if (value != 0) memset (value, 0, count * eltsize); return value; }
But in general, it is not guaranteed that calloc
calls
reallocarray
and memset
internally. For example, if the
calloc
implementation knows for other reasons that the new
memory block is zero, it need not zero out the block again with
memset
. Also, if an application provides its own
reallocarray
outside the C library, calloc
might not use
that redefinition. See Replacing malloc
.
The address of a block returned by malloc
or realloc
in
GNU systems is always a multiple of eight (or sixteen on 64-bit
systems). If you need a block whose address is a multiple of a higher
power of two than that, use aligned_alloc
or posix_memalign
.
aligned_alloc
and posix_memalign
are declared in
stdlib.h.
void *
aligned_alloc (size_t alignment, size_t size)
¶Preliminary: | MT-Safe | AS-Unsafe lock | AC-Unsafe lock fd mem | See POSIX Safety Concepts.
The aligned_alloc
function allocates a block of size bytes whose
address is a multiple of alignment. The alignment must be a
power of two.
The aligned_alloc
function returns a null pointer on error and sets
errno
to one of the following values:
ENOMEM
There was insufficient memory available to satisfy the request.
EINVAL
alignment is not a power of two.
This function was introduced in ISO C11 and hence may have better
portability to modern non-POSIX systems than posix_memalign
.
void *
memalign (size_t boundary, size_t size)
¶Preliminary: | MT-Safe | AS-Unsafe lock | AC-Unsafe lock fd mem | See POSIX Safety Concepts.
The memalign
function allocates a block of size bytes whose
address is a multiple of boundary. The boundary must be a
power of two! The function memalign
works by allocating a
somewhat larger block, and then returning an address within the block
that is on the specified boundary.
The memalign
function returns a null pointer on error and sets
errno
to one of the following values:
ENOMEM
There was insufficient memory available to satisfy the request.
EINVAL
boundary is not a power of two.
The memalign
function is obsolete and aligned_alloc
or
posix_memalign
should be used instead.
int
posix_memalign (void **memptr, size_t alignment, size_t size)
¶Preliminary: | MT-Safe | AS-Unsafe lock | AC-Unsafe lock fd mem | See POSIX Safety Concepts.
The posix_memalign
function is similar to the memalign
function in that it returns a buffer of size bytes aligned to a
multiple of alignment. But it adds one requirement to the
parameter alignment: the value must be a power of two multiple of
sizeof (void *)
.
If the function succeeds in allocation memory a pointer to the allocated
memory is returned in *memptr
and the return value is zero.
Otherwise the function returns an error value indicating the problem.
The possible error values returned are:
ENOMEM
There was insufficient memory available to satisfy the request.
EINVAL
alignment is not a power of two multiple of sizeof (void *)
.
This function was introduced in POSIX 1003.1d. Although this function is
superseded by aligned_alloc
, it is more portable to older POSIX
systems that do not support ISO C11.
void *
valloc (size_t size)
¶Preliminary: | MT-Unsafe init | AS-Unsafe init lock | AC-Unsafe init lock fd mem | See POSIX Safety Concepts.
Using valloc
is like using memalign
and passing the page size
as the value of the first argument. It is implemented like this:
void * valloc (size_t size) { return memalign (getpagesize (), size); }
How to get information about the memory subsystem? for more information about the memory subsystem.
The valloc
function is obsolete and aligned_alloc
or
posix_memalign
should be used instead.
You can adjust some parameters for dynamic memory allocation with the
mallopt
function. This function is the general SVID/XPG
interface, defined in malloc.h.
int
mallopt (int param, int value)
¶Preliminary: | MT-Unsafe init const:mallopt | AS-Unsafe init lock | AC-Unsafe init lock | See POSIX Safety Concepts.
When calling mallopt
, the param argument specifies the
parameter to be set, and value the new value to be set. Possible
choices for param, as defined in malloc.h, are:
M_MMAP_MAX
¶The maximum number of chunks to allocate with mmap
. Setting this
to zero disables all use of mmap
.
The default value of this parameter is 65536
.
This parameter can also be set for the process at startup by setting the
environment variable MALLOC_MMAP_MAX_
to the desired value.
M_MMAP_THRESHOLD
¶All chunks larger than this value are allocated outside the normal
heap, using the mmap
system call. This way it is guaranteed
that the memory for these chunks can be returned to the system on
free
. Note that requests smaller than this threshold might still
be allocated via mmap
.
If this parameter is not set, the default value is set as 128 KiB and the threshold is adjusted dynamically to suit the allocation patterns of the program. If the parameter is set, the dynamic adjustment is disabled and the value is set statically to the input value.
This parameter can also be set for the process at startup by setting the
environment variable MALLOC_MMAP_THRESHOLD_
to the desired value.
M_PERTURB
¶If non-zero, memory blocks are filled with values depending on some
low order bits of this parameter when they are allocated (except when
allocated by calloc
) and freed. This can be used to debug the
use of uninitialized or freed heap memory. Note that this option does not
guarantee that the freed block will have any specific values. It only
guarantees that the content the block had before it was freed will be
overwritten.
The default value of this parameter is 0
.
This parameter can also be set for the process at startup by setting the
environment variable MALLOC_PERTURB_
to the desired value.
M_TOP_PAD
¶This parameter determines the amount of extra memory to obtain from the system when an arena needs to be extended. It also specifies the number of bytes to retain when shrinking an arena. This provides the necessary hysteresis in heap size such that excessive amounts of system calls can be avoided.
The default value of this parameter is 0
.
This parameter can also be set for the process at startup by setting the
environment variable MALLOC_TOP_PAD_
to the desired value.
M_TRIM_THRESHOLD
¶This is the minimum size (in bytes) of the top-most, releasable chunk that will trigger a system call in order to return memory to the system.
If this parameter is not set, the default value is set as 128 KiB and the threshold is adjusted dynamically to suit the allocation patterns of the program. If the parameter is set, the dynamic adjustment is disabled and the value is set statically to the provided input.
This parameter can also be set for the process at startup by setting the
environment variable MALLOC_TRIM_THRESHOLD_
to the desired value.
M_ARENA_TEST
¶This parameter specifies the number of arenas that can be created before the
test on the limit to the number of arenas is conducted. The value is ignored if
M_ARENA_MAX
is set.
The default value of this parameter is 2 on 32-bit systems and 8 on 64-bit systems.
This parameter can also be set for the process at startup by setting the
environment variable MALLOC_ARENA_TEST
to the desired value.
M_ARENA_MAX
¶This parameter sets the number of arenas to use regardless of the number of cores in the system.
The default value of this tunable is 0
, meaning that the limit on the
number of arenas is determined by the number of CPU cores online. For 32-bit
systems the limit is twice the number of cores online and on 64-bit systems, it
is eight times the number of cores online. Note that the default value is not
derived from the default value of M_ARENA_TEST and is computed independently.
This parameter can also be set for the process at startup by setting the
environment variable MALLOC_ARENA_MAX
to the desired value.
You can ask malloc
to check the consistency of dynamic memory by
using the mcheck
function and preloading the malloc debug library
libc_malloc_debug using the LD_PRELOAD environment variable.
This function is a GNU extension, declared in mcheck.h.
int
mcheck (void (*abortfn) (enum mcheck_status status))
¶Preliminary: | MT-Unsafe race:mcheck const:malloc_hooks | AS-Unsafe corrupt | AC-Unsafe corrupt | See POSIX Safety Concepts.
Calling mcheck
tells malloc
to perform occasional
consistency checks. These will catch things such as writing
past the end of a block that was allocated with malloc
.
The abortfn argument is the function to call when an inconsistency
is found. If you supply a null pointer, then mcheck
uses a
default function which prints a message and calls abort
(see Aborting a Program). The function you supply is called with
one argument, which says what sort of inconsistency was detected; its
type is described below.
It is too late to begin allocation checking once you have allocated
anything with malloc
. So mcheck
does nothing in that
case. The function returns -1
if you call it too late, and
0
otherwise (when it is successful).
The easiest way to arrange to call mcheck
early enough is to use
the option ‘-lmcheck’ when you link your program; then you don’t
need to modify your program source at all. Alternatively you might use
a debugger to insert a call to mcheck
whenever the program is
started, for example these gdb commands will automatically call mcheck
whenever the program starts:
(gdb) break main Breakpoint 1, main (argc=2, argv=0xbffff964) at whatever.c:10 (gdb) command 1 Type commands for when breakpoint 1 is hit, one per line. End with a line saying just "end". >call mcheck(0) >continue >end (gdb) ...
This will however only work if no initialization function of any object
involved calls any of the malloc
functions since mcheck
must be called before the first such function.
enum mcheck_status
mprobe (void *pointer)
¶Preliminary: | MT-Unsafe race:mcheck const:malloc_hooks | AS-Unsafe corrupt | AC-Unsafe corrupt | See POSIX Safety Concepts.
The mprobe
function lets you explicitly check for inconsistencies
in a particular allocated block. You must have already called
mcheck
at the beginning of the program, to do its occasional
checks; calling mprobe
requests an additional consistency check
to be done at the time of the call.
The argument pointer must be a pointer returned by malloc
or realloc
. mprobe
returns a value that says what
inconsistency, if any, was found. The values are described below.
This enumerated type describes what kind of inconsistency was detected in an allocated block, if any. Here are the possible values:
MCHECK_DISABLED
mcheck
was not called before the first allocation.
No consistency checking can be done.
MCHECK_OK
No inconsistency detected.
MCHECK_HEAD
The data immediately before the block was modified. This commonly happens when an array index or pointer is decremented too far.
MCHECK_TAIL
The data immediately after the block was modified. This commonly happens when an array index or pointer is incremented too far.
MCHECK_FREE
The block was already freed.
Another possibility to check for and guard against bugs in the use of
malloc
, realloc
and free
is to set the environment
variable MALLOC_CHECK_
. When MALLOC_CHECK_
is set to a
non-zero value less than 4, a special (less efficient) implementation is
used which is designed to be tolerant against simple errors, such as
double calls of free
with the same argument, or overruns of a
single byte (off-by-one bugs). Not all such errors can be protected
against, however, and memory leaks can result. Like in the case of
mcheck
, one would need to preload the libc_malloc_debug
library to enable MALLOC_CHECK_
functionality. Without this
preloaded library, setting MALLOC_CHECK_
will have no effect.
Any detected heap corruption results in immediate termination of the process.
There is one problem with MALLOC_CHECK_
: in SUID or SGID binaries
it could possibly be exploited since diverging from the normal programs
behavior it now writes something to the standard error descriptor.
Therefore the use of MALLOC_CHECK_
is disabled by default for
SUID and SGID binaries.
So, what’s the difference between using MALLOC_CHECK_
and linking
with ‘-lmcheck’? MALLOC_CHECK_
is orthogonal with respect to
‘-lmcheck’. ‘-lmcheck’ has been added for backward
compatibility. Both MALLOC_CHECK_
and ‘-lmcheck’ should
uncover the same bugs - but using MALLOC_CHECK_
you don’t need to
recompile your application.
malloc
¶You can get information about dynamic memory allocation by calling the
mallinfo2
function. This function and its associated data type
are declared in malloc.h; they are an extension of the standard
SVID/XPG version.
This structure type is used to return information about the dynamic memory allocator. It contains the following members:
size_t arena
This is the total size of memory allocated with sbrk
by
malloc
, in bytes.
size_t ordblks
This is the number of chunks not in use. (The memory allocator
internally gets chunks of memory from the operating system, and then
carves them up to satisfy individual malloc
requests;
see The GNU Allocator.)
size_t smblks
This field is unused.
size_t hblks
This is the total number of chunks allocated with mmap
.
size_t hblkhd
This is the total size of memory allocated with mmap
, in bytes.
size_t usmblks
This field is unused and always 0.
size_t fsmblks
This field is unused.
size_t uordblks
This is the total size of memory occupied by chunks handed out by
malloc
.
size_t fordblks
This is the total size of memory occupied by free (not in use) chunks.
size_t keepcost
This is the size of the top-most releasable chunk that normally borders the end of the heap (i.e., the high end of the virtual address space’s data segment).
struct mallinfo2
mallinfo2 (void)
¶Preliminary: | MT-Unsafe init const:mallopt | AS-Unsafe init lock | AC-Unsafe init lock | See POSIX Safety Concepts.
This function returns information about the current dynamic memory usage
in a structure of type struct mallinfo2
.
malloc
-Related Functions ¶Here is a summary of the functions that work with malloc
:
void *malloc (size_t size)
Allocate a block of size bytes. See Basic Memory Allocation.
void free (void *addr)
Free a block previously allocated by malloc
. See Freeing Memory Allocated with malloc
.
void *realloc (void *addr, size_t size)
Make a block previously allocated by malloc
larger or smaller,
possibly by copying it to a new location. See Changing the Size of a Block.
void *reallocarray (void *ptr, size_t nmemb, size_t size)
Change the size of a block previously allocated by malloc
to
nmemb * size
bytes as with realloc
. See Changing the Size of a Block.
void *calloc (size_t count, size_t eltsize)
Allocate a block of count * eltsize bytes using
malloc
, and set its contents to zero. See Allocating Cleared Space.
void *valloc (size_t size)
Allocate a block of size bytes, starting on a page boundary. See Allocating Aligned Memory Blocks.
void *aligned_alloc (size_t alignment, size_t size)
Allocate a block of size bytes, starting on an address that is a multiple of alignment. See Allocating Aligned Memory Blocks.
int posix_memalign (void **memptr, size_t alignment, size_t size)
Allocate a block of size bytes, starting on an address that is a multiple of alignment. See Allocating Aligned Memory Blocks.
void *memalign (size_t boundary, size_t size)
Allocate a block of size bytes, starting on an address that is a multiple of boundary. See Allocating Aligned Memory Blocks.
int mallopt (int param, int value)
Adjust a tunable parameter. See Malloc Tunable Parameters.
int mcheck (void (*abortfn) (void))
Tell malloc
to perform occasional consistency checks on
dynamically allocated memory, and to call abortfn when an
inconsistency is found. See Heap Consistency Checking.
struct mallinfo2 mallinfo2 (void)
Return information about the current dynamic memory usage.
See Statistics for Memory Allocation with malloc
.
A complicated task when programming with languages which do not use garbage collected dynamic memory allocation is to find memory leaks. Long running programs must ensure that dynamically allocated objects are freed at the end of their lifetime. If this does not happen the system runs out of memory, sooner or later.
The malloc
implementation in the GNU C Library provides some
simple means to detect such leaks and obtain some information to find
the location. To do this the application must be started in a special
mode which is enabled by an environment variable. There are no speed
penalties for the program if the debugging mode is not enabled.
void
mtrace (void)
¶Preliminary: | MT-Unsafe env race:mtrace init | AS-Unsafe init heap corrupt lock | AC-Unsafe init corrupt lock fd mem | See POSIX Safety Concepts.
The mtrace
function provides a way to trace memory allocation
events in the program that calls it. It is disabled by default in the
library and can be enabled by preloading the debugging library
libc_malloc_debug using the LD_PRELOAD
environment
variable.
When the mtrace
function is called it looks for an environment
variable named MALLOC_TRACE
. This variable is supposed to
contain a valid file name. The user must have write access. If the
file already exists it is truncated. If the environment variable is not
set or it does not name a valid file which can be opened for writing
nothing is done. The behavior of malloc
etc. is not changed.
For obvious reasons this also happens if the application is installed
with the SUID or SGID bit set.
If the named file is successfully opened, mtrace
installs special
handlers for the functions malloc
, realloc
, and
free
. From then on, all uses of these functions are traced and
protocolled into the file. There is now of course a speed penalty for all
calls to the traced functions so tracing should not be enabled during normal
use.
This function is a GNU extension and generally not available on other systems. The prototype can be found in mcheck.h.
void
muntrace (void)
¶Preliminary: | MT-Unsafe race:mtrace locale | AS-Unsafe corrupt heap | AC-Unsafe corrupt mem lock fd | See POSIX Safety Concepts.
The muntrace
function can be called after mtrace
was used
to enable tracing the malloc
calls. If no (successful) call of
mtrace
was made muntrace
does nothing.
Otherwise it deinstalls the handlers for malloc
, realloc
,
and free
and then closes the protocol file. No calls are
protocolled anymore and the program runs again at full speed.
This function is a GNU extension and generally not available on other systems. The prototype can be found in mcheck.h.
Even though the tracing functionality does not influence the runtime
behavior of the program it is not a good idea to call mtrace
in
all programs. Just imagine that you debug a program using mtrace
and all other programs used in the debugging session also trace their
malloc
calls. The output file would be the same for all programs
and thus is unusable. Therefore one should call mtrace
only if
compiled for debugging. A program could therefore start like this:
#include <mcheck.h> int main (int argc, char *argv[]) { #ifdef DEBUGGING mtrace (); #endif ... }
This is all that is needed if you want to trace the calls during the
whole runtime of the program. Alternatively you can stop the tracing at
any time with a call to muntrace
. It is even possible to restart
the tracing again with a new call to mtrace
. But this can cause
unreliable results since there may be calls of the functions which are
not called. Please note that not only the application uses the traced
functions, also libraries (including the C library itself) use these
functions.
This last point is also why it is not a good idea to call muntrace
before the program terminates. The libraries are informed about the
termination of the program only after the program returns from
main
or calls exit
and so cannot free the memory they use
before this time.
So the best thing one can do is to call mtrace
as the very first
function in the program and never call muntrace
. So the program
traces almost all uses of the malloc
functions (except those
calls which are executed by constructors of the program or used
libraries).
You know the situation. The program is prepared for debugging and in all debugging sessions it runs well. But once it is started without debugging the error shows up. A typical example is a memory leak that becomes visible only when we turn off the debugging. If you foresee such situations you can still win. Simply use something equivalent to the following little program:
#include <mcheck.h> #include <signal.h> static void enable (int sig) { mtrace (); signal (SIGUSR1, enable); } static void disable (int sig) { muntrace (); signal (SIGUSR2, disable); } int main (int argc, char *argv[]) { ... signal (SIGUSR1, enable); signal (SIGUSR2, disable); ... }
I.e., the user can start the memory debugger any time s/he wants if the
program was started with MALLOC_TRACE
set in the environment.
The output will of course not show the allocations which happened before
the first signal but if there is a memory leak this will show up
nevertheless.
If you take a look at the output it will look similar to this:
= Start [0x8048209] - 0x8064cc8 [0x8048209] - 0x8064ce0 [0x8048209] - 0x8064cf8 [0x80481eb] + 0x8064c48 0x14 [0x80481eb] + 0x8064c60 0x14 [0x80481eb] + 0x8064c78 0x14 [0x80481eb] + 0x8064c90 0x14 = End
What this all means is not really important since the trace file is not
meant to be read by a human. Therefore no attention is given to
readability. Instead there is a program which comes with the GNU C Library
which interprets the traces and outputs a summary in an
user-friendly way. The program is called mtrace
(it is in fact a
Perl script) and it takes one or two arguments. In any case the name of
the file with the trace output must be specified. If an optional
argument precedes the name of the trace file this must be the name of
the program which generated the trace.
drepper$ mtrace tst-mtrace log No memory leaks.
In this case the program tst-mtrace
was run and it produced a
trace file log. The message printed by mtrace
shows there
are no problems with the code, all allocated memory was freed
afterwards.
If we call mtrace
on the example trace given above we would get a
different output:
drepper$ mtrace errlog - 0x08064cc8 Free 2 was never alloc'd 0x8048209 - 0x08064ce0 Free 3 was never alloc'd 0x8048209 - 0x08064cf8 Free 4 was never alloc'd 0x8048209 Memory not freed: ----------------- Address Size Caller 0x08064c48 0x14 at 0x80481eb 0x08064c60 0x14 at 0x80481eb 0x08064c78 0x14 at 0x80481eb 0x08064c90 0x14 at 0x80481eb
We have called mtrace
with only one argument and so the script
has no chance to find out what is meant with the addresses given in the
trace. We can do better:
drepper$ mtrace tst errlog - 0x08064cc8 Free 2 was never alloc'd /home/drepper/tst.c:39 - 0x08064ce0 Free 3 was never alloc'd /home/drepper/tst.c:39 - 0x08064cf8 Free 4 was never alloc'd /home/drepper/tst.c:39 Memory not freed: ----------------- Address Size Caller 0x08064c48 0x14 at /home/drepper/tst.c:33 0x08064c60 0x14 at /home/drepper/tst.c:33 0x08064c78 0x14 at /home/drepper/tst.c:33 0x08064c90 0x14 at /home/drepper/tst.c:33
Suddenly the output makes much more sense and the user can see immediately where the function calls causing the trouble can be found.
Interpreting this output is not complicated. There are at most two
different situations being detected. First, free
was called for
pointers which were never returned by one of the allocation functions.
This is usually a very bad problem and what this looks like is shown in
the first three lines of the output. Situations like this are quite
rare and if they appear they show up very drastically: the program
normally crashes.
The other situation which is much harder to detect are memory leaks. As
you can see in the output the mtrace
function collects all this
information and so can say that the program calls an allocation function
from line 33 in the source file /home/drepper/tst-mtrace.c four
times without freeing this memory before the program terminates.
Whether this is a real problem remains to be investigated.
malloc
¶The GNU C Library supports replacing the built-in malloc
implementation
with a different allocator with the same interface. For dynamically
linked programs, this happens through ELF symbol interposition, either
using shared object dependencies or LD_PRELOAD
. For static
linking, the malloc
replacement library must be linked in before
linking against libc.a
(explicitly or implicitly).
Care must be taken not to use functionality from the GNU C Library that uses
malloc
internally. For example, the fopen
,
opendir
, dlopen
, and pthread_setspecific
functions
currently use the malloc
subsystem internally. If the
replacement malloc
or its dependencies use thread-local storage
(TLS), it must use the initial-exec TLS model, and not one of the
dynamic TLS variants.
Note: Failure to provide a complete set of replacement
functions (that is, all the functions used by the application,
the GNU C Library, and other linked-in libraries) can lead to static linking
failures, and, at run time, to heap corruption and application crashes.
Replacement functions should implement the behavior documented for
their counterparts in the GNU C Library; for example, the replacement
free
should also preserve errno
.
The minimum set of functions which has to be provided by a custom
malloc
is given in the table below.
malloc
free
calloc
realloc
These malloc
-related functions are required for the GNU C Library to
work.1
The malloc
implementation in the GNU C Library provides additional
functionality not used by the library itself, but which is often used by
other system libraries and applications. A general-purpose replacement
malloc
implementation should provide definitions of these
functions, too. Their names are listed in the following table.
aligned_alloc
malloc_usable_size
memalign
posix_memalign
pvalloc
valloc
In addition, very old applications may use the obsolete cfree
function.
Further malloc
-related functions such as mallopt
or
mallinfo2
will not have any effect or return incorrect statistics
when a replacement malloc
is in use. However, failure to replace
these functions typically does not result in crashes or other incorrect
application behavior, but may result in static linking failures.
There are other functions (reallocarray
, strdup
, etc.) in
the GNU C Library that are not listed above but return newly allocated memory to
callers. Replacement of these functions is not supported and may produce
incorrect results. The GNU C Library implementations of these functions call
the replacement allocator functions whenever available, so they will work
correctly with malloc
replacement.
An obstack is a pool of memory containing a stack of objects. You can create any number of separate obstacks, and then allocate objects in specified obstacks. Within each obstack, the last object allocated must always be the first one freed, but distinct obstacks are independent of each other.
Aside from this one constraint of order of freeing, obstacks are totally general: an obstack can contain any number of objects of any size. They are implemented with macros, so allocation is usually very fast as long as the objects are usually small. And the only space overhead per object is the padding needed to start each object on a suitable boundary.
The utilities for manipulating obstacks are declared in the header file obstack.h.
An obstack is represented by a data structure of type struct
obstack
. This structure has a small fixed size; it records the status
of the obstack and how to find the space in which objects are allocated.
It does not contain any of the objects themselves. You should not try
to access the contents of the structure directly; use only the functions
described in this chapter.
You can declare variables of type struct obstack
and use them as
obstacks, or you can allocate obstacks dynamically like any other kind
of object. Dynamic allocation of obstacks allows your program to have a
variable number of different stacks. (You can even allocate an
obstack structure in another obstack, but this is rarely useful.)
All the functions that work with obstacks require you to specify which
obstack to use. You do this with a pointer of type struct obstack
*
. In the following, we often say “an obstack” when strictly
speaking the object at hand is such a pointer.
The objects in the obstack are packed into large blocks called
chunks. The struct obstack
structure points to a chain of
the chunks currently in use.
The obstack library obtains a new chunk whenever you allocate an object
that won’t fit in the previous chunk. Since the obstack library manages
chunks automatically, you don’t need to pay much attention to them, but
you do need to supply a function which the obstack library should use to
get a chunk. Usually you supply a function which uses malloc
directly or indirectly. You must also supply a function to free a chunk.
These matters are described in the following section.
Each source file in which you plan to use the obstack functions must include the header file obstack.h, like this:
#include <obstack.h>
Also, if the source file uses the macro obstack_init
, it must
declare or define two functions or macros that will be called by the
obstack library. One, obstack_chunk_alloc
, is used to allocate
the chunks of memory into which objects are packed. The other,
obstack_chunk_free
, is used to return chunks when the objects in
them are freed. These macros should appear before any use of obstacks
in the source file.
Usually these are defined to use malloc
via the intermediary
xmalloc
(see Unconstrained Allocation). This is done with
the following pair of macro definitions:
#define obstack_chunk_alloc xmalloc #define obstack_chunk_free free
Though the memory you get using obstacks really comes from malloc
,
using obstacks is faster because malloc
is called less often, for
larger blocks of memory. See Obstack Chunks, for full details.
At run time, before the program can use a struct obstack
object
as an obstack, it must initialize the obstack by calling
obstack_init
.
int
obstack_init (struct obstack *obstack-ptr)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Safe mem | See POSIX Safety Concepts.
Initialize obstack obstack-ptr for allocation of objects. This
function calls the obstack’s obstack_chunk_alloc
function. If
allocation of memory fails, the function pointed to by
obstack_alloc_failed_handler
is called. The obstack_init
function always returns 1 (Compatibility notice: Former versions of
obstack returned 0 if allocation failed).
Here are two examples of how to allocate the space for an obstack and initialize it. First, an obstack that is a static variable:
static struct obstack myobstack; ... obstack_init (&myobstack);
Second, an obstack that is itself dynamically allocated:
struct obstack *myobstack_ptr = (struct obstack *) xmalloc (sizeof (struct obstack)); obstack_init (myobstack_ptr);
The value of this variable is a pointer to a function that
obstack
uses when obstack_chunk_alloc
fails to allocate
memory. The default action is to print a message and abort.
You should supply a function that either calls exit
(see Program Termination) or longjmp
(see Non-Local Exits) and doesn’t return.
void my_obstack_alloc_failed (void) ... obstack_alloc_failed_handler = &my_obstack_alloc_failed;
The most direct way to allocate an object in an obstack is with
obstack_alloc
, which is invoked almost like malloc
.
void *
obstack_alloc (struct obstack *obstack-ptr, int size)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Unsafe corrupt mem | See POSIX Safety Concepts.
This allocates an uninitialized block of size bytes in an obstack
and returns its address. Here obstack-ptr specifies which obstack
to allocate the block in; it is the address of the struct obstack
object which represents the obstack. Each obstack function or macro
requires you to specify an obstack-ptr as the first argument.
This function calls the obstack’s obstack_chunk_alloc
function if
it needs to allocate a new chunk of memory; it calls
obstack_alloc_failed_handler
if allocation of memory by
obstack_chunk_alloc
failed.
For example, here is a function that allocates a copy of a string str
in a specific obstack, which is in the variable string_obstack
:
struct obstack string_obstack; char * copystring (char *string) { size_t len = strlen (string) + 1; char *s = (char *) obstack_alloc (&string_obstack, len); memcpy (s, string, len); return s; }
To allocate a block with specified contents, use the function
obstack_copy
, declared like this:
void *
obstack_copy (struct obstack *obstack-ptr, void *address, int size)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Unsafe corrupt mem | See POSIX Safety Concepts.
This allocates a block and initializes it by copying size
bytes of data starting at address. It calls
obstack_alloc_failed_handler
if allocation of memory by
obstack_chunk_alloc
failed.
void *
obstack_copy0 (struct obstack *obstack-ptr, void *address, int size)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Unsafe corrupt mem | See POSIX Safety Concepts.
Like obstack_copy
, but appends an extra byte containing a null
character. This extra byte is not counted in the argument size.
The obstack_copy0
function is convenient for copying a sequence
of characters into an obstack as a null-terminated string. Here is an
example of its use:
char * obstack_savestring (char *addr, int size) { return obstack_copy0 (&myobstack, addr, size); }
Contrast this with the previous example of savestring
using
malloc
(see Basic Memory Allocation).
To free an object allocated in an obstack, use the function
obstack_free
. Since the obstack is a stack of objects, freeing
one object automatically frees all other objects allocated more recently
in the same obstack.
void
obstack_free (struct obstack *obstack-ptr, void *object)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Unsafe corrupt | See POSIX Safety Concepts.
If object is a null pointer, everything allocated in the obstack is freed. Otherwise, object must be the address of an object allocated in the obstack. Then object is freed, along with everything allocated in obstack-ptr since object.
Note that if object is a null pointer, the result is an
uninitialized obstack. To free all memory in an obstack but leave it
valid for further allocation, call obstack_free
with the address
of the first object allocated on the obstack:
obstack_free (obstack_ptr, first_object_allocated_ptr);
Recall that the objects in an obstack are grouped into chunks. When all the objects in a chunk become free, the obstack library automatically frees the chunk (see Preparing for Using Obstacks). Then other obstacks, or non-obstack allocation, can reuse the space of the chunk.
The interfaces for using obstacks may be defined either as functions or as macros, depending on the compiler. The obstack facility works with all C compilers, including both ISO C and traditional C, but there are precautions you must take if you plan to use compilers other than GNU C.
If you are using an old-fashioned non-ISO C compiler, all the obstack “functions” are actually defined only as macros. You can call these macros like functions, but you cannot use them in any other way (for example, you cannot take their address).
Calling the macros requires a special precaution: namely, the first operand (the obstack pointer) may not contain any side effects, because it may be computed more than once. For example, if you write this:
obstack_alloc (get_obstack (), 4);
you will find that get_obstack
may be called several times.
If you use *obstack_list_ptr++
as the obstack pointer argument,
you will get very strange results since the incrementation may occur
several times.
In ISO C, each function has both a macro definition and a function definition. The function definition is used if you take the address of the function without calling it. An ordinary call uses the macro definition by default, but you can request the function definition instead by writing the function name in parentheses, as shown here:
char *x; void *(*funcp) (); /* Use the macro. */ x = (char *) obstack_alloc (obptr, size); /* Call the function. */ x = (char *) (obstack_alloc) (obptr, size); /* Take the address of the function. */ funcp = obstack_alloc;
This is the same situation that exists in ISO C for the standard library functions. See Macro Definitions of Functions.
Warning: When you do use the macros, you must observe the precaution of avoiding side effects in the first operand, even in ISO C.
If you use the GNU C compiler, this precaution is not necessary, because various language extensions in GNU C permit defining the macros so as to compute each argument only once.
Because memory in obstack chunks is used sequentially, it is possible to build up an object step by step, adding one or more bytes at a time to the end of the object. With this technique, you do not need to know how much data you will put in the object until you come to the end of it. We call this the technique of growing objects. The special functions for adding data to the growing object are described in this section.
You don’t need to do anything special when you start to grow an object.
Using one of the functions to add data to the object automatically
starts it. However, it is necessary to say explicitly when the object is
finished. This is done with the function obstack_finish
.
The actual address of the object thus built up is not known until the object is finished. Until then, it always remains possible that you will add so much data that the object must be copied into a new chunk.
While the obstack is in use for a growing object, you cannot use it for ordinary allocation of another object. If you try to do so, the space already added to the growing object will become part of the other object.
void
obstack_blank (struct obstack *obstack-ptr, int size)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Unsafe corrupt mem | See POSIX Safety Concepts.
The most basic function for adding to a growing object is
obstack_blank
, which adds space without initializing it.
void
obstack_grow (struct obstack *obstack-ptr, void *data, int size)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Unsafe corrupt mem | See POSIX Safety Concepts.
To add a block of initialized space, use obstack_grow
, which is
the growing-object analogue of obstack_copy
. It adds size
bytes of data to the growing object, copying the contents from
data.
void
obstack_grow0 (struct obstack *obstack-ptr, void *data, int size)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Unsafe corrupt mem | See POSIX Safety Concepts.
This is the growing-object analogue of obstack_copy0
. It adds
size bytes copied from data, followed by an additional null
character.
void
obstack_1grow (struct obstack *obstack-ptr, char c)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Unsafe corrupt mem | See POSIX Safety Concepts.
To add one character at a time, use the function obstack_1grow
.
It adds a single byte containing c to the growing object.
void
obstack_ptr_grow (struct obstack *obstack-ptr, void *data)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Unsafe corrupt mem | See POSIX Safety Concepts.
Adding the value of a pointer one can use the function
obstack_ptr_grow
. It adds sizeof (void *)
bytes
containing the value of data.
void
obstack_int_grow (struct obstack *obstack-ptr, int data)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Unsafe corrupt mem | See POSIX Safety Concepts.
A single value of type int
can be added by using the
obstack_int_grow
function. It adds sizeof (int)
bytes to
the growing object and initializes them with the value of data.
void *
obstack_finish (struct obstack *obstack-ptr)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Unsafe corrupt | See POSIX Safety Concepts.
When you are finished growing the object, use the function
obstack_finish
to close it off and return its final address.
Once you have finished the object, the obstack is available for ordinary allocation or for growing another object.
This function can return a null pointer under the same conditions as
obstack_alloc
(see Allocation in an Obstack).
When you build an object by growing it, you will probably need to know
afterward how long it became. You need not keep track of this as you grow
the object, because you can find out the length from the obstack just
before finishing the object with the function obstack_object_size
,
declared as follows:
int
obstack_object_size (struct obstack *obstack-ptr)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function returns the current size of the growing object, in bytes.
Remember to call this function before finishing the object.
After it is finished, obstack_object_size
will return zero.
If you have started growing an object and wish to cancel it, you should finish it and then free it, like this:
obstack_free (obstack_ptr, obstack_finish (obstack_ptr));
This has no effect if no object was growing.
You can use obstack_blank
with a negative size argument to make
the current object smaller. Just don’t try to shrink it beyond zero
length—there’s no telling what will happen if you do that.
The usual functions for growing objects incur overhead for checking whether there is room for the new growth in the current chunk. If you are frequently constructing objects in small steps of growth, this overhead can be significant.
You can reduce the overhead by using special “fast growth” functions that grow the object without checking. In order to have a robust program, you must do the checking yourself. If you do this checking in the simplest way each time you are about to add data to the object, you have not saved anything, because that is what the ordinary growth functions do. But if you can arrange to check less often, or check more efficiently, then you make the program faster.
The function obstack_room
returns the amount of room available
in the current chunk. It is declared as follows:
int
obstack_room (struct obstack *obstack-ptr)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This returns the number of bytes that can be added safely to the current growing object (or to an object about to be started) in obstack obstack-ptr using the fast growth functions.
While you know there is room, you can use these fast growth functions for adding data to a growing object:
void
obstack_1grow_fast (struct obstack *obstack-ptr, char c)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Unsafe corrupt mem | See POSIX Safety Concepts.
The function obstack_1grow_fast
adds one byte containing the
character c to the growing object in obstack obstack-ptr.
void
obstack_ptr_grow_fast (struct obstack *obstack-ptr, void *data)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The function obstack_ptr_grow_fast
adds sizeof (void *)
bytes containing the value of data to the growing object in
obstack obstack-ptr.
void
obstack_int_grow_fast (struct obstack *obstack-ptr, int data)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The function obstack_int_grow_fast
adds sizeof (int)
bytes
containing the value of data to the growing object in obstack
obstack-ptr.
void
obstack_blank_fast (struct obstack *obstack-ptr, int size)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The function obstack_blank_fast
adds size bytes to the
growing object in obstack obstack-ptr without initializing them.
When you check for space using obstack_room
and there is not
enough room for what you want to add, the fast growth functions
are not safe. In this case, simply use the corresponding ordinary
growth function instead. Very soon this will copy the object to a
new chunk; then there will be lots of room available again.
So, each time you use an ordinary growth function, check afterward for
sufficient space using obstack_room
. Once the object is copied
to a new chunk, there will be plenty of space again, so the program will
start using the fast growth functions again.
Here is an example:
void add_string (struct obstack *obstack, const char *ptr, int len) { while (len > 0) { int room = obstack_room (obstack); if (room == 0) { /* Not enough room. Add one character slowly, which may copy to a new chunk and make room. */ obstack_1grow (obstack, *ptr++); len--; } else { if (room > len) room = len; /* Add fast as much as we have room for. */ len -= room; while (room-- > 0) obstack_1grow_fast (obstack, *ptr++); } } }
Here are functions that provide information on the current status of allocation in an obstack. You can use them to learn about an object while still growing it.
void *
obstack_base (struct obstack *obstack-ptr)
¶Preliminary: | MT-Safe | AS-Unsafe corrupt | AC-Safe | See POSIX Safety Concepts.
This function returns the tentative address of the beginning of the currently growing object in obstack-ptr. If you finish the object immediately, it will have that address. If you make it larger first, it may outgrow the current chunk—then its address will change!
If no object is growing, this value says where the next object you allocate will start (once again assuming it fits in the current chunk).
void *
obstack_next_free (struct obstack *obstack-ptr)
¶Preliminary: | MT-Safe | AS-Unsafe corrupt | AC-Safe | See POSIX Safety Concepts.
This function returns the address of the first free byte in the current
chunk of obstack obstack-ptr. This is the end of the currently
growing object. If no object is growing, obstack_next_free
returns the same value as obstack_base
.
int
obstack_object_size (struct obstack *obstack-ptr)
¶Preliminary: | MT-Safe race:obstack-ptr | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function returns the size in bytes of the currently growing object. This is equivalent to
obstack_next_free (obstack-ptr) - obstack_base (obstack-ptr)
Each obstack has an alignment boundary; each object allocated in the obstack automatically starts on an address that is a multiple of the specified boundary. By default, this boundary is aligned so that the object can hold any type of data.
To access an obstack’s alignment boundary, use the macro
obstack_alignment_mask
, whose function prototype looks like
this:
int
obstack_alignment_mask (struct obstack *obstack-ptr)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The value is a bit mask; a bit that is 1 indicates that the corresponding bit in the address of an object should be 0. The mask value should be one less than a power of 2; the effect is that all object addresses are multiples of that power of 2. The default value of the mask is a value that allows aligned objects to hold any type of data: for example, if its value is 3, any type of data can be stored at locations whose addresses are multiples of 4. A mask value of 0 means an object can start on any multiple of 1 (that is, no alignment is required).
The expansion of the macro obstack_alignment_mask
is an lvalue,
so you can alter the mask by assignment. For example, this statement:
obstack_alignment_mask (obstack_ptr) = 0;
has the effect of turning off alignment processing in the specified obstack.
Note that a change in alignment mask does not take effect until
after the next time an object is allocated or finished in the
obstack. If you are not growing an object, you can make the new
alignment mask take effect immediately by calling obstack_finish
.
This will finish a zero-length object and then do proper alignment for
the next object.
Obstacks work by allocating space for themselves in large chunks, and then parceling out space in the chunks to satisfy your requests. Chunks are normally 4096 bytes long unless you specify a different chunk size. The chunk size includes 8 bytes of overhead that are not actually used for storing objects. Regardless of the specified size, longer chunks will be allocated when necessary for long objects.
The obstack library allocates chunks by calling the function
obstack_chunk_alloc
, which you must define. When a chunk is no
longer needed because you have freed all the objects in it, the obstack
library frees the chunk by calling obstack_chunk_free
, which you
must also define.
These two must be defined (as macros) or declared (as functions) in each
source file that uses obstack_init
(see Creating Obstacks).
Most often they are defined as macros like this:
#define obstack_chunk_alloc malloc #define obstack_chunk_free free
Note that these are simple macros (no arguments). Macro definitions with
arguments will not work! It is necessary that obstack_chunk_alloc
or obstack_chunk_free
, alone, expand into a function name if it is
not itself a function name.
If you allocate chunks with malloc
, the chunk size should be a
power of 2. The default chunk size, 4096, was chosen because it is long
enough to satisfy many typical requests on the obstack yet short enough
not to waste too much memory in the portion of the last chunk not yet used.
int
obstack_chunk_size (struct obstack *obstack-ptr)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This returns the chunk size of the given obstack.
Since this macro expands to an lvalue, you can specify a new chunk size by assigning it a new value. Doing so does not affect the chunks already allocated, but will change the size of chunks allocated for that particular obstack in the future. It is unlikely to be useful to make the chunk size smaller, but making it larger might improve efficiency if you are allocating many objects whose size is comparable to the chunk size. Here is how to do so cleanly:
if (obstack_chunk_size (obstack_ptr) < new-chunk-size) obstack_chunk_size (obstack_ptr) = new-chunk-size;
Here is a summary of all the functions associated with obstacks. Each
takes the address of an obstack (struct obstack *
) as its first
argument.
void obstack_init (struct obstack *obstack-ptr)
Initialize use of an obstack. See Creating Obstacks.
void *obstack_alloc (struct obstack *obstack-ptr, int size)
Allocate an object of size uninitialized bytes. See Allocation in an Obstack.
void *obstack_copy (struct obstack *obstack-ptr, void *address, int size)
Allocate an object of size bytes, with contents copied from address. See Allocation in an Obstack.
void *obstack_copy0 (struct obstack *obstack-ptr, void *address, int size)
Allocate an object of size+1 bytes, with size of them copied from address, followed by a null character at the end. See Allocation in an Obstack.
void obstack_free (struct obstack *obstack-ptr, void *object)
Free object (and everything allocated in the specified obstack more recently than object). See Freeing Objects in an Obstack.
void obstack_blank (struct obstack *obstack-ptr, int size)
Add size uninitialized bytes to a growing object. See Growing Objects.
void obstack_grow (struct obstack *obstack-ptr, void *address, int size)
Add size bytes, copied from address, to a growing object. See Growing Objects.
void obstack_grow0 (struct obstack *obstack-ptr, void *address, int size)
Add size bytes, copied from address, to a growing object, and then add another byte containing a null character. See Growing Objects.
void obstack_1grow (struct obstack *obstack-ptr, char data-char)
Add one byte containing data-char to a growing object. See Growing Objects.
void *obstack_finish (struct obstack *obstack-ptr)
Finalize the object that is growing and return its permanent address. See Growing Objects.
int obstack_object_size (struct obstack *obstack-ptr)
Get the current size of the currently growing object. See Growing Objects.
void obstack_blank_fast (struct obstack *obstack-ptr, int size)
Add size uninitialized bytes to a growing object without checking that there is enough room. See Extra Fast Growing Objects.
void obstack_1grow_fast (struct obstack *obstack-ptr, char data-char)
Add one byte containing data-char to a growing object without checking that there is enough room. See Extra Fast Growing Objects.
int obstack_room (struct obstack *obstack-ptr)
Get the amount of room now available for growing the current object. See Extra Fast Growing Objects.
int obstack_alignment_mask (struct obstack *obstack-ptr)
The mask used for aligning the beginning of an object. This is an lvalue. See Alignment of Data in Obstacks.
int obstack_chunk_size (struct obstack *obstack-ptr)
The size for allocating chunks. This is an lvalue. See Obstack Chunks.
void *obstack_base (struct obstack *obstack-ptr)
Tentative starting address of the currently growing object. See Status of an Obstack.
void *obstack_next_free (struct obstack *obstack-ptr)
Address just after the end of the currently growing object. See Status of an Obstack.
The function alloca
supports a kind of half-dynamic allocation in
which blocks are allocated dynamically but freed automatically.
Allocating a block with alloca
is an explicit action; you can
allocate as many blocks as you wish, and compute the size at run time. But
all the blocks are freed when you exit the function that alloca
was
called from, just as if they were automatic variables declared in that
function. There is no way to free the space explicitly.
The prototype for alloca
is in stdlib.h. This function is
a BSD extension.
void *
alloca (size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The return value of alloca
is the address of a block of size
bytes of memory, allocated in the stack frame of the calling function.
Do not use alloca
inside the arguments of a function call—you
will get unpredictable results, because the stack space for the
alloca
would appear on the stack in the middle of the space for
the function arguments. An example of what to avoid is foo (x,
alloca (4), y)
.
alloca
Example ¶As an example of the use of alloca
, here is a function that opens
a file name made from concatenating two argument strings, and returns a
file descriptor or minus one signifying failure:
int open2 (char *str1, char *str2, int flags, int mode) { char *name = (char *) alloca (strlen (str1) + strlen (str2) + 1); stpcpy (stpcpy (name, str1), str2); return open (name, flags, mode); }
Here is how you would get the same results with malloc
and
free
:
int open2 (char *str1, char *str2, int flags, int mode) { char *name = malloc (strlen (str1) + strlen (str2) + 1); int desc; if (name == 0) fatal ("virtual memory exceeded"); stpcpy (stpcpy (name, str1), str2); desc = open (name, flags, mode); free (name); return desc; }
As you can see, it is simpler with alloca
. But alloca
has
other, more important advantages, and some disadvantages.
alloca
¶Here are the reasons why alloca
may be preferable to malloc
:
alloca
wastes very little space and is very fast. (It is
open-coded by the GNU C compiler.)
alloca
does not have separate pools for different sizes of
blocks, space used for any size block can be reused for any other size.
alloca
does not cause memory fragmentation.
longjmp
(see Non-Local Exits)
automatically free the space allocated with alloca
when they exit
through the function that called alloca
. This is the most
important reason to use alloca
.
To illustrate this, suppose you have a function
open_or_report_error
which returns a descriptor, like
open
, if it succeeds, but does not return to its caller if it
fails. If the file cannot be opened, it prints an error message and
jumps out to the command level of your program using longjmp
.
Let’s change open2
(see alloca
Example) to use this
subroutine:
int open2 (char *str1, char *str2, int flags, int mode) { char *name = (char *) alloca (strlen (str1) + strlen (str2) + 1); stpcpy (stpcpy (name, str1), str2); return open_or_report_error (name, flags, mode); }
Because of the way alloca
works, the memory it allocates is
freed even when an error occurs, with no special effort required.
By contrast, the previous definition of open2
(which uses
malloc
and free
) would develop a memory leak if it were
changed in this way. Even if you are willing to make more changes to
fix it, there is no easy way to do so.
alloca
¶These are the disadvantages of alloca
in comparison with
malloc
:
alloca
, so it is less
portable. However, a slower emulation of alloca
written in C
is available for use on systems with this deficiency.
In GNU C, you can replace most uses of alloca
with an array of
variable size. Here is how open2
would look then:
int open2 (char *str1, char *str2, int flags, int mode) { char name[strlen (str1) + strlen (str2) + 1]; stpcpy (stpcpy (name, str1), str2); return open (name, flags, mode); }
But alloca
is not always equivalent to a variable-sized array, for
several reasons:
alloca
remains until the end of the function.
alloca
within a loop, allocating an
additional block on each iteration. This is impossible with
variable-sized arrays.
NB: If you mix use of alloca
and variable-sized arrays
within one function, exiting a scope in which a variable-sized array was
declared frees all blocks allocated with alloca
during the
execution of that scope.
The symbols in this section are declared in unistd.h.
You will not normally use the functions in this section, because the functions described in Allocating Storage For Program Data are easier to use. Those are interfaces to a GNU C Library memory allocator that uses the functions below itself. The functions below are simple interfaces to system calls.
int
brk (void *addr)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
brk
sets the high end of the calling process’ data segment to
addr.
The address of the end of a segment is defined to be the address of the last byte in the segment plus 1.
The function has no effect if addr is lower than the low end of the data segment. (This is considered success, by the way.)
The function fails if it would cause the data segment to overlap another segment or exceed the process’ data storage limit (see Limiting Resource Usage).
The function is named for a common historical case where data storage and the stack are in the same segment. Data storage allocation grows upward from the bottom of the segment while the stack grows downward toward it from the top of the segment and the curtain between them is called the break.
The return value is zero on success. On failure, the return value is
-1
and errno
is set accordingly. The following errno
values are specific to this function:
ENOMEM
The request would cause the data segment to overlap another segment or exceed the process’ data storage limit.
void *
sbrk (ptrdiff_t delta)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is the same as brk
except that you specify the new
end of the data segment as an offset delta from the current end
and on success the return value is the address of the resulting end of
the data segment instead of zero.
This means you can use ‘sbrk(0)’ to find out what the current end of the data segment is.
When a page is mapped using mmap
, page protection flags can be
specified using the protection flags argument. See Memory-mapped I/O.
The following flags are available:
PROT_WRITE
¶The memory can be written to.
PROT_READ
¶The memory can be read. On some architectures, this flag implies that
the memory can be executed as well (as if PROT_EXEC
had been
specified at the same time).
PROT_EXEC
¶The memory can be used to store instructions which can then be executed.
On most architectures, this flag implies that the memory can be read (as
if PROT_READ
had been specified).
PROT_NONE
¶This flag must be specified on its own.
The memory is reserved, but cannot be read, written, or executed. If
this flag is specified in a call to mmap
, a virtual memory area
will be set aside for future use in the process, and mmap
calls
without the MAP_FIXED
flag will not use it for subsequent
allocations. For anonymous mappings, the kernel will not reserve any
physical memory for the allocation at the time the mapping is created.
The operating system may keep track of these flags separately even if
the underlying hardware treats them the same for the purposes of access
checking (as happens with PROT_READ
and PROT_EXEC
on some
platforms). On GNU systems, PROT_EXEC
always implies
PROT_READ
, so that users can view the machine code which is
executing on their system.
Inappropriate access will cause a segfault (see Program Error Signals).
After allocation, protection flags can be changed using the
mprotect
function.
int
mprotect (void *address, size_t length, int protection)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
A successful call to the mprotect
function changes the protection
flags of at least length bytes of memory, starting at
address.
address must be aligned to the page size for the mapping. The
system page size can be obtained by calling sysconf
with the
_SC_PAGESIZE
parameter (see Definition of sysconf
). The system
page size is the granularity in which the page protection of anonymous
memory mappings and most file mappings can be changed. Memory which is
mapped from special files or devices may have larger page granularity
than the system page size and may require larger alignment.
length is the number of bytes whose protection flags must be changed. It is automatically rounded up to the next multiple of the system page size.
protection is a combination of the PROT_*
flags described
above.
The mprotect
function returns 0 on success and -1
on failure.
The following errno
error conditions are defined for this
function:
ENOMEM
The system was not able to allocate resources to fulfill the request. This can happen if there is not enough physical memory in the system for the allocation of backing storage. The error can also occur if the new protection flags would cause the memory region to be split from its neighbors, and the process limit for the number of such distinct memory regions would be exceeded.
EINVAL
address is not properly aligned to a page boundary for the mapping, or length (after rounding up to the system page size) is not a multiple of the applicable page size for the mapping, or the combination of flags in protection is not valid.
EACCES
The file for a file-based mapping was not opened with open flags which are compatible with protection.
EPERM
The system security policy does not allow a mapping with the specified
flags. For example, mappings which are both PROT_EXEC
and
PROT_WRITE
at the same time might not be allowed.
If the mprotect
function is used to make a region of memory
inaccessible by specifying the PROT_NONE
protection flag and
access is later restored, the memory retains its previous contents.
On some systems, it may not be possible to specify additional flags which were not present when the mapping was first created. For example, an attempt to make a region of memory executable could fail if the initial protection flags were ‘PROT_READ | PROT_WRITE’.
In general, the mprotect
function can be used to change any
process memory, no matter how it was allocated. However, portable use
of the function requires that it is only used with memory regions
returned by mmap
or mmap64
.
On some systems, further access restrictions can be added to specific pages using memory protection keys. These restrictions work as follows:
pkey_alloc
function, and applied to pages using
pkey_mprotect
.
pkey_set
and pkey_get
functions.
PROT_
* protection flags
set by mprotect
or pkey_mprotect
.
New threads and subprocesses inherit the access restrictions of the current thread. If a protection key is allocated subsequently, existing threads (except the current) will use an unspecified system default for the access restrictions associated with newly allocated keys.
Upon entering a signal handler, the system resets the access restrictions of the current thread so that pages with the default key can be accessed, but the access restrictions for other protection keys are unspecified.
Applications are expected to allocate a key once using
pkey_alloc
, and apply the key to memory regions which need
special protection with pkey_mprotect
:
int key = pkey_alloc (0, PKEY_DISABLE_ACCESS); if (key < 0) /* Perform error checking, including fallback for lack of support. */ ...; /* Apply the key to a special memory region used to store critical data. */ if (pkey_mprotect (region, region_length, PROT_READ | PROT_WRITE, key) < 0) ...; /* Perform error checking (generally fatal). */
If the key allocation fails due to lack of support for memory protection
keys, the pkey_mprotect
call can usually be skipped. In this
case, the region will not be protected by default. It is also possible
to call pkey_mprotect
with a key value of -1, in which
case it will behave in the same way as mprotect
.
After key allocation assignment to memory pages, pkey_set
can be
used to temporarily acquire access to the memory region and relinquish
it again:
if (key >= 0 && pkey_set (key, 0) < 0) ...; /* Perform error checking (generally fatal). */ /* At this point, the current thread has read-write access to the memory region. */ ... /* Revoke access again. */ if (key >= 0 && pkey_set (key, PKEY_DISABLE_ACCESS) < 0) ...; /* Perform error checking (generally fatal). */
In this example, a negative key value indicates that no key had been allocated, which means that the system lacks support for memory protection keys and it is not necessary to change the the access restrictions of the current thread (because it always has access).
Compared to using mprotect
to change the page protection flags,
this approach has two advantages: It is thread-safe in the sense that
the access restrictions are only changed for the current thread, so another
thread which changes its own access restrictions concurrently to gain access
to the mapping will not suddenly see its access restrictions updated. And
pkey_set
typically does not involve a call into the kernel and a
context switch, so it is more efficient.
int
pkey_alloc (unsigned int flags, unsigned int access_restrictions)
¶Preliminary: | MT-Safe | AS-Safe | AC-Unsafe corrupt | See POSIX Safety Concepts.
Allocate a new protection key. The flags argument is reserved and
must be zero. The access_restrictions argument specifies access restrictions
which are applied to the current thread (as if with pkey_set
below). Access restrictions of other threads are not changed.
The function returns the new protection key, a non-negative number, or -1 on error.
The following errno
error conditions are defined for this
function:
ENOSYS
The system does not implement memory protection keys.
EINVAL
The flags argument is not zero.
The access_restrictions argument is invalid.
The system does not implement memory protection keys or runs in a mode in which memory protection keys are disabled.
ENOSPC
All available protection keys already have been allocated.
The system does not implement memory protection keys or runs in a mode in which memory protection keys are disabled.
int
pkey_free (int key)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Deallocate the protection key, so that it can be reused by
pkey_alloc
.
Calling this function does not change the access restrictions of the freed
protection key. The calling thread and other threads may retain access
to it, even if it is subsequently allocated again. For this reason, it
is not recommended to call the pkey_free
function.
ENOSYS
The system does not implement memory protection keys.
EINVAL
The key argument is not a valid protection key.
int
pkey_mprotect (void *address, size_t length, int protection, int key)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Similar to mprotect
, but also set the memory protection key for
the memory region to key
.
Some systems use memory protection keys to emulate certain combinations
of protection flags. Under such circumstances, specifying an
explicit protection key may behave as if additional flags have been
specified in protection, even though this does not happen with the
default protection key. For example, some systems can support
PROT_EXEC
-only mappings only with a default protection key, and
memory with a key which was allocated using pkey_alloc
will still
be readable if PROT_EXEC
is specified without PROT_READ
.
If key is -1, the default protection key is applied to the
mapping, just as if mprotect
had been called.
The pkey_mprotect
function returns 0 on success and
-1 on failure. The same errno
error conditions as for
mprotect
are defined for this function, with the following
addition:
EINVAL
The key argument is not -1 or a valid memory protection
key allocated using pkey_alloc
.
ENOSYS
The system does not implement memory protection keys, and key is not -1.
int
pkey_set (int key, unsigned int access_restrictions)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Change the access restrictions of the current thread for memory pages with the protection key key to access_restrictions. If access_restrictions is zero, no additional access restrictions on top of the page protection flags are applied. Otherwise, access_restrictions is a combination of the following flags:
PKEY_DISABLE_READ
¶Subsequent attempts to read from memory with the specified protection key will fault. At present only AArch64 platforms with enabled Stage 1 permission overlays feature support this type of restriction.
PKEY_DISABLE_WRITE
¶Subsequent attempts to write to memory with the specified protection key will fault.
PKEY_DISABLE_ACCESS
¶Subsequent attempts to write to or read from memory with the specified
protection key will fault. On AArch64 platforms with enabled Stage 1
permission overlays feature this restriction value has the same effect
as combination of PKEY_DISABLE_READ
and PKEY_DISABLE_WRITE
.
PKEY_DISABLE_EXECUTE
¶Subsequent attempts to execute from memory with the specified protection key will fault. At present only AArch64 platforms with enabled Stage 1 permission overlays feature support this type of restriction.
Operations not specified as flags are not restricted. In particular,
this means that the memory region will remain executable if it was
mapped with the PROT_EXEC
protection flag and
PKEY_DISABLE_ACCESS
has been specified.
Calling the pkey_set
function with a protection key which was not
allocated by pkey_alloc
results in undefined behavior. This
means that calling this function on systems which do not support memory
protection keys is undefined.
The pkey_set
function returns 0 on success and -1
on failure.
The following errno
error conditions are defined for this
function:
EINVAL
The system does not support the access restrictions expressed in the access_restrictions argument.
int
pkey_get (int key)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Return the access restrictions of the current thread for memory pages
with protection key key. The return value is zero or a combination of
the PKEY_DISABLE_
* flags; see the pkey_set
function.
The returned value should be checked for presence or absence of specific flags using bitwise operations. Comparing the returned value with any of the flags or their combination using equals will almost certainly fail.
Calling the pkey_get
function with a protection key which was not
allocated by pkey_alloc
results in undefined behavior. This
means that calling this function on systems which do not support memory
protection keys is undefined.
You can tell the system to associate a particular virtual memory page with a real page frame and keep it that way — i.e., cause the page to be paged in if it isn’t already and mark it so it will never be paged out and consequently will never cause a page fault. This is called locking a page.
The functions in this chapter lock and unlock the calling process’ pages.
Because page faults cause paged out pages to be paged in transparently, a process rarely needs to be concerned about locking pages. However, there are two reasons people sometimes are:
A process that needs to lock pages for this reason probably also needs priority among other processes for use of the CPU. See Process CPU Priority And Scheduling.
In some cases, the programmer knows better than the system’s demand paging allocator which pages should remain in real memory to optimize system performance. In this case, locking pages can help.
Be aware that when you lock a page, that’s one fewer page frame that can be used to back other virtual memory (by the same or other processes), which can mean more page faults, which means the system runs more slowly. In fact, if you lock enough memory, some programs may not be able to run at all for lack of real memory.
A memory lock is associated with a virtual page, not a real frame. The paging rule is: If a frame backs at least one locked page, don’t page it out.
Memory locks do not stack. I.e., you can’t lock a particular page twice so that it has to be unlocked twice before it is truly unlocked. It is either locked or it isn’t.
A memory lock persists until the process that owns the memory explicitly unlocks it. (But process termination and exec cause the virtual memory to cease to exist, which you might say means it isn’t locked any more).
Memory locks are not inherited by child processes. (But note that on a modern Unix system, immediately after a fork, the parent’s and the child’s virtual address space are backed by the same real page frames, so the child enjoys the parent’s locks). See Creating a Process.
Because of its ability to impact other processes, only the superuser can lock a page. Any process can unlock its own page.
The system sets limits on the amount of memory a process can have locked and the amount of real memory it can have dedicated to it. See Limiting Resource Usage.
In Linux, locked pages aren’t as locked as you might think. Two virtual pages that are not shared memory can nonetheless be backed by the same real frame. The kernel does this in the name of efficiency when it knows both virtual pages contain identical data, and does it even if one or both of the virtual pages are locked.
But when a process modifies one of those pages, the kernel must get it a separate frame and fill it with the page’s data. This is known as a copy-on-write page fault. It takes a small amount of time and in a pathological case, getting that frame may require I/O.
To make sure this doesn’t happen to your program, don’t just lock the pages. Write to them as well, unless you know you won’t write to them ever. And to make sure you have pre-allocated frames for your stack, enter a scope that declares a C automatic variable larger than the maximum stack size you will need, set it to something, then return from its scope.
The symbols in this section are declared in sys/mman.h. These functions are defined by POSIX.1b, but their availability depends on your kernel. If your kernel doesn’t allow these functions, they exist but always fail. They are available with a Linux kernel.
Portability Note: POSIX.1b requires that when the mlock
and munlock
functions are available, the file unistd.h
define the macro _POSIX_MEMLOCK_RANGE
and the file
limits.h
define the macro PAGESIZE
to be the size of a
memory page in bytes. It requires that when the mlockall
and
munlockall
functions are available, the unistd.h file
define the macro _POSIX_MEMLOCK
. The GNU C Library conforms to
this requirement.
int
mlock (const void *addr, size_t len)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
mlock
locks a range of the calling process’ virtual pages.
The range of memory starts at address addr and is len bytes long. Actually, since you must lock whole pages, it is the range of pages that include any part of the specified range.
When the function returns successfully, each of those pages is backed by (connected to) a real frame (is resident) and is marked to stay that way. This means the function may cause page-ins and have to wait for them.
When the function fails, it does not affect the lock status of any pages.
The return value is zero if the function succeeds. Otherwise, it is
-1
and errno
is set accordingly. errno
values
specific to this function are:
ENOMEM
EPERM
The calling process is not superuser.
EINVAL
len is not positive.
ENOSYS
The kernel does not provide mlock
capability.
int
mlock2 (const void *addr, size_t len, unsigned int flags)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is similar to mlock
. If flags is zero, a
call to mlock2
behaves exactly as the equivalent call to mlock
.
The flags argument must be a combination of zero or more of the following flags:
MLOCK_ONFAULT
¶Only those pages in the specified address range which are already in memory are locked immediately. Additional pages in the range are automatically locked in case of a page fault and allocation of memory.
Like mlock
, mlock2
returns zero on success and -1
on failure, setting errno
accordingly. Additional errno
values defined for mlock2
are:
EINVAL
The specified (non-zero) flags argument is not supported by this system.
You can lock all a process’ memory with mlockall
. You
unlock memory with munlock
or munlockall
.
To avoid all page faults in a C program, you have to use
mlockall
, because some of the memory a program uses is hidden
from the C code, e.g. the stack and automatic variables, and you
wouldn’t know what address to tell mlock
.
int
munlock (const void *addr, size_t len)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
munlock
unlocks a range of the calling process’ virtual pages.
munlock
is the inverse of mlock
and functions completely
analogously to mlock
, except that there is no EPERM
failure.
int
mlockall (int flags)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
mlockall
locks all the pages in a process’ virtual memory address
space, and/or any that are added to it in the future. This includes the
pages of the code, data and stack segment, as well as shared libraries,
user space kernel data, shared memory, and memory mapped files.
flags is a string of single bit flags represented by the following
macros. They tell mlockall
which of its functions you want. All
other bits must be zero.
MCL_CURRENT
¶Lock all pages which currently exist in the calling process’ virtual address space.
MCL_FUTURE
¶Set a mode such that any pages added to the process’ virtual address
space in the future will be locked from birth. This mode does not
affect future address spaces owned by the same process so exec, which
replaces a process’ address space, wipes out MCL_FUTURE
.
See Executing a File.
When the function returns successfully, and you specified
MCL_CURRENT
, all of the process’ pages are backed by (connected
to) real frames (they are resident) and are marked to stay that way.
This means the function may cause page-ins and have to wait for them.
When the process is in MCL_FUTURE
mode because it successfully
executed this function and specified MCL_CURRENT
, any system call
by the process that requires space be added to its virtual address space
fails with errno
= ENOMEM
if locking the additional space
would cause the process to exceed its locked page limit. In the case
that the address space addition that can’t be accommodated is stack
expansion, the stack expansion fails and the kernel sends a
SIGSEGV
signal to the process.
When the function fails, it does not affect the lock status of any pages or the future locking mode.
The return value is zero if the function succeeds. Otherwise, it is
-1
and errno
is set accordingly. errno
values
specific to this function are:
ENOMEM
EPERM
The calling process is not superuser.
EINVAL
Undefined bits in flags are not zero.
ENOSYS
The kernel does not provide mlockall
capability.
You can lock just specific pages with mlock
. You unlock pages
with munlockall
and munlock
.
int
munlockall (void)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
munlockall
unlocks every page in the calling process’ virtual
address space and turns off MCL_FUTURE
future locking mode.
The return value is zero if the function succeeds. Otherwise, it is
-1
and errno
is set accordingly. The only way this
function can fail is for generic reasons that all functions and system
calls can fail, so there are no specific errno
values.
Programs that work with characters and strings often need to classify a character—is it alphabetic, is it a digit, is it whitespace, and so on—and perform case conversion operations on characters. The functions in the header file ctype.h are provided for this purpose.
Since the choice of locale and character set can alter the
classifications of particular character codes, all of these functions
are affected by the current locale. (More precisely, they are affected
by the locale currently selected for character classification—the
LC_CTYPE
category; see Locale Categories.)
The ISO C standard specifies two different sets of functions. The
one set works on char
type characters, the other one on
wchar_t
wide characters (see Introduction to Extended Characters).
This section explains the library functions for classifying characters.
For example, isalpha
is the function to test for an alphabetic
character. It takes one argument, the character to test as an
unsigned char
value, and returns a nonzero integer if the
character is alphabetic, and zero otherwise. You would use it like
this:
if (isalpha ((unsigned char) c)) printf ("The character `%c' is alphabetic.\n", c);
Each of the functions in this section tests for membership in a
particular class of characters; each has a name starting with ‘is’.
Each of them takes one argument, which is a character to test. The
character argument must be in the value range of unsigned char
(0
to 255 for the GNU C Library). On a machine where the char
type is
signed, it may be necessary to cast the argument to unsigned
char
, or mask it with ‘& 0xff’. (On unsigned char
machines, this step is harmless, so portable code should always perform
it.) The ‘is’ functions return an int
which is treated as a
boolean value.
All ‘is’ functions accept the special value EOF
and return
zero. (Note that EOF
must not be cast to unsigned char
for this to work.)
As an extension, the GNU C Library accepts signed char
values as
‘is’ functions arguments in the range -128 to -2, and returns the
result for the corresponding unsigned character. However, as there
might be an actual character corresponding to the EOF
integer
constant, doing so may introduce bugs, and it is recommended to apply
the conversion to the unsigned character range as appropriate.
The attributes of any given character can vary between locales. See Locales and Internationalization, for more information on locales.
These functions are declared in the header file ctype.h.
int
islower (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if c is a lower-case letter. The letter need not be from the Latin alphabet, any alphabet representable is valid.
int
isupper (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if c is an upper-case letter. The letter need not be from the Latin alphabet, any alphabet representable is valid.
int
isalpha (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if c is an alphabetic character (a letter). If
islower
or isupper
is true of a character, then
isalpha
is also true.
In some locales, there may be additional characters for which
isalpha
is true—letters which are neither upper case nor lower
case. But in the standard "C"
locale, there are no such
additional characters.
int
isdigit (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if c is a decimal digit (‘0’ through ‘9’).
int
isalnum (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if c is an alphanumeric character (a letter or
number); in other words, if either isalpha
or isdigit
is
true of a character, then isalnum
is also true.
int
isxdigit (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if c is a hexadecimal digit. Hexadecimal digits include the normal decimal digits ‘0’ through ‘9’ and the letters ‘A’ through ‘F’ and ‘a’ through ‘f’.
int
ispunct (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if c is a punctuation character. This means any printing character that is not alphanumeric or a space character.
int
isspace (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if c is a whitespace character. In the standard
"C"
locale, isspace
returns true for only the standard
whitespace characters:
' '
space
'\f'
formfeed
'\n'
newline
'\r'
carriage return
'\t'
horizontal tab
'\v'
vertical tab
int
isblank (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if c is a blank character; that is, a space or a tab. This function was originally a GNU extension, but was added in ISO C99.
int
isgraph (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if c is a graphic character; that is, a character that has a glyph associated with it. The whitespace characters are not considered graphic.
int
isprint (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if c is a printing character. Printing characters include all the graphic characters, plus the space (‘ ’) character.
int
iscntrl (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if c is a control character (that is, a character that is not a printing character).
int
isascii (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if c is a 7-bit unsigned char
value that fits
into the US/UK ASCII character set. This function is a BSD extension
and is also an SVID extension.
This section explains the library functions for performing conversions
such as case mappings on characters. For example, toupper
converts any character to upper case if possible. If the character
can’t be converted, toupper
returns it unchanged.
These functions take one argument of type int
, which is the
character to convert, and return the converted character as an
int
. If the conversion is not applicable to the argument given,
the argument is returned unchanged.
Compatibility Note: In pre-ISO C dialects, instead of
returning the argument unchanged, these functions may fail when the
argument is not suitable for the conversion. Thus for portability, you
may need to write islower(c) ? toupper(c) : c
rather than just
toupper(c)
.
These functions are declared in the header file ctype.h.
int
tolower (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
If c is an upper-case letter, tolower
returns the corresponding
lower-case letter. If c is not an upper-case letter,
c is returned unchanged.
int
toupper (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
If c is a lower-case letter, toupper
returns the corresponding
upper-case letter. Otherwise c is returned unchanged.
int
toascii (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function converts c to a 7-bit unsigned char
value
that fits into the US/UK ASCII character set, by clearing the high-order
bits. This function is a BSD extension and is also an SVID extension.
int
_tolower (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This is identical to tolower
, and is provided for compatibility
with the SVID. See SVID (The System V Interface Description).
int
_toupper (int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This is identical to toupper
, and is provided for compatibility
with the SVID.
Amendment 1 to ISO C90 defines functions to classify wide
characters. Although the original ISO C90 standard already defined
the type wchar_t
, no functions operating on them were defined.
The general design of the classification functions for wide characters
is more general. It allows extensions to the set of available
classifications, beyond those which are always available. The POSIX
standard specifies how extensions can be made, and this is already
implemented in the GNU C Library implementation of the localedef
program.
The character class functions are normally implemented with bitsets, with a bitset per character. For a given character, the appropriate bitset is read from a table and a test is performed as to whether a certain bit is set. Which bit is tested for is determined by the class.
For the wide character classification functions this is made visible.
There is a type classification type defined, a function to retrieve this
value for a given class, and a function to test whether a given
character is in this class, using the classification value. On top of
this the normal character classification functions as used for
char
objects can be defined.
The wctype_t
can hold a value which represents a character class.
The only defined way to generate such a value is by using the
wctype
function.
This type is defined in wctype.h.
wctype_t
wctype (const char *property)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
wctype
returns a value representing a class of wide
characters which is identified by the string property. Besides
some standard properties each locale can define its own ones. In case
no property with the given name is known for the current locale
selected for the LC_CTYPE
category, the function returns zero.
The properties known in every locale are:
"alnum" | "alpha" | "cntrl" | "digit" |
"graph" | "lower" | "print" | "punct" |
"space" | "upper" | "xdigit" |
This function is declared in wctype.h.
To test the membership of a character to one of the non-standard classes the ISO C standard defines a completely new function.
int
iswctype (wint_t wc, wctype_t desc)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function returns a nonzero value if wc is in the character
class specified by desc. desc must previously be returned
by a successful call to wctype
.
This function is declared in wctype.h.
To make it easier to use the commonly-used classification functions,
they are defined in the C library. There is no need to use
wctype
if the property string is one of the known character
classes. In some situations it is desirable to construct the property
strings, and then it is important that wctype
can also handle the
standard classes.
int
iswalnum (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function returns a nonzero value if wc is an alphanumeric
character (a letter or number); in other words, if either iswalpha
or iswdigit
is true of a character, then iswalnum
is also
true.
This function can be implemented using
iswctype (wc, wctype ("alnum"))
It is declared in wctype.h.
int
iswalpha (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if wc is an alphabetic character (a letter). If
iswlower
or iswupper
is true of a character, then
iswalpha
is also true.
In some locales, there may be additional characters for which
iswalpha
is true—letters which are neither upper case nor lower
case. But in the standard "C"
locale, there are no such
additional characters.
This function can be implemented using
iswctype (wc, wctype ("alpha"))
It is declared in wctype.h.
int
iswcntrl (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if wc is a control character (that is, a character that is not a printing character).
This function can be implemented using
iswctype (wc, wctype ("cntrl"))
It is declared in wctype.h.
int
iswdigit (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if wc is a digit (e.g., ‘0’ through ‘9’). Please note that this function does not only return a nonzero value for decimal digits, but for all kinds of digits. A consequence is that code like the following will not work unconditionally for wide characters:
n = 0; while (iswdigit (*wc)) { n *= 10; n += *wc++ - L'0'; }
This function can be implemented using
iswctype (wc, wctype ("digit"))
It is declared in wctype.h.
int
iswgraph (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if wc is a graphic character; that is, a character that has a glyph associated with it. The whitespace characters are not considered graphic.
This function can be implemented using
iswctype (wc, wctype ("graph"))
It is declared in wctype.h.
int
iswlower (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if wc is a lower-case letter. The letter need not be from the Latin alphabet, any alphabet representable is valid.
This function can be implemented using
iswctype (wc, wctype ("lower"))
It is declared in wctype.h.
int
iswprint (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if wc is a printing character. Printing characters include all the graphic characters, plus the space (‘ ’) character.
This function can be implemented using
iswctype (wc, wctype ("print"))
It is declared in wctype.h.
int
iswpunct (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if wc is a punctuation character. This means any printing character that is not alphanumeric or a space character.
This function can be implemented using
iswctype (wc, wctype ("punct"))
It is declared in wctype.h.
int
iswspace (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if wc is a whitespace character. In the standard
"C"
locale, iswspace
returns true for only the standard
whitespace characters:
L' '
space
L'\f'
formfeed
L'\n'
newline
L'\r'
carriage return
L'\t'
horizontal tab
L'\v'
vertical tab
This function can be implemented using
iswctype (wc, wctype ("space"))
It is declared in wctype.h.
int
iswupper (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if wc is an upper-case letter. The letter need not be from the Latin alphabet, any alphabet representable is valid.
This function can be implemented using
iswctype (wc, wctype ("upper"))
It is declared in wctype.h.
int
iswxdigit (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if wc is a hexadecimal digit. Hexadecimal digits include the normal decimal digits ‘0’ through ‘9’ and the letters ‘A’ through ‘F’ and ‘a’ through ‘f’.
This function can be implemented using
iswctype (wc, wctype ("xdigit"))
It is declared in wctype.h.
The GNU C Library also provides a function which is not defined in the ISO C standard but which is available as a version for single byte characters as well.
int
iswblank (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns true if wc is a blank character; that is, a space or a tab. This function was originally a GNU extension, but was added in ISO C99. It is declared in wchar.h.
The first note is probably not astonishing but still occasionally a
cause of problems. The iswXXX
functions can be implemented
using macros and in fact, the GNU C Library does this. They are still
available as real functions but when the wctype.h header is
included the macros will be used. This is the same as the
char
type versions of these functions.
The second note covers something new. It can be best illustrated by a (real-world) example. The first piece of code is an excerpt from the original code. It is truncated a bit but the intention should be clear.
int is_in_class (int c, const char *class) { if (strcmp (class, "alnum") == 0) return isalnum (c); if (strcmp (class, "alpha") == 0) return isalpha (c); if (strcmp (class, "cntrl") == 0) return iscntrl (c); ... return 0; }
Now, with the wctype
and iswctype
you can avoid the
if
cascades, but rewriting the code as follows is wrong:
int is_in_class (int c, const char *class) { wctype_t desc = wctype (class); return desc ? iswctype ((wint_t) c, desc) : 0; }
The problem is that it is not guaranteed that the wide character representation of a single-byte character can be found using casting. In fact, usually this fails miserably. The correct solution to this problem is to write the code as follows:
int is_in_class (int c, const char *class) { wctype_t desc = wctype (class); return desc ? iswctype (btowc (c), desc) : 0; }
See Converting Single Characters, for more information on btowc
.
Note that this change probably does not improve the performance
of the program a lot since the wctype
function still has to make
the string comparisons. It gets really interesting if the
is_in_class
function is called more than once for the
same class name. In this case the variable desc could be computed
once and reused for all the calls. Therefore the above form of the
function is probably not the final one.
The classification functions are also generalized by the ISO C
standard. Instead of just allowing the two standard mappings, a
locale can contain others. Again, the localedef
program
already supports generating such locale data files.
This data type is defined as a scalar type which can hold a value
representing the locale-dependent character mapping. There is no way to
construct such a value apart from using the return value of the
wctrans
function.
This type is defined in wctype.h.
wctrans_t
wctrans (const char *property)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The wctrans
function has to be used to find out whether a named
mapping is defined in the current locale selected for the
LC_CTYPE
category. If the returned value is non-zero, you can use
it afterwards in calls to towctrans
. If the return value is
zero no such mapping is known in the current locale.
Beside locale-specific mappings there are two mappings which are guaranteed to be available in every locale:
"tolower" | "toupper" |
These functions are declared in wctype.h.
wint_t
towctrans (wint_t wc, wctrans_t desc)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
towctrans
maps the input character wc
according to the rules of the mapping for which desc is a
descriptor, and returns the value it finds. desc must be
obtained by a successful call to wctrans
.
This function is declared in wctype.h.
For the generally available mappings, the ISO C standard defines
convenient shortcuts so that it is not necessary to call wctrans
for them.
wint_t
towlower (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
If wc is an upper-case letter, towlower
returns the corresponding
lower-case letter. If wc is not an upper-case letter,
wc is returned unchanged.
towlower
can be implemented using
towctrans (wc, wctrans ("tolower"))
This function is declared in wctype.h.
wint_t
towupper (wint_t wc)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
If wc is a lower-case letter, towupper
returns the corresponding
upper-case letter. Otherwise wc is returned unchanged.
towupper
can be implemented using
towctrans (wc, wctrans ("toupper"))
This function is declared in wctype.h.
The same warnings given in the last section for the use of the wide
character classification functions apply here. It is not possible to
simply cast a char
type value to a wint_t
and use it as an
argument to towctrans
calls.
Operations on strings (null-terminated byte sequences) are an important part of
many programs. The GNU C Library provides an extensive set of string
utility functions, including functions for copying, concatenating,
comparing, and searching strings. Many of these functions can also
operate on arbitrary regions of storage; for example, the memcpy
function can be used to copy the contents of any kind of array.
It’s fairly common for beginning C programmers to “reinvent the wheel” by duplicating this functionality in their own code, but it pays to become familiar with the library functions and to make use of them, since this offers benefits in maintenance, efficiency, and portability.
For instance, you could easily compare one string to another in two
lines of C code, but if you use the built-in strcmp
function,
you’re less likely to make a mistake. And, since these library
functions are typically highly optimized, your program may run faster
too.
This section is a quick summary of string concepts for beginning C programmers. It describes how strings are represented in C and some common pitfalls. If you are already familiar with this material, you can skip this section.
A string is a null-terminated array of bytes of type char
,
including the terminating null byte. String-valued
variables are usually declared to be pointers of type char *
.
Such variables do not include space for the contents of a string; that has
to be stored somewhere else—in an array variable, a string constant,
or dynamically allocated memory (see Allocating Storage For Program Data). It’s up to
you to store the address of the chosen memory space into the pointer
variable. Alternatively you can store a null pointer in the
pointer variable. The null pointer does not point anywhere, so
attempting to reference the string it points to gets an error.
A multibyte character is a sequence of one or more bytes that
represents a single character using the locale’s encoding scheme; a
null byte always represents the null character. A multibyte
string is a string that consists entirely of multibyte
characters. In contrast, a wide string is a null-terminated
sequence of wchar_t
objects. A wide-string variable is usually
declared to be a pointer of type wchar_t *
, by analogy with
string variables and char *
. See Introduction to Extended Characters.
By convention, the null byte, '\0'
,
marks the end of a string and the null wide character,
L'\0'
, marks the end of a wide string. For example, in
testing to see whether the char *
variable p points to a
null byte marking the end of a string, you can write
!*p
or *p == '\0'
.
A null byte is quite different conceptually from a null pointer,
although both are represented by the integer constant 0
.
A string literal appears in C program source as a multibyte
string between double-quote characters (‘"’). If the
initial double-quote character is immediately preceded by a capital
‘L’ (ell) character (as in L"foo"
), it is a wide string
literal. String literals can also contribute to string
concatenation: "a" "b"
is the same as "ab"
.
For wide strings one can use either
L"a" L"b"
or L"a" "b"
. Modification of string literals is
not allowed by the GNU C compiler, because literals are placed in
read-only storage.
Arrays that are declared const
cannot be modified
either. It’s generally good style to declare non-modifiable string
pointers to be of type const char *
, since this often allows the
C compiler to detect accidental modifications as well as providing some
amount of documentation about what your program intends to do with the
string.
The amount of memory allocated for a byte array may extend past the null byte that marks the end of the string that the array contains. In this document, the term allocated size is always used to refer to the total amount of memory allocated for an array, while the term length refers to the number of bytes up to (but not including) the terminating null byte. Wide strings are similar, except their sizes and lengths count wide characters, not bytes.
A notorious source of program bugs is trying to put more bytes into a string than fit in its allocated size. When writing code that extends strings or moves bytes into a pre-allocated array, you should be very careful to keep track of the length of the string and make explicit checks for overflowing the array. Many of the library functions do not do this for you! Remember also that you need to allocate an extra byte to hold the null byte that marks the end of the string.
Originally strings were sequences of bytes where each byte represented a single character. This is still true today if the strings are encoded using a single-byte character encoding. Things are different if the strings are encoded using a multibyte encoding (for more information on encodings see Introduction to Extended Characters). There is no difference in the programming interface for these two kind of strings; the programmer has to be aware of this and interpret the byte sequences accordingly.
But since there is no separate interface taking care of these
differences the byte-based string functions are sometimes hard to use.
Since the count parameters of these functions specify bytes a call to
memcpy
could cut a multibyte character in the middle and put an
incomplete (and therefore unusable) byte sequence in the target buffer.
To avoid these problems later versions of the ISO C standard introduce a second set of functions which are operating on wide characters (see Introduction to Extended Characters). These functions don’t have the problems the single-byte versions have since every wide character is a legal, interpretable value. This does not mean that cutting wide strings at arbitrary points is without problems. It normally is for alphabet-based languages (except for non-normalized text) but languages based on syllables still have the problem that more than one wide character is necessary to complete a logical unit. This is a higher level problem which the C library functions are not designed to solve. But it is at least good that no invalid byte sequences can be created. Also, the higher level functions can also much more easily operate on wide characters than on multibyte characters so that a common strategy is to use wide characters internally whenever text is more than simply copied.
The remaining of this chapter will discuss the functions for handling wide strings in parallel with the discussion of strings since there is almost always an exact equivalent available.
This chapter describes both functions that work on arbitrary arrays or blocks of memory, and functions that are specific to strings and wide strings.
Functions that operate on arbitrary blocks of memory have names
beginning with ‘mem’ and ‘wmem’ (such as memcpy
and
wmemcpy
) and invariably take an argument which specifies the size
(in bytes and wide characters respectively) of the block of memory to
operate on. The array arguments and return values for these functions
have type void *
or wchar_t *
. As a matter of style, the
elements of the arrays used with the ‘mem’ functions are referred
to as “bytes”. You can pass any kind of pointer to these functions,
and the sizeof
operator is useful in computing the value for the
size argument. Parameters to the ‘wmem’ functions must be of type
wchar_t *
. These functions are not really usable with anything
but arrays of this type.
In contrast, functions that operate specifically on strings and wide
strings have names beginning with ‘str’ and ‘wcs’
respectively (such as strcpy
and wcscpy
) and look for a
terminating null byte or null wide character instead of requiring an explicit
size argument to be passed. (Some of these functions accept a specified
maximum length, but they also check for premature termination.)
The array arguments and return values for these
functions have type char *
and wchar_t *
respectively, and
the array elements are referred to as “bytes” and “wide
characters”.
In many cases, there are both ‘mem’ and ‘str’/‘wcs’ versions of a function. The one that is more appropriate to use depends on the exact situation. When your program is manipulating arbitrary arrays or blocks of storage, then you should always use the ‘mem’ functions. On the other hand, when you are manipulating strings it is usually more convenient to use the ‘str’/‘wcs’ functions, unless you already know the length of the string in advance. The ‘wmem’ functions should be used for wide character arrays with known size.
Some of the memory and string functions take single characters as
arguments. Since a value of type char
is automatically promoted
into a value of type int
when used as a parameter, the functions
are declared with int
as the type of the parameter in question.
In case of the wide character functions the situation is similar: the
parameter type for a single wide character is wint_t
and not
wchar_t
. This would for many implementations not be necessary
since wchar_t
is large enough to not be automatically
promoted, but since the ISO C standard does not require such a
choice of types the wint_t
type is used.
You can get the length of a string using the strlen
function.
This function is declared in the header file string.h.
size_t
strlen (const char *s)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The strlen
function returns the length of the
string s in bytes. (In other words, it returns the offset of the
terminating null byte within the array.)
For example,
strlen ("hello, world") ⇒ 12
When applied to an array, the strlen
function returns
the length of the string stored there, not its allocated size. You can
get the allocated size of the array that holds a string using
the sizeof
operator:
char string[32] = "hello, world"; sizeof (string) ⇒ 32 strlen (string) ⇒ 12
But beware, this will not work unless string is the array itself, not a pointer to it. For example:
char string[32] = "hello, world";
char *ptr = string;
sizeof (string)
⇒ 32
sizeof (ptr)
⇒ 4 /* (on a machine with 4 byte pointers) */
This is an easy mistake to make when you are working with functions that take string arguments; those arguments are always pointers, not arrays.
It must also be noted that for multibyte encoded strings the return
value does not have to correspond to the number of characters in the
string. To get this value the string can be converted to wide
characters and wcslen
can be used or something like the following
code can be used:
/* The input is instring
. The length is expected inn
. */ { mbstate_t t; char *scopy = string; /* In initial state. */ memset (&t, '\0', sizeof (t)); /* Determine number of characters. */ n = mbsrtowcs (NULL, &scopy, strlen (scopy), &t); }
This is cumbersome to do so if the number of characters (as opposed to bytes) is needed often it is better to work with wide characters.
The wide character equivalent is declared in wchar.h.
size_t
wcslen (const wchar_t *ws)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The wcslen
function is the wide character equivalent to
strlen
. The return value is the number of wide characters in the
wide string pointed to by ws (this is also the offset of
the terminating null wide character of ws).
Since there are no multi wide character sequences making up one wide character the return value is not only the offset in the array, it is also the number of wide characters.
This function was introduced in Amendment 1 to ISO C90.
size_t
strnlen (const char *s, size_t maxlen)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This returns the offset of the first null byte in the array s,
except that it returns maxlen if the first maxlen bytes
are all non-null.
Therefore this function is equivalent to
(strlen (s) < maxlen ? strlen (s) : maxlen)
but it
is more efficient and works even if s is not null-terminated so
long as maxlen does not exceed the size of s’s array.
char string[32] = "hello, world"; strnlen (string, 32) ⇒ 12 strnlen (string, 5) ⇒ 5
This function is part of POSIX.1-2008 and later editions, but was available in the GNU C Library and other systems as an extension long before it was standardized. It is declared in string.h.
size_t
wcsnlen (const wchar_t *ws, size_t maxlen)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
wcsnlen
is the wide character equivalent to strnlen
. The
maxlen parameter specifies the maximum number of wide characters.
This function is part of POSIX.1-2008 and later editions, and is declared in wchar.h.
You can use the functions described in this section to copy the contents of strings, wide strings, and arrays. The ‘str’ and ‘mem’ functions are declared in string.h while the ‘w’ functions are declared in wchar.h.
A helpful way to remember the ordering of the arguments to the functions in this section is that it corresponds to an assignment expression, with the destination array specified to the left of the source array. Most of these functions return the address of the destination array; a few return the address of the destination’s terminating null, or of just past the destination.
Most of these functions do not work properly if the source and destination arrays overlap. For example, if the beginning of the destination array overlaps the end of the source array, the original contents of that part of the source array may get overwritten before it is copied. Even worse, in the case of the string functions, the null byte marking the end of the string may be lost, and the copy function might get stuck in a loop trashing all the memory allocated to your program.
All functions that have problems copying between overlapping arrays are
explicitly identified in this manual. In addition to functions in this
section, there are a few others like sprintf
(see Formatted Output Functions) and scanf
(see Formatted Input Functions).
void *
memcpy (void *restrict to, const void *restrict from, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The memcpy
function copies size bytes from the object
beginning at from into the object beginning at to. The
behavior of this function is undefined if the two arrays to and
from overlap; use memmove
instead if overlapping is possible.
The value returned by memcpy
is the value of to.
Here is an example of how you might use memcpy
to copy the
contents of an array:
struct foo *oldarray, *newarray; int arraysize; ... memcpy (new, old, arraysize * sizeof (struct foo));
wchar_t *
wmemcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The wmemcpy
function copies size wide characters from the object
beginning at wfrom into the object beginning at wto. The
behavior of this function is undefined if the two arrays wto and
wfrom overlap; use wmemmove
instead if overlapping is possible.
The following is a possible implementation of wmemcpy
but there
are more optimizations possible.
wchar_t * wmemcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size) { return (wchar_t *) memcpy (wto, wfrom, size * sizeof (wchar_t)); }
The value returned by wmemcpy
is the value of wto.
This function was introduced in Amendment 1 to ISO C90.
void *
mempcpy (void *restrict to, const void *restrict from, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The mempcpy
function is nearly identical to the memcpy
function. It copies size bytes from the object beginning at
from
into the object pointed to by to. But instead of
returning the value of to it returns a pointer to the byte
following the last written byte in the object beginning at to.
I.e., the value is ((void *) ((char *) to + size))
.
This function is useful in situations where a number of objects shall be copied to consecutive memory positions.
void * combine (void *o1, size_t s1, void *o2, size_t s2) { void *result = malloc (s1 + s2); if (result != NULL) mempcpy (mempcpy (result, o1, s1), o2, s2); return result; }
This function is a GNU extension.
wchar_t *
wmempcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The wmempcpy
function is nearly identical to the wmemcpy
function. It copies size wide characters from the object
beginning at wfrom
into the object pointed to by wto. But
instead of returning the value of wto it returns a pointer to the
wide character following the last written wide character in the object
beginning at wto. I.e., the value is wto + size
.
This function is useful in situations where a number of objects shall be copied to consecutive memory positions.
The following is a possible implementation of wmemcpy
but there
are more optimizations possible.
wchar_t * wmempcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size) { return (wchar_t *) mempcpy (wto, wfrom, size * sizeof (wchar_t)); }
This function is a GNU extension.
void *
memmove (void *to, const void *from, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
memmove
copies the size bytes at from into the
size bytes at to, even if those two blocks of space
overlap. In the case of overlap, memmove
is careful to copy the
original values of the bytes in the block at from, including those
bytes which also belong to the block at to.
The value returned by memmove
is the value of to.
wchar_t *
wmemmove (wchar_t *wto, const wchar_t *wfrom, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
wmemmove
copies the size wide characters at wfrom
into the size wide characters at wto, even if those two
blocks of space overlap. In the case of overlap, wmemmove
is
careful to copy the original values of the wide characters in the block
at wfrom, including those wide characters which also belong to the
block at wto.
The following is a possible implementation of wmemcpy
but there
are more optimizations possible.
wchar_t * wmempcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size) { return (wchar_t *) mempcpy (wto, wfrom, size * sizeof (wchar_t)); }
The value returned by wmemmove
is the value of wto.
This function is a GNU extension.
void *
memccpy (void *restrict to, const void *restrict from, int c, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function copies no more than size bytes from from to to, stopping if a byte matching c is found. The return value is a pointer into to one byte past where c was copied, or a null pointer if no byte matching c appeared in the first size bytes of from.
void *
memset (void *block, int c, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function copies the value of c (converted to an
unsigned char
) into each of the first size bytes of the
object beginning at block. It returns the value of block.
wchar_t *
wmemset (wchar_t *block, wchar_t wc, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function copies the value of wc into each of the first size wide characters of the object beginning at block. It returns the value of block.
char *
strcpy (char *restrict to, const char *restrict from)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This copies bytes from the string from (up to and including
the terminating null byte) into the string to. Like
memcpy
, this function has undefined results if the strings
overlap. The return value is the value of to.
wchar_t *
wcscpy (wchar_t *restrict wto, const wchar_t *restrict wfrom)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This copies wide characters from the wide string wfrom (up to and
including the terminating null wide character) into the string
wto. Like wmemcpy
, this function has undefined results if
the strings overlap. The return value is the value of wto.
char *
strdup (const char *s)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
This function copies the string s into a newly
allocated string. The string is allocated using malloc
; see
Unconstrained Allocation. If malloc
cannot allocate space
for the new string, strdup
returns a null pointer. Otherwise it
returns a pointer to the new string.
wchar_t *
wcsdup (const wchar_t *ws)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
This function copies the wide string ws
into a newly allocated string. The string is allocated using
malloc
; see Unconstrained Allocation. If malloc
cannot allocate space for the new string, wcsdup
returns a null
pointer. Otherwise it returns a pointer to the new wide string.
This function is a GNU extension.
char *
stpcpy (char *restrict to, const char *restrict from)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is like strcpy
, except that it returns a pointer to
the end of the string to (that is, the address of the terminating
null byte to + strlen (from)
) rather than the beginning.
For example, this program uses stpcpy
to concatenate ‘foo’
and ‘bar’ to produce ‘foobar’, which it then prints.
#include <string.h> #include <stdio.h> int main (void) { char buffer[10]; char *to = buffer; to = stpcpy (to, "foo"); to = stpcpy (to, "bar"); puts (buffer); return 0; }
This function is part of POSIX.1-2008 and later editions, but was available in the GNU C Library and other systems as an extension long before it was standardized.
Its behavior is undefined if the strings overlap. The function is declared in string.h.
wchar_t *
wcpcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is like wcscpy
, except that it returns a pointer to
the end of the string wto (that is, the address of the terminating
null wide character wto + wcslen (wfrom)
) rather than the beginning.
This function is not part of ISO or POSIX but was found useful while developing the GNU C Library itself.
The behavior of wcpcpy
is undefined if the strings overlap.
wcpcpy
is a GNU extension and is declared in wchar.h.
char *
strdupa (const char *s)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This macro is similar to strdup
but allocates the new string
using alloca
instead of malloc
(see Automatic Storage with Variable Size). This means of course the returned string has the same
limitations as any block of memory allocated using alloca
.
For obvious reasons strdupa
is implemented only as a macro;
you cannot get the address of this function. Despite this limitation
it is a useful function. The following code shows a situation where
using malloc
would be a lot more expensive.
#include <paths.h> #include <string.h> #include <stdio.h> const char path[] = _PATH_STDPATH; int main (void) { char *wr_path = strdupa (path); char *cp = strtok (wr_path, ":"); while (cp != NULL) { puts (cp); cp = strtok (NULL, ":"); } return 0; }
Please note that calling strtok
using path directly is
invalid. It is also not allowed to call strdupa
in the argument
list of strtok
since strdupa
uses alloca
(see Automatic Storage with Variable Size) can interfere with the parameter
passing.
This function is only available if GNU CC is used.
void
bcopy (const void *from, void *to, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This is a partially obsolete alternative for memmove
, derived from
BSD. Note that it is not quite equivalent to memmove
, because the
arguments are not in the same order and there is no return value.
void
bzero (void *block, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This is a partially obsolete alternative for memset
, derived from
BSD. Note that it is not as general as memset
, because the only
value it can store is zero.
The functions described in this section concatenate the contents of a string or wide string to another. They follow the string-copying functions in their conventions. See Copying Strings and Arrays. ‘strcat’ is declared in the header file string.h while ‘wcscat’ is declared in wchar.h.
As noted below, these functions are problematic as their callers may have performance issues.
char *
strcat (char *restrict to, const char *restrict from)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The strcat
function is similar to strcpy
, except that the
bytes from from are concatenated or appended to the end of
to, instead of overwriting it. That is, the first byte from
from overwrites the null byte marking the end of to.
An equivalent definition for strcat
would be:
char * strcat (char *restrict to, const char *restrict from) { strcpy (to + strlen (to), from); return to; }
This function has undefined results if the strings overlap.
As noted below, this function has significant performance issues.
wchar_t *
wcscat (wchar_t *restrict wto, const wchar_t *restrict wfrom)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The wcscat
function is similar to wcscpy
, except that the
wide characters from wfrom are concatenated or appended to the end of
wto, instead of overwriting it. That is, the first wide character from
wfrom overwrites the null wide character marking the end of wto.
An equivalent definition for wcscat
would be:
wchar_t * wcscat (wchar_t *wto, const wchar_t *wfrom) { wcscpy (wto + wcslen (wto), wfrom); return wto; }
This function has undefined results if the strings overlap.
As noted below, this function has significant performance issues.
Programmers using the strcat
or wcscat
functions (or the
strlcat
, strncat
and wcsncat
functions defined in
a later section, for that matter)
can easily be recognized as lazy and reckless. In almost all situations
the lengths of the participating strings are known (it better should be
since how can one otherwise ensure the allocated size of the buffer is
sufficient?) Or at least, one could know them if one keeps track of the
results of the various function calls. But then it is very inefficient
to use strcat
/wcscat
. A lot of time is wasted finding the
end of the destination string so that the actual copying can start.
This is a common example:
/* This function concatenates arbitrarily many strings. The last
parameter must be NULL
. */
char *
concat (const char *str, ...)
{
va_list ap, ap2;
size_t total = 1;
va_start (ap, str);
va_copy (ap2, ap);
/* Determine how much space we need. */
for (const char *s = str; s != NULL; s = va_arg (ap, const char *))
total += strlen (s);
va_end (ap);
char *result = malloc (total);
if (result != NULL)
{
result[0] = '\0';
/* Copy the strings. */
for (s = str; s != NULL; s = va_arg (ap2, const char *))
strcat (result, s);
}
va_end (ap2);
return result;
}
This looks quite simple, especially the second loop where the strings are actually copied. But these innocent lines hide a major performance penalty. Just imagine that ten strings of 100 bytes each have to be concatenated. For the second string we search the already stored 100 bytes for the end of the string so that we can append the next string. For all strings in total the comparisons necessary to find the end of the intermediate results sums up to 5500! If we combine the copying with the search for the allocation we can write this function more efficiently:
char * concat (const char *str, ...) { size_t allocated = 100; char *result = malloc (allocated); if (result != NULL) { va_list ap; size_t resultlen = 0; char *newp; va_start (ap, str); for (const char *s = str; s != NULL; s = va_arg (ap, const char *)) { size_t len = strlen (s); /* Resize the allocated memory if necessary. */ if (resultlen + len + 1 > allocated) { allocated += len; newp = reallocarray (result, allocated, 2); allocated *= 2; if (newp == NULL) { free (result); return NULL; } result = newp; } memcpy (result + resultlen, s, len); resultlen += len; } /* Terminate the result string. */ result[resultlen++] = '\0'; /* Resize memory to the optimal size. */ newp = realloc (result, resultlen); if (newp != NULL) result = newp; va_end (ap); } return result; }
With a bit more knowledge about the input strings one could fine-tune
the memory allocation. The difference we are pointing to here is that
we don’t use strcat
anymore. We always keep track of the length
of the current intermediate result so we can save ourselves the search for the
end of the string and use mempcpy
. Please note that we also
don’t use stpcpy
which might seem more natural since we are handling
strings. But this is not necessary since we already know the
length of the string and therefore can use the faster memory copying
function. The example would work for wide characters the same way.
Whenever a programmer feels the need to use strcat
she or he
should think twice and look through the program to see whether the code cannot
be rewritten to take advantage of already calculated results.
The related functions strlcat
, strncat
,
wcscat
and wcsncat
are almost always unnecessary, too.
Again: it is almost always unnecessary to use functions like strcat
.
The functions described in this section copy or concatenate the possibly-truncated contents of a string or array to another, and similarly for wide strings. They follow the string-copying functions in their header conventions. See Copying Strings and Arrays. The ‘str’ functions are declared in the header file string.h and the ‘wc’ functions are declared in the file wchar.h.
As noted below, these functions are problematic as their callers may have truncation-related bugs and performance issues.
char *
strncpy (char *restrict to, const char *restrict from, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is similar to strcpy
but always copies exactly
size bytes into to.
If from does not contain a null byte in its first size
bytes, strncpy
copies just the first size bytes. In this
case no null terminator is written into to.
Otherwise from must be a string with length less than
size. In this case strncpy
copies all of from,
followed by enough null bytes to add up to size bytes in all.
The behavior of strncpy
is undefined if the strings overlap.
This function was designed for now-rarely-used arrays consisting of non-null bytes followed by zero or more null bytes. It needs to set all size bytes of the destination, even when size is much greater than the length of from. As noted below, this function is generally a poor choice for processing strings.
wchar_t *
wcsncpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is similar to wcscpy
but always copies exactly
size wide characters into wto.
If wfrom does not contain a null wide character in its first
size wide characters, then wcsncpy
copies just the first
size wide characters. In this case no null terminator is
written into wto.
Otherwise wfrom must be a wide string with length less than
size. In this case wcsncpy
copies all of wfrom,
followed by enough null wide characters to add up to size wide
characters in all.
The behavior of wcsncpy
is undefined if the strings overlap.
This function is the wide-character counterpart of strncpy
and
suffers from most of the problems that strncpy
does. For
example, as noted below, this function is generally a poor choice for
processing strings.
char *
strndup (const char *s, size_t size)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
This function is similar to strdup
but always copies at most
size bytes into the newly allocated string.
If the length of s is more than size, then strndup
copies just the first size bytes and adds a closing null byte.
Otherwise all bytes are copied and the string is terminated.
This function differs from strncpy
in that it always terminates
the destination string.
As noted below, this function is generally a poor choice for processing strings.
strndup
is a GNU extension.
char *
strndupa (const char *s, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is similar to strndup
but like strdupa
it
allocates the new string using alloca
see Automatic Storage with Variable Size. The same advantages and limitations of strdupa
are
valid for strndupa
, too.
This function is implemented only as a macro, just like strdupa
.
Just as strdupa
this macro also must not be used inside the
parameter list in a function call.
As noted below, this function is generally a poor choice for processing strings.
strndupa
is only available if GNU CC is used.
char *
stpncpy (char *restrict to, const char *restrict from, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is similar to stpcpy
but copies always exactly
size bytes into to.
If the length of from is more than size, then stpncpy
copies just the first size bytes and returns a pointer to the
byte directly following the one which was copied last. Note that in
this case there is no null terminator written into to.
If the length of from is less than size, then stpncpy
copies all of from, followed by enough null bytes to add up
to size bytes in all. This behavior is rarely useful, but it
is implemented to be useful in contexts where this behavior of the
strncpy
is used. stpncpy
returns a pointer to the
first written null byte.
This function is not part of ISO or POSIX but was found useful while developing the GNU C Library itself.
Its behavior is undefined if the strings overlap. The function is declared in string.h.
As noted below, this function is generally a poor choice for processing strings.
wchar_t *
wcpncpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is similar to wcpcpy
but copies always exactly
wsize wide characters into wto.
If the length of wfrom is more than size, then
wcpncpy
copies just the first size wide characters and
returns a pointer to the wide character directly following the last
non-null wide character which was copied last. Note that in this case
there is no null terminator written into wto.
If the length of wfrom is less than size, then wcpncpy
copies all of wfrom, followed by enough null wide characters to add up
to size wide characters in all. This behavior is rarely useful, but it
is implemented to be useful in contexts where this behavior of the
wcsncpy
is used. wcpncpy
returns a pointer to the
first written null wide character.
This function is not part of ISO or POSIX but was found useful while developing the GNU C Library itself.
Its behavior is undefined if the strings overlap.
As noted below, this function is generally a poor choice for processing strings.
wcpncpy
is a GNU extension.
char *
strncat (char *restrict to, const char *restrict from, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is like strcat
except that not more than size
bytes from from are appended to the end of to, and
from need not be null-terminated. A single null byte is also
always appended to to, so the total
allocated size of to must be at least size + 1
bytes
longer than its initial length.
The strncat
function could be implemented like this:
char * strncat (char *to, const char *from, size_t size) { size_t len = strlen (to); memcpy (to + len, from, strnlen (from, size)); to[len + strnlen (from, size)] = '\0'; return to; }
The behavior of strncat
is undefined if the strings overlap.
As a companion to strncpy
, strncat
was designed for
now-rarely-used arrays consisting of non-null bytes followed by zero
or more null bytes. However, As noted below, this function is generally a poor
choice for processing strings. Also, this function has significant
performance issues. See Concatenating Strings.
wchar_t *
wcsncat (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is like wcscat
except that not more than size
wide characters from from are appended to the end of to,
and from need not be null-terminated. A single null wide
character is also always appended to to, so the total allocated
size of to must be at least wcsnlen (wfrom,
size) + 1
wide characters longer than its initial length.
The wcsncat
function could be implemented like this:
wchar_t * wcsncat (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size) { size_t len = wcslen (wto); memcpy (wto + len, wfrom, wcsnlen (wfrom, size) * sizeof (wchar_t)); wto[len + wcsnlen (wfrom, size)] = L'\0'; return wto; }
The behavior of wcsncat
is undefined if the strings overlap.
As noted below, this function is generally a poor choice for processing strings. Also, this function has significant performance issues. See Concatenating Strings.
size_t
strlcpy (char *restrict to, const char *restrict from, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function copies the string from to the destination array to, limiting the result’s size (including the null terminator) to size. The caller should ensure that size includes room for the result’s terminating null byte.
If size is greater than the length of the string from,
this function copies the non-null bytes of the string
from to the destination array to,
and terminates the copy with a null byte. Like other
string functions such as strcpy
, but unlike strncpy
, any
remaining bytes in the destination array remain unchanged.
If size is nonzero and less than or equal to the the length of the string from, this function copies only the first ‘size - 1’ bytes to the destination array to, and writes a terminating null byte to the last byte of the array.
This function returns the length of the string from. This means that truncation occurs if and only if the returned value is greater than or equal to size.
The behavior is undefined if to or from is a null pointer, or if the destination array’s size is less than size, or if the string from overlaps the first size bytes of the destination array.
As noted below, this function is generally a poor choice for processing strings. Also, this function has a performance issue, as its time cost is proportional to the length of from even when size is small.
This function is derived from OpenBSD 2.4.
size_t
wcslcpy (wchar_t *restrict to, const wchar_t *restrict from, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is a variant of strlcpy
for wide strings.
The size argument counts the length of the destination buffer in
wide characters (and not bytes).
This function is derived from BSD.
size_t
strlcat (char *restrict to, const char *restrict from, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function appends the string from to the string to, limiting the result’s total size (including the null terminator) to size. The caller should ensure that size includes room for the result’s terminating null byte.
This function copies as much as possible of the string from into the array at to of size bytes, starting at the terminating null byte of the original string to. In effect, this appends the string from to the string to. Although the resulting string will contain a null terminator, it can be truncated (not all bytes in from may be copied).
This function returns the sum of the original length of to and the length of from. This means that truncation occurs if and only if the returned value is greater than or equal to size.
The behavior is undefined if to or from is a null pointer, or if the destination array’s size is less than size, or if the destination array does not contain a null byte in its first size bytes, or if the string from overlaps the first size bytes of the destination array.
As noted below, this function is generally a poor choice for processing strings. Also, this function has significant performance issues. See Concatenating Strings.
This function is derived from OpenBSD 2.4.
size_t
wcslcat (wchar_t *restrict to, const wchar_t *restrict from, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is a variant of strlcat
for wide strings.
The size argument counts the length of the destination buffer in
wide characters (and not bytes).
This function is derived from BSD.
Because these functions can abruptly truncate strings or wide strings, they are generally poor choices for processing them. When copying or concatening multibyte strings, they can truncate within a multibyte character so that the result is not a valid multibyte string. When combining or concatenating multibyte or wide strings, they may truncate the output after a combining character, resulting in a corrupted grapheme. They can cause bugs even when processing single-byte strings: for example, when calculating an ASCII-only user name, a truncated name can identify the wrong user.
Although some buffer overruns can be prevented by manually replacing calls to copying functions with calls to truncation functions, there are often easier and safer automatic techniques, such as fortification (see Fortification of function calls) and AddressSanitizer (see Program Instrumentation Options in Using GCC). Because truncation functions can mask application bugs that would otherwise be caught by the automatic techniques, these functions should be used only when the application’s underlying logic requires truncation.
Note: GNU programs should not truncate strings or wide
strings to fit arbitrary size limits. See Writing
Robust Programs in The GNU Coding Standards. Instead of
string-truncation functions, it is usually better to use dynamic
memory allocation (see Unconstrained Allocation) and functions
such as strdup
or asprintf
to construct strings.
You can use the functions in this section to perform comparisons on the contents of strings and arrays. As well as checking for equality, these functions can also be used as the ordering functions for sorting operations. See Searching and Sorting, for an example of this.
Unlike most comparison operations in C, the string comparison functions return a nonzero value if the strings are not equivalent rather than if they are. The sign of the value indicates the relative ordering of the first part of the strings that are not equivalent: a negative value indicates that the first string is “less” than the second, while a positive value indicates that the first string is “greater”.
The most common use of these functions is to check only for equality. This is canonically done with an expression like ‘! strcmp (s1, s2)’.
All of these functions are declared in the header file string.h.
int
memcmp (const void *a1, const void *a2, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The function memcmp
compares the size bytes of memory
beginning at a1 against the size bytes of memory beginning
at a2. The value returned has the same sign as the difference
between the first differing pair of bytes (interpreted as unsigned
char
objects, then promoted to int
).
If the contents of the two blocks are equal, memcmp
returns
0
.
int
wmemcmp (const wchar_t *a1, const wchar_t *a2, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The function wmemcmp
compares the size wide characters
beginning at a1 against the size wide characters beginning
at a2. The value returned is smaller than or larger than zero
depending on whether the first differing wide character is a1 is
smaller or larger than the corresponding wide character in a2.
If the contents of the two blocks are equal, wmemcmp
returns
0
.
On arbitrary arrays, the memcmp
function is mostly useful for
testing equality. It usually isn’t meaningful to do byte-wise ordering
comparisons on arrays of things other than bytes. For example, a
byte-wise comparison on the bytes that make up floating-point numbers
isn’t likely to tell you anything about the relationship between the
values of the floating-point numbers.
wmemcmp
is really only useful to compare arrays of type
wchar_t
since the function looks at sizeof (wchar_t)
bytes
at a time and this number of bytes is system dependent.
You should also be careful about using memcmp
to compare objects
that can contain “holes”, such as the padding inserted into structure
objects to enforce alignment requirements, extra space at the end of
unions, and extra bytes at the ends of strings whose length is less
than their allocated size. The contents of these “holes” are
indeterminate and may cause strange behavior when performing byte-wise
comparisons. For more predictable results, perform an explicit
component-wise comparison.
For example, given a structure type definition like:
struct foo { unsigned char tag; union { double f; long i; char *p; } value; };
you are better off writing a specialized comparison function to compare
struct foo
objects instead of comparing them with memcmp
.
int
strcmp (const char *s1, const char *s2)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The strcmp
function compares the string s1 against
s2, returning a value that has the same sign as the difference
between the first differing pair of bytes (interpreted as
unsigned char
objects, then promoted to int
).
If the two strings are equal, strcmp
returns 0
.
A consequence of the ordering used by strcmp
is that if s1
is an initial substring of s2, then s1 is considered to be
“less than” s2.
strcmp
does not take sorting conventions of the language the
strings are written in into account. To get that one has to use
strcoll
.
int
wcscmp (const wchar_t *ws1, const wchar_t *ws2)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The wcscmp
function compares the wide string ws1
against ws2. The value returned is smaller than or larger than zero
depending on whether the first differing wide character is ws1 is
smaller or larger than the corresponding wide character in ws2.
If the two strings are equal, wcscmp
returns 0
.
A consequence of the ordering used by wcscmp
is that if ws1
is an initial substring of ws2, then ws1 is considered to be
“less than” ws2.
wcscmp
does not take sorting conventions of the language the
strings are written in into account. To get that one has to use
wcscoll
.
int
strcasecmp (const char *s1, const char *s2)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is like strcmp
, except that differences in case are
ignored, and its arguments must be multibyte strings.
How uppercase and lowercase characters are related is
determined by the currently selected locale. In the standard "C"
locale the characters Ä and ä do not match but in a locale which
regards these characters as parts of the alphabet they do match.
strcasecmp
is derived from BSD.
int
wcscasecmp (const wchar_t *ws1, const wchar_t *ws2)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is like wcscmp
, except that differences in case are
ignored. How uppercase and lowercase characters are related is
determined by the currently selected locale. In the standard "C"
locale the characters Ä and ä do not match but in a locale which
regards these characters as parts of the alphabet they do match.
wcscasecmp
is a GNU extension.
int
strncmp (const char *s1, const char *s2, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is the similar to strcmp
, except that no more than
size bytes are compared. In other words, if the two
strings are the same in their first size bytes, the
return value is zero.
int
wcsncmp (const wchar_t *ws1, const wchar_t *ws2, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is similar to wcscmp
, except that no more than
size wide characters are compared. In other words, if the two
strings are the same in their first size wide characters, the
return value is zero.
int
strncasecmp (const char *s1, const char *s2, size_t n)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is like strncmp
, except that differences in case
are ignored, and the compared parts of the arguments should consist of
valid multibyte characters.
Like strcasecmp
, it is locale dependent how
uppercase and lowercase characters are related.
strncasecmp
is a GNU extension.
int
wcsncasecmp (const wchar_t *ws1, const wchar_t *s2, size_t n)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function is like wcsncmp
, except that differences in case
are ignored. Like wcscasecmp
, it is locale dependent how
uppercase and lowercase characters are related.
wcsncasecmp
is a GNU extension.
Here are some examples showing the use of strcmp
and
strncmp
(equivalent examples can be constructed for the wide
character functions). These examples assume the use of the ASCII
character set. (If some other character set—say, EBCDIC—is used
instead, then the glyphs are associated with different numeric codes,
and the return values and ordering may differ.)
strcmp ("hello", "hello") ⇒ 0 /* These two strings are the same. */ strcmp ("hello", "Hello") ⇒ 32 /* Comparisons are case-sensitive. */ strcmp ("hello", "world") ⇒ -15 /* The byte'h'
comes before'w'
. */ strcmp ("hello", "hello, world") ⇒ -44 /* Comparing a null byte against a comma. */ strncmp ("hello", "hello, world", 5) ⇒ 0 /* The initial 5 bytes are the same. */ strncmp ("hello, world", "hello, stupid world!!!", 5) ⇒ 0 /* The initial 5 bytes are the same. */
int
strverscmp (const char *s1, const char *s2)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The strverscmp
function compares the string s1 against
s2, considering them as holding indices/version numbers. The
return value follows the same conventions as found in the
strcmp
function. In fact, if s1 and s2 contain no
digits, strverscmp
behaves like strcmp
(in the sense that the sign of the result is the same).
The comparison algorithm which the strverscmp
function implements
differs slightly from other version-comparison algorithms. The
implementation is based on a finite-state machine, whose behavior is
approximated below.
isdigit
function and are
thus subject to the current locale.
The treatment of leading zeros and the tie-breaking extension characters (which in effect propagate across non-digit/digit sequence boundaries) differs from other version-comparison algorithms.
strverscmp ("no digit", "no digit") ⇒ 0 /* same behavior as strcmp. */ strverscmp ("item#99", "item#100") ⇒ <0 /* same prefix, but 99 < 100. */ strverscmp ("alpha1", "alpha001") ⇒ >0 /* different number of leading zeros (0 and 2). */ strverscmp ("part1_f012", "part1_f01") ⇒ >0 /* lexicographical comparison with leading zeros. */ strverscmp ("foo.009", "foo.0") ⇒ <0 /* different number of leading zeros (2 and 1). */
strverscmp
is a GNU extension.
int
bcmp (const void *a1, const void *a2, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This is an obsolete alias for memcmp
, derived from BSD.
In some locales, the conventions for lexicographic ordering differ from the strict numeric ordering of character codes. For example, in Spanish most glyphs with diacritical marks such as accents are not considered distinct letters for the purposes of collation. On the other hand, in Czech the two-character sequence ‘ch’ is treated as a single letter that is collated between ‘h’ and ‘i’.
You can use the functions strcoll
and strxfrm
(declared in
the headers file string.h) and wcscoll
and wcsxfrm
(declared in the headers file wchar) to compare strings using a
collation ordering appropriate for the current locale. The locale used
by these functions in particular can be specified by setting the locale
for the LC_COLLATE
category; see Locales and Internationalization.
In the standard C locale, the collation sequence for strcoll
is
the same as that for strcmp
. Similarly, wcscoll
and
wcscmp
are the same in this situation.
Effectively, the way these functions work is by applying a mapping to transform the characters in a multibyte string to a byte sequence that represents the string’s position in the collating sequence of the current locale. Comparing two such byte sequences in a simple fashion is equivalent to comparing the strings with the locale’s collating sequence.
The functions strcoll
and wcscoll
perform this translation
implicitly, in order to do one comparison. By contrast, strxfrm
and wcsxfrm
perform the mapping explicitly. If you are making
multiple comparisons using the same string or set of strings, it is
likely to be more efficient to use strxfrm
or wcsxfrm
to
transform all the strings just once, and subsequently compare the
transformed strings with strcmp
or wcscmp
.
int
strcoll (const char *s1, const char *s2)
¶Preliminary: | MT-Safe locale | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
The strcoll
function is similar to strcmp
but uses the
collating sequence of the current locale for collation (the
LC_COLLATE
locale). The arguments are multibyte strings.
int
wcscoll (const wchar_t *ws1, const wchar_t *ws2)
¶Preliminary: | MT-Safe locale | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
The wcscoll
function is similar to wcscmp
but uses the
collating sequence of the current locale for collation (the
LC_COLLATE
locale).
Here is an example of sorting an array of strings, using strcoll
to compare them. The actual sort algorithm is not written here; it
comes from qsort
(see Array Sort Function). The job of the
code shown here is to say how to compare the strings while sorting them.
(Later on in this section, we will show a way to do this more
efficiently using strxfrm
.)
/* This is the comparison function used withqsort
. */ int compare_elements (const void *v1, const void *v2) { char * const *p1 = v1; char * const *p2 = v2; return strcoll (*p1, *p2); } /* This is the entry point—the function to sort strings using the locale’s collating sequence. */ void sort_strings (char **array, int nstrings) { /* Sorttemp_array
by comparing the strings. */ qsort (array, nstrings, sizeof (char *), compare_elements); }
size_t
strxfrm (char *restrict to, const char *restrict from, size_t size)
¶Preliminary: | MT-Safe locale | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
The function strxfrm
transforms the multibyte string
from using the
collation transformation determined by the locale currently selected for
collation, and stores the transformed string in the array to. Up
to size bytes (including a terminating null byte) are
stored.
The behavior is undefined if the strings to and from overlap; see Copying Strings and Arrays.
The return value is the length of the entire transformed string. This
value is not affected by the value of size, but if it is greater
or equal than size, it means that the transformed string did not
entirely fit in the array to. In this case, only as much of the
string as actually fits was stored. To get the whole transformed
string, call strxfrm
again with a bigger output array.
The transformed string may be longer than the original string, and it may also be shorter.
If size is zero, no bytes are stored in to. In this
case, strxfrm
simply returns the number of bytes that would
be the length of the transformed string. This is useful for determining
what size the allocated array should be. It does not matter what
to is if size is zero; to may even be a null pointer.
size_t
wcsxfrm (wchar_t *restrict wto, const wchar_t *wfrom, size_t size)
¶Preliminary: | MT-Safe locale | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
The function wcsxfrm
transforms wide string wfrom
using the collation transformation determined by the locale currently
selected for collation, and stores the transformed string in the array
wto. Up to size wide characters (including a terminating null
wide character) are stored.
The behavior is undefined if the strings wto and wfrom overlap; see Copying Strings and Arrays.
The return value is the length of the entire transformed wide
string. This value is not affected by the value of size, but if
it is greater or equal than size, it means that the transformed
wide string did not entirely fit in the array wto. In
this case, only as much of the wide string as actually fits
was stored. To get the whole transformed wide string, call
wcsxfrm
again with a bigger output array.
The transformed wide string may be longer than the original wide string, and it may also be shorter.
If size is zero, no wide characters are stored in to. In this
case, wcsxfrm
simply returns the number of wide characters that
would be the length of the transformed wide string. This is
useful for determining what size the allocated array should be (remember
to multiply with sizeof (wchar_t)
). It does not matter what
wto is if size is zero; wto may even be a null pointer.
Here is an example of how you can use strxfrm
when
you plan to do many comparisons. It does the same thing as the previous
example, but much faster, because it has to transform each string only
once, no matter how many times it is compared with other strings. Even
the time needed to allocate and free storage is much less than the time
we save, when there are many strings.
struct sorter { char *input; char *transformed; }; /* This is the comparison function used withqsort
to sort an array ofstruct sorter
. */ int compare_elements (const void *v1, const void *v2) { const struct sorter *p1 = v1; const struct sorter *p2 = v2; return strcmp (p1->transformed, p2->transformed); } /* This is the entry point—the function to sort strings using the locale’s collating sequence. */ void sort_strings_fast (char **array, int nstrings) { struct sorter temp_array[nstrings]; int i; /* Set uptemp_array
. Each element contains one input string and its transformed string. */ for (i = 0; i < nstrings; i++) { size_t length = strlen (array[i]) * 2; char *transformed; size_t transformed_length; temp_array[i].input = array[i]; /* First try a buffer perhaps big enough. */ transformed = (char *) xmalloc (length); /* Transformarray[i]
. */ transformed_length = strxfrm (transformed, array[i], length); /* If the buffer was not large enough, resize it and try again. */ if (transformed_length >= length) { /* Allocate the needed space. +1 for terminating'\0'
byte. */ transformed = xrealloc (transformed, transformed_length + 1); /* The return value is not interesting because we know how long the transformed string is. */ (void) strxfrm (transformed, array[i], transformed_length + 1); } temp_array[i].transformed = transformed; } /* Sorttemp_array
by comparing transformed strings. */ qsort (temp_array, nstrings, sizeof (struct sorter), compare_elements); /* Put the elements back in the permanent array in their sorted order. */ for (i = 0; i < nstrings; i++) array[i] = temp_array[i].input; /* Free the strings we allocated. */ for (i = 0; i < nstrings; i++) free (temp_array[i].transformed); }
The interesting part of this code for the wide character version would look like this:
void sort_strings_fast (wchar_t **array, int nstrings) { ... /* Transformarray[i]
. */ transformed_length = wcsxfrm (transformed, array[i], length); /* If the buffer was not large enough, resize it and try again. */ if (transformed_length >= length) { /* Allocate the needed space. +1 for terminatingL'\0'
wide character. */ transformed = xreallocarray (transformed, transformed_length + 1, sizeof *transformed); /* The return value is not interesting because we know how long the transformed string is. */ (void) wcsxfrm (transformed, array[i], transformed_length + 1); } ...
Note the additional multiplication with sizeof (wchar_t)
in the
realloc
call.
Compatibility Note: The string collation functions are a new feature of ISO C90. Older C dialects have no equivalent feature. The wide character versions were introduced in Amendment 1 to ISO C90.
This section describes library functions which perform various kinds of searching operations on strings and arrays. These functions are declared in the header file string.h.
void *
memchr (const void *block, int c, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function finds the first occurrence of the byte c (converted
to an unsigned char
) in the initial size bytes of the
object beginning at block. The return value is a pointer to the
located byte, or a null pointer if no match was found.
wchar_t *
wmemchr (const wchar_t *block, wchar_t wc, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function finds the first occurrence of the wide character wc in the initial size wide characters of the object beginning at block. The return value is a pointer to the located wide character, or a null pointer if no match was found.
void *
rawmemchr (const void *block, int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Often the memchr
function is used with the knowledge that the
byte c is available in the memory block specified by the
parameters. But this means that the size parameter is not really
needed and that the tests performed with it at runtime (to check whether
the end of the block is reached) are not needed.
The rawmemchr
function exists for just this situation which is
surprisingly frequent. The interface is similar to memchr
except
that the size parameter is missing. The function will look beyond
the end of the block pointed to by block in case the programmer
made an error in assuming that the byte c is present in the block.
In this case the result is unspecified. Otherwise the return value is a
pointer to the located byte.
When looking for the end of a string, use strchr
.
This function is a GNU extension.
void *
memrchr (const void *block, int c, size_t size)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The function memrchr
is like memchr
, except that it searches
backwards from the end of the block defined by block and size
(instead of forwards from the front).
This function is a GNU extension.
char *
strchr (const char *string, int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The strchr
function finds the first occurrence of the byte
c (converted to a char
) in the string
beginning at string. The return value is a pointer to the located
byte, or a null pointer if no match was found.
For example,
strchr ("hello, world", 'l') ⇒ "llo, world" strchr ("hello, world", '?') ⇒ NULL
The terminating null byte is considered to be part of the string, so you can use this function get a pointer to the end of a string by specifying zero as the value of the c argument.
When strchr
returns a null pointer, it does not let you know
the position of the terminating null byte it has found. If you
need that information, it is better (but less portable) to use
strchrnul
than to search for it a second time.
wchar_t *
wcschr (const wchar_t *wstring, wchar_t wc)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The wcschr
function finds the first occurrence of the wide
character wc in the wide string
beginning at wstring. The return value is a pointer to the
located wide character, or a null pointer if no match was found.
The terminating null wide character is considered to be part of the wide
string, so you can use this function get a pointer to the end
of a wide string by specifying a null wide character as the
value of the wc argument. It would be better (but less portable)
to use wcschrnul
in this case, though.
char *
strchrnul (const char *string, int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
strchrnul
is the same as strchr
except that if it does
not find the byte, it returns a pointer to string’s terminating
null byte rather than a null pointer.
This function is a GNU extension.
wchar_t *
wcschrnul (const wchar_t *wstring, wchar_t wc)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
wcschrnul
is the same as wcschr
except that if it does not
find the wide character, it returns a pointer to the wide string’s
terminating null wide character rather than a null pointer.
This function is a GNU extension.
One useful, but unusual, use of the strchr
function is when one wants to have a pointer pointing to the null byte
terminating a string. This is often written in this way:
s += strlen (s);
This is almost optimal but the addition operation duplicated a bit of
the work already done in the strlen
function. A better solution
is this:
s = strchr (s, '\0');
There is no restriction on the second parameter of strchr
so it
could very well also be zero. Those readers thinking very
hard about this might now point out that the strchr
function is
more expensive than the strlen
function since we have two abort
criteria. This is right. But in the GNU C Library the implementation of
strchr
is optimized in a special way so that strchr
actually is faster.
char *
strrchr (const char *string, int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The function strrchr
is like strchr
, except that it searches
backwards from the end of the string string (instead of forwards
from the front).
For example,
strrchr ("hello, world", 'l') ⇒ "ld"
wchar_t *
wcsrchr (const wchar_t *wstring, wchar_t wc)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The function wcsrchr
is like wcschr
, except that it searches
backwards from the end of the string wstring (instead of forwards
from the front).
char *
strstr (const char *haystack, const char *needle)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This is like strchr
, except that it searches haystack for a
substring needle rather than just a single byte. It
returns a pointer into the string haystack that is the first
byte of the substring, or a null pointer if no match was found. If
needle is an empty string, the function returns haystack.
For example,
strstr ("hello, world", "l") ⇒ "llo, world" strstr ("hello, world", "wo") ⇒ "world"
wchar_t *
wcsstr (const wchar_t *haystack, const wchar_t *needle)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This is like wcschr
, except that it searches haystack for a
substring needle rather than just a single wide character. It
returns a pointer into the string haystack that is the first wide
character of the substring, or a null pointer if no match was found. If
needle is an empty string, the function returns haystack.
wchar_t *
wcswcs (const wchar_t *haystack, const wchar_t *needle)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
wcswcs
is a deprecated alias for wcsstr
. This is the
name originally used in the X/Open Portability Guide before the
Amendment 1 to ISO C90 was published.
char *
strcasestr (const char *haystack, const char *needle)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This is like strstr
, except that it ignores case in searching for
the substring. Like strcasecmp
, it is locale dependent how
uppercase and lowercase characters are related, and arguments are
multibyte strings.
For example,
strcasestr ("hello, world", "L") ⇒ "llo, world" strcasestr ("hello, World", "wo") ⇒ "World"
void *
memmem (const void *haystack, size_t haystack-len,
const void *needle, size_t needle-len)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This is like strstr
, but needle and haystack are byte
arrays rather than strings. needle-len is the
length of needle and haystack-len is the length of
haystack.
This function is a GNU extension.
size_t
strspn (const char *string, const char *skipset)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The strspn
(“string span”) function returns the length of the
initial substring of string that consists entirely of bytes that
are members of the set specified by the string skipset. The order
of the bytes in skipset is not important.
For example,
strspn ("hello, world", "abcdefghijklmnopqrstuvwxyz") ⇒ 5
In a multibyte string, characters consisting of more than one byte are not treated as single entities. Each byte is treated separately. The function is not locale-dependent.
size_t
wcsspn (const wchar_t *wstring, const wchar_t *skipset)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The wcsspn
(“wide character string span”) function returns the
length of the initial substring of wstring that consists entirely
of wide characters that are members of the set specified by the string
skipset. The order of the wide characters in skipset is not
important.
size_t
strcspn (const char *string, const char *stopset)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The strcspn
(“string complement span”) function returns the length
of the initial substring of string that consists entirely of bytes
that are not members of the set specified by the string stopset.
(In other words, it returns the offset of the first byte in string
that is a member of the set stopset.)
For example,
strcspn ("hello, world", " \t\n,.;!?") ⇒ 5
In a multibyte string, characters consisting of more than one byte are not treated as a single entities. Each byte is treated separately. The function is not locale-dependent.
size_t
wcscspn (const wchar_t *wstring, const wchar_t *stopset)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The wcscspn
(“wide character string complement span”) function
returns the length of the initial substring of wstring that
consists entirely of wide characters that are not members of the
set specified by the string stopset. (In other words, it returns
the offset of the first wide character in string that is a member of
the set stopset.)
char *
strpbrk (const char *string, const char *stopset)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The strpbrk
(“string pointer break”) function is related to
strcspn
, except that it returns a pointer to the first byte
in string that is a member of the set stopset instead of the
length of the initial substring. It returns a null pointer if no such
byte from stopset is found.
For example,
strpbrk ("hello, world", " \t\n,.;!?") ⇒ ", world"
In a multibyte string, characters consisting of more than one byte are not treated as single entities. Each byte is treated separately. The function is not locale-dependent.
wchar_t *
wcspbrk (const wchar_t *wstring, const wchar_t *stopset)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The wcspbrk
(“wide character string pointer break”) function is
related to wcscspn
, except that it returns a pointer to the first
wide character in wstring that is a member of the set
stopset instead of the length of the initial substring. It
returns a null pointer if no such wide character from stopset is found.
char *
index (const char *string, int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
index
is another name for strchr
; they are exactly the same.
New code should always use strchr
since this name is defined in
ISO C while index
is a BSD invention which never was available
on System V derived systems.
char *
rindex (const char *string, int c)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
rindex
is another name for strrchr
; they are exactly the same.
New code should always use strrchr
since this name is defined in
ISO C while rindex
is a BSD invention which never was available
on System V derived systems.
It’s fairly common for programs to have a need to do some simple kinds
of lexical analysis and parsing, such as splitting a command string up
into tokens. You can do this with the strtok
function, declared
in the header file string.h.
char *
strtok (char *restrict newstring, const char *restrict delimiters)
¶Preliminary: | MT-Unsafe race:strtok | AS-Unsafe | AC-Safe | See POSIX Safety Concepts.
A string can be split into tokens by making a series of calls to the
function strtok
.
The string to be split up is passed as the newstring argument on
the first call only. The strtok
function uses this to set up
some internal state information. Subsequent calls to get additional
tokens from the same string are indicated by passing a null pointer as
the newstring argument. Calling strtok
with another
non-null newstring argument reinitializes the state information.
It is guaranteed that no other library function ever calls strtok
behind your back (which would mess up this internal state information).
The delimiters argument is a string that specifies a set of delimiters that may surround the token being extracted. All the initial bytes that are members of this set are discarded. The first byte that is not a member of this set of delimiters marks the beginning of the next token. The end of the token is found by looking for the next byte that is a member of the delimiter set. This byte in the original string newstring is overwritten by a null byte, and the pointer to the beginning of the token in newstring is returned.
On the next call to strtok
, the searching begins at the next
byte beyond the one that marked the end of the previous token.
Note that the set of delimiters delimiters do not have to be the
same on every call in a series of calls to strtok
.
If the end of the string newstring is reached, or if the remainder of
string consists only of delimiter bytes, strtok
returns
a null pointer.
In a multibyte string, characters consisting of more than one byte are not treated as single entities. Each byte is treated separately. The function is not locale-dependent.
wchar_t *
wcstok (wchar_t *newstring, const wchar_t *delimiters, wchar_t **save_ptr)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
A string can be split into tokens by making a series of calls to the
function wcstok
.
The string to be split up is passed as the newstring argument on
the first call only. The wcstok
function uses this to set up
some internal state information. Subsequent calls to get additional
tokens from the same wide string are indicated by passing a
null pointer as the newstring argument, which causes the pointer
previously stored in save_ptr to be used instead.
The delimiters argument is a wide string that specifies a set of delimiters that may surround the token being extracted. All the initial wide characters that are members of this set are discarded. The first wide character that is not a member of this set of delimiters marks the beginning of the next token. The end of the token is found by looking for the next wide character that is a member of the delimiter set. This wide character in the original wide string newstring is overwritten by a null wide character, the pointer past the overwritten wide character is saved in save_ptr, and the pointer to the beginning of the token in newstring is returned.
On the next call to wcstok
, the searching begins at the next
wide character beyond the one that marked the end of the previous token.
Note that the set of delimiters delimiters do not have to be the
same on every call in a series of calls to wcstok
.
If the end of the wide string newstring is reached, or
if the remainder of string consists only of delimiter wide characters,
wcstok
returns a null pointer.
Warning: Since strtok
and wcstok
alter the string
they is parsing, you should always copy the string to a temporary buffer
before parsing it with strtok
/wcstok
(see Copying Strings and Arrays). If you allow strtok
or wcstok
to modify
a string that came from another part of your program, you are asking for
trouble; that string might be used for other purposes after
strtok
or wcstok
has modified it, and it would not have
the expected value.
The string that you are operating on might even be a constant. Then
when strtok
or wcstok
tries to modify it, your program
will get a fatal signal for writing in read-only memory. See Program Error Signals. Even if the operation of strtok
or wcstok
would not require a modification of the string (e.g., if there is
exactly one token) the string can (and in the GNU C Library case will) be
modified.
This is a special case of a general principle: if a part of a program does not have as its purpose the modification of a certain data structure, then it is error-prone to modify the data structure temporarily.
The function strtok
is not reentrant, whereas wcstok
is.
See Signal Handling and Nonreentrant Functions, for a discussion of where and why reentrancy is
important.
Here is a simple example showing the use of strtok
.
#include <string.h> #include <stddef.h> ... const char string[] = "words separated by spaces -- and, punctuation!"; const char delimiters[] = " .,;:!-"; char *token, *cp; ... cp = strdupa (string); /* Make writable copy. */ token = strtok (cp, delimiters); /* token => "words" */ token = strtok (NULL, delimiters); /* token => "separated" */ token = strtok (NULL, delimiters); /* token => "by" */ token = strtok (NULL, delimiters); /* token => "spaces" */ token = strtok (NULL, delimiters); /* token => "and" */ token = strtok (NULL, delimiters); /* token => "punctuation" */ token = strtok (NULL, delimiters); /* token => NULL */
The GNU C Library contains two more functions for tokenizing a string which overcome the limitation of non-reentrancy. They are not available available for wide strings.
char *
strtok_r (char *newstring, const char *delimiters, char **save_ptr)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Just like strtok
, this function splits the string into several
tokens which can be accessed by successive calls to strtok_r
.
The difference is that, as in wcstok
, the information about the
next token is stored in the space pointed to by the third argument,
save_ptr, which is a pointer to a string pointer. Calling
strtok_r
with a null pointer for newstring and leaving
save_ptr between the calls unchanged does the job without
hindering reentrancy.
This function is defined in POSIX.1 and can be found on many systems which support multi-threading.
char *
strsep (char **string_ptr, const char *delimiter)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This function has a similar functionality as strtok_r
with the
newstring argument replaced by the save_ptr argument. The
initialization of the moving pointer has to be done by the user.
Successive calls to strsep
move the pointer along the tokens
separated by delimiter, returning the address of the next token
and updating string_ptr to point to the beginning of the next
token.
One difference between strsep
and strtok_r
is that if the
input string contains more than one byte from delimiter in a
row strsep
returns an empty string for each pair of bytes
from delimiter. This means that a program normally should test
for strsep
returning an empty string before processing it.
This function was introduced in 4.3BSD and therefore is widely available.
Here is how the above example looks like when strsep
is used.
#include <string.h> #include <stddef.h> ... const char string[] = "words separated by spaces -- and, punctuation!"; const char delimiters[] = " .,;:!-"; char *running; char *token; ... running = strdupa (string); token = strsep (&running, delimiters); /* token => "words" */ token = strsep (&running, delimiters); /* token => "separated" */ token = strsep (&running, delimiters); /* token => "by" */ token = strsep (&running, delimiters); /* token => "spaces" */ token = strsep (&running, delimiters); /* token => "" */ token = strsep (&running, delimiters); /* token => "" */ token = strsep (&running, delimiters); /* token => "" */ token = strsep (&running, delimiters); /* token => "and" */ token = strsep (&running, delimiters); /* token => "" */ token = strsep (&running, delimiters); /* token => "punctuation" */ token = strsep (&running, delimiters); /* token => "" */ token = strsep (&running, delimiters); /* token => NULL */
char *
basename (const char *filename)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The GNU version of the basename
function returns the last
component of the path in filename. This function is the preferred
usage, since it does not modify the argument, filename, and
respects trailing slashes. The prototype for basename
can be
found in string.h. Note, this function is overridden by the XPG
version, if libgen.h is included.
Example of using GNU basename
:
#include <string.h> int main (int argc, char *argv[]) { char *prog = basename (argv[0]); if (argc < 2) { fprintf (stderr, "Usage %s <arg>\n", prog); exit (1); } ... }
Portability Note: This function may produce different results on different systems.
char *
basename (char *path)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
This is the standard XPG defined basename
. It is similar in
spirit to the GNU version, but may modify the path by removing
trailing ’/’ bytes. If the path is made up entirely of ’/’
bytes, then "/" will be returned. Also, if path is
NULL
or an empty string, then "." is returned. The prototype for
the XPG version can be found in libgen.h.
Example of using XPG basename
:
#include <libgen.h> int main (int argc, char *argv[]) { char *prog; char *path = strdupa (argv[0]); prog = basename (path); if (argc < 2) { fprintf (stderr, "Usage %s <arg>\n", prog); exit (1); } ... }
char *
dirname (char *path)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The dirname
function is the compliment to the XPG version of
basename
. It returns the parent directory of the file specified
by path. If path is NULL
, an empty string, or
contains no ’/’ bytes, then "." is returned. The prototype for this
function can be found in libgen.h.
Sensitive data, such as cryptographic keys, should be erased from memory after use, to reduce the risk that a bug will expose it to the outside world. However, compiler optimizations may determine that an erasure operation is “unnecessary,” and remove it from the generated code, because no correct program could access the variable or heap object containing the sensitive data after it’s deallocated. Since erasure is a precaution against bugs, this optimization is inappropriate.
The function explicit_bzero
erases a block of memory, and
guarantees that the compiler will not remove the erasure as
“unnecessary.”
#include <string.h> extern void encrypt (const char *key, const char *in, char *out, size_t n); extern void genkey (const char *phrase, char *key); void encrypt_with_phrase (const char *phrase, const char *in, char *out, size_t n) { char key[16]; genkey (phrase, key); encrypt (key, in, out, n); explicit_bzero (key, 16); }
In this example, if memset
, bzero
, or a hand-written
loop had been used, the compiler might remove them as “unnecessary.”
Warning: explicit_bzero
does not guarantee that
sensitive data is completely erased from the computer’s memory.
There may be copies in temporary storage areas, such as registers and
“scratch” stack space; since these are invisible to the source code,
a library function cannot erase them.
Also, explicit_bzero
only operates on RAM. If a sensitive data
object never needs to have its address taken other than to call
explicit_bzero
, it might be stored entirely in CPU registers
until the call to explicit_bzero
. Then it will be
copied into RAM, the copy will be erased, and the original will remain
intact. Data in RAM is more likely to be exposed by a bug than data
in registers, so this creates a brief window where the data is at
greater risk of exposure than it would have been if the program didn’t
try to erase it at all.
Declaring sensitive variables as volatile
will make both the
above problems worse; a volatile
variable will be stored
in memory for its entire lifetime, and the compiler will make
more copies of it than it would otherwise have. Attempting to
erase a normal variable “by hand” through a
volatile
-qualified pointer doesn’t work at all—because the
variable itself is not volatile
, some compilers will ignore the
qualification on the pointer and remove the erasure anyway.
Having said all that, in most situations, using explicit_bzero
is better than not using it. At present, the only way to do a more
thorough job is to write the entire sensitive operation in assembly
language. We anticipate that future compilers will recognize calls to
explicit_bzero
and take appropriate steps to erase all the
copies of the affected data, wherever they may be.
void
explicit_bzero (void *block, size_t len)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
explicit_bzero
writes zero into len bytes of memory
beginning at block, just as bzero
would. The zeroes are
always written, even if the compiler could determine that this is
“unnecessary” because no correct program could read them back.
Note: The only optimization that explicit_bzero
disables is removal of “unnecessary” writes to memory. The compiler
can perform all the other optimizations that it could for a call to
memset
. For instance, it may replace the function call with
inline memory writes, and it may assume that block cannot be a
null pointer.
Portability Note: This function first appeared in OpenBSD 5.5
and has not been standardized. Other systems may provide the same
functionality under a different name, such as explicit_memset
,
memset_s
, or SecureZeroMemory
.
The GNU C Library declares this function in string.h, but on other systems it may be in strings.h instead.
The function below addresses the perennial programming quandary: “How do I take good data in string form and painlessly turn it into garbage?” This is not a difficult thing to code for oneself, but the authors of the GNU C Library wish to make it as convenient as possible.
To erase data, use explicit_bzero
(see Erasing Sensitive Data); to obfuscate it reversibly, use memfrob
(see Obfuscating Data).
char *
strfry (char *string)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
strfry
performs an in-place shuffle on string. Each
character is swapped to a position selected at random, within the
portion of the string starting with the character’s original position.
(This is the Fisher-Yates algorithm for unbiased shuffling.)
Calling strfry
will not disturb any of the random number
generators that have global state (see Pseudo-Random Numbers).
The return value of strfry
is always string.
Portability Note: This function is unique to the GNU C Library. It is declared in string.h.
The memfrob
function reversibly obfuscates an array of binary
data. This is not true encryption; the obfuscated data still bears a
clear relationship to the original, and no secret key is required to
undo the obfuscation. It is analogous to the “Rot13” cipher used on
Usenet for obscuring offensive jokes, spoilers for works of fiction,
and so on, but it can be applied to arbitrary binary data.
Programs that need true encryption—a transformation that completely obscures the original and cannot be reversed without knowledge of a secret key—should use a dedicated cryptography library, such as libgcrypt.
Programs that need to destroy data should use
explicit_bzero
(see Erasing Sensitive Data), or possibly
strfry
(see Shuffling Bytes).
void *
memfrob (void *mem, size_t length)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The function memfrob
obfuscates length bytes of data
beginning at mem, in place. Each byte is bitwise xor-ed with
the binary pattern 00101010 (hexadecimal 0x2A). The return value is
always mem.
memfrob
a second time on the same data returns it to
its original state.
Portability Note: This function is unique to the GNU C Library. It is declared in string.h.
To store or transfer binary data in environments which only support text one has to encode the binary data by mapping the input bytes to bytes in the range allowed for storing or transferring. SVID systems (and nowadays XPG compliant systems) provide minimal support for this task.
char *
l64a (long int n)
¶Preliminary: | MT-Unsafe race:l64a | AS-Unsafe | AC-Safe | See POSIX Safety Concepts.
This function encodes a 32-bit input value using bytes from the
basic character set. It returns a pointer to a 7 byte buffer which
contains an encoded version of n. To encode a series of bytes the
user must copy the returned string to a destination buffer. It returns
the empty string if n is zero, which is somewhat bizarre but
mandated by the standard.
Warning: Since a static buffer is used this function should not
be used in multi-threaded programs. There is no thread-safe alternative
to this function in the C library.
Compatibility Note: The XPG standard states that the return
value of l64a
is undefined if n is negative. In the GNU
implementation, l64a
treats its argument as unsigned, so it will
return a sensible encoding for any nonzero n; however, portable
programs should not rely on this.
To encode a large buffer l64a
must be called in a loop, once for
each 32-bit word of the buffer. For example, one could do something
like this:
char * encode (const void *buf, size_t len) { /* We know in advance how long the buffer has to be. */ unsigned char *in = (unsigned char *) buf; char *out = malloc (6 + ((len + 3) / 4) * 6 + 1); char *cp = out, *p; /* Encode the length. */ /* Using ‘htonl’ is necessary so that the data can be decoded even on machines with different byte order. ‘l64a’ can return a string shorter than 6 bytes, so we pad it with encoding of 0 ('.') at the end by hand. */ p = stpcpy (cp, l64a (htonl (len))); cp = mempcpy (p, "......", 6 - (p - cp)); while (len > 3) { unsigned long int n = *in++; n = (n << 8) | *in++; n = (n << 8) | *in++; n = (n << 8) | *in++; len -= 4; p = stpcpy (cp, l64a (htonl (n))); cp = mempcpy (p, "......", 6 - (p - cp)); } if (len > 0) { unsigned long int n = *in++; if (--len > 0) { n = (n << 8) | *in++; if (--len > 0) n = (n << 8) | *in; } cp = stpcpy (cp, l64a (htonl (n))); } *cp = '\0'; return out; }
It is strange that the library does not provide the complete functionality needed but so be it.
To decode data produced with l64a
the following function should be
used.
long int
a64l (const char *string)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The parameter string should contain a string which was produced by
a call to l64a
. The function processes at least 6 bytes of
this string, and decodes the bytes it finds according to the table
below. It stops decoding when it finds a byte not in the table,
rather like atoi
; if you have a buffer which has been broken into
lines, you must be careful to skip over the end-of-line bytes.
The decoded number is returned as a long int
value.
The l64a
and a64l
functions use a base 64 encoding, in
which each byte of an encoded string represents six bits of an
input word. These symbols are used for the base 64 digits:
0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | |
0 | . | / | 0 | 1 | 2 | 3 | 4 | 5 |
8 | 6 | 7 | 8 | 9 | A | B | C | D |
16 | E | F | G | H | I | J | K | L |
24 | M | N | O | P | Q | R | S | T |
32 | U | V | W | X | Y | Z | a | b |
40 | c | d | e | f | g | h | i | j |
48 | k | l | m | n | o | p | q | r |
56 | s | t | u | v | w | x | y | z |
This encoding scheme is not standard. There are some other encoding methods which are much more widely used (UU encoding, MIME encoding). Generally, it is better to use one of these encodings.
argz vectors are vectors of strings in a contiguous block of
memory, each element separated from its neighbors by null bytes
('\0'
).
Envz vectors are an extension of argz vectors where each element is a
name-value pair, separated by a '='
byte (as in a Unix
environment).
Each argz vector is represented by a pointer to the first element, of
type char *
, and a size, of type size_t
, both of which can
be initialized to 0
to represent an empty argz vector. All argz
functions accept either a pointer and a size argument, or pointers to
them, if they will be modified.
The argz functions use malloc
/realloc
to allocate/grow
argz vectors, and so any argz vector created using these functions may
be freed by using free
; conversely, any argz function that may
grow a string expects that string to have been allocated using
malloc
(those argz functions that only examine their arguments or
modify them in place will work on any sort of memory).
See Unconstrained Allocation.
All argz functions that do memory allocation have a return type of
error_t
, and return 0
for success, and ENOMEM
if an
allocation error occurs.
These functions are declared in the standard include file argz.h.
error_t
argz_create (char *const argv[], char **argz, size_t *argz_len)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
The argz_create
function converts the Unix-style argument vector
argv (a vector of pointers to normal C strings, terminated by
(char *)0
; see Program Arguments) into an argz vector with
the same elements, which is returned in argz and argz_len.
error_t
argz_create_sep (const char *string, int sep, char **argz, size_t *argz_len)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
The argz_create_sep
function converts the string
string into an argz vector (returned in argz and
argz_len) by splitting it into elements at every occurrence of the
byte sep.
size_t
argz_count (const char *argz, size_t argz_len)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
Returns the number of elements in the argz vector argz and argz_len.
void
argz_extract (const char *argz, size_t argz_len, char **argv)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The argz_extract
function converts the argz vector argz and
argz_len into a Unix-style argument vector stored in argv,
by putting pointers to every element in argz into successive
positions in argv, followed by a terminator of 0
.
Argv must be pre-allocated with enough space to hold all the
elements in argz plus the terminating (char *)0
((argz_count (argz, argz_len) + 1) * sizeof (char *)
bytes should be enough). Note that the string pointers stored into
argv point into argz—they are not copies—and so
argz must be copied if it will be changed while argv is
still active. This function is useful for passing the elements in
argz to an exec function (see Executing a File).
void
argz_stringify (char *argz, size_t len, int sep)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The argz_stringify
converts argz into a normal string with
the elements separated by the byte sep, by replacing each
'\0'
inside argz (except the last one, which terminates the
string) with sep. This is handy for printing argz in a
readable manner.
error_t
argz_add (char **argz, size_t *argz_len, const char *str)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
The argz_add
function adds the string str to the end of the
argz vector *argz
, and updates *argz
and
*argz_len
accordingly.
error_t
argz_add_sep (char **argz, size_t *argz_len, const char *str, int delim)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
The argz_add_sep
function is similar to argz_add
, but
str is split into separate elements in the result at occurrences of
the byte delim. This is useful, for instance, for
adding the components of a Unix search path to an argz vector, by using
a value of ':'
for delim.
error_t
argz_append (char **argz, size_t *argz_len, const char *buf, size_t buf_len)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
The argz_append
function appends buf_len bytes starting at
buf to the argz vector *argz
, reallocating
*argz
to accommodate it, and adding buf_len to
*argz_len
.
void
argz_delete (char **argz, size_t *argz_len, char *entry)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
If entry points to the beginning of one of the elements in the
argz vector *argz
, the argz_delete
function will
remove this entry and reallocate *argz
, modifying
*argz
and *argz_len
accordingly. Note that as
destructive argz functions usually reallocate their argz argument,
pointers into argz vectors such as entry will then become invalid.
error_t
argz_insert (char **argz, size_t *argz_len, char *before, const char *entry)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
The argz_insert
function inserts the string entry into the
argz vector *argz
at a point just before the existing
element pointed to by before, reallocating *argz
and
updating *argz
and *argz_len
. If before
is 0
, entry is added to the end instead (as if by
argz_add
). Since the first element is in fact the same as
*argz
, passing in *argz
as the value of
before will result in entry being inserted at the beginning.
char *
argz_next (const char *argz, size_t argz_len, const char *entry)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The argz_next
function provides a convenient way of iterating
over the elements in the argz vector argz. It returns a pointer
to the next element in argz after the element entry, or
0
if there are no elements following entry. If entry
is 0
, the first element of argz is returned.
This behavior suggests two styles of iteration:
char *entry = 0; while ((entry = argz_next (argz, argz_len, entry))) action;
(the double parentheses are necessary to make some C compilers shut up
about what they consider a questionable while
-test) and:
char *entry; for (entry = argz; entry; entry = argz_next (argz, argz_len, entry)) action;
Note that the latter depends on argz having a value of 0
if
it is empty (rather than a pointer to an empty block of memory); this
invariant is maintained for argz vectors created by the functions here.
error_t
argz_replace (char **argz, size_t *argz_len, const char *str, const char *with, unsigned *replace_count)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
Replace any occurrences of the string str in argz with
with, reallocating argz as necessary. If
replace_count is non-zero, *replace_count
will be
incremented by the number of replacements performed.
Envz vectors are just argz vectors with additional constraints on the form of each element; as such, argz functions can also be used on them, where it makes sense.
Each element in an envz vector is a name-value pair, separated by a '='
byte; if multiple '='
bytes are present in an element, those
after the first are considered part of the value, and treated like all other
non-'\0'
bytes.
If no '='
bytes are present in an element, that element is
considered the name of a “null” entry, as distinct from an entry with an
empty value: envz_get
will return 0
if given the name of null
entry, whereas an entry with an empty value would result in a value of
""
; envz_entry
will still find such entries, however. Null
entries can be removed with the envz_strip
function.
As with argz functions, envz functions that may allocate memory (and thus
fail) have a return type of error_t
, and return either 0
or
ENOMEM
.
These functions are declared in the standard include file envz.h.
char *
envz_entry (const char *envz, size_t envz_len, const char *name)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The envz_entry
function finds the entry in envz with the name
name, and returns a pointer to the whole entry—that is, the argz
element which begins with name followed by a '='
byte. If
there is no entry with that name, 0
is returned.
char *
envz_get (const char *envz, size_t envz_len, const char *name)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The envz_get
function finds the entry in envz with the name
name (like envz_entry
), and returns a pointer to the value
portion of that entry (following the '='
). If there is no entry with
that name (or only a null entry), 0
is returned.
error_t
envz_add (char **envz, size_t *envz_len, const char *name, const char *value)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
The envz_add
function adds an entry to *envz
(updating *envz
and *envz_len
) with the name
name, and value value. If an entry with the same name
already exists in envz, it is removed first. If value is
0
, then the new entry will be the special null type of entry
(mentioned above).
error_t
envz_merge (char **envz, size_t *envz_len, const char *envz2, size_t envz2_len, int override)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
The envz_merge
function adds each entry in envz2 to envz,
as if with envz_add
, updating *envz
and
*envz_len
. If override is true, then values in envz2
will supersede those with the same name in envz, otherwise not.
Null entries are treated just like other entries in this respect, so a null entry in envz can prevent an entry of the same name in envz2 from being added to envz, if override is false.
void
envz_strip (char **envz, size_t *envz_len)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The envz_strip
function removes any null entries from envz,
updating *envz
and *envz_len
.
void
envz_remove (char **envz, size_t *envz_len, const char *name)
¶Preliminary: | MT-Safe | AS-Unsafe heap | AC-Unsafe mem | See POSIX Safety Concepts.
The envz_remove
function removes an entry named name from
envz, updating *envz
and *envz_len
.
Character sets used in the early days of computing had only six, seven, or eight bits for each character: there was never a case where more than eight bits (one byte) were used to represent a single character. The limitations of this approach became more apparent as more people grappled with non-Roman character sets, where not all the characters that make up a language’s character set can be represented by 2^8 choices. This chapter shows the functionality that was added to the C library to support multiple character sets.
A variety of solutions are available to overcome the differences between character sets with a 1:1 relation between bytes and characters and character sets with ratios of 2:1 or 4:1. The remainder of this section gives a few examples to help understand the design decisions made while developing the functionality of the C library.
A distinction we have to make right away is between internal and external representation. Internal representation means the representation used by a program while keeping the text in memory. External representations are used when text is stored or transmitted through some communication channel. Examples of external representations include files waiting in a directory to be read and parsed.
Traditionally there has been no difference between the two representations. It was equally comfortable and useful to use the same single-byte representation internally and externally. This comfort level decreases with more and larger character sets.
One of the problems to overcome with the internal representation is handling text that is externally encoded using different character sets. Assume a program that reads two texts and compares them using some metric. The comparison can be usefully done only if the texts are internally kept in a common format.
For such a common format (= character set) eight bits are certainly no longer enough. So the smallest entity will have to grow: wide characters will now be used. Instead of one byte per character, two or four will be used instead. (Three are not good to address in memory and more than four bytes seem not to be necessary).
As shown in some other part of this manual,
a completely new family has been created of functions that can handle wide
character texts in memory. The most commonly used character sets for such
internal wide character representations are Unicode and ISO 10646
(also known as UCS for Universal Character Set). Unicode was originally
planned as a 16-bit character set; whereas, ISO 10646 was designed to
be a 31-bit large code space. The two standards are practically identical.
They have the same character repertoire and code table, but Unicode specifies
added semantics. At the moment, only characters in the first 0x10000
code positions (the so-called Basic Multilingual Plane, BMP) have been
assigned, but the assignment of more specialized characters outside this
16-bit space is already in progress. A number of encodings have been
defined for Unicode and ISO 10646 characters:
UCS-2 is a 16-bit word that can only represent characters
from the BMP, UCS-4 is a 32-bit word than can represent any Unicode
and ISO 10646 character, UTF-8 is an ASCII compatible encoding where
ASCII characters are represented by ASCII bytes and non-ASCII characters
by sequences of 2-6 non-ASCII bytes, and finally UTF-16 is an extension
of UCS-2 in which pairs of certain UCS-2 words can be used to encode
non-BMP characters up to 0x10ffff
.
To represent wide characters the char
type is not suitable. For
this reason the ISO C standard introduces a new type that is
designed to keep one character of a wide character string. To maintain
the similarity there is also a type corresponding to int
for
those functions that take a single wide character.
This data type is used as the base type for wide character strings.
In other words, arrays of objects of this type are the equivalent of
char[]
for multibyte character strings. The type is defined in
stddef.h.
The ISO C90 standard, where wchar_t
was introduced, does not
say anything specific about the representation. It only requires that
this type is capable of storing all elements of the basic character set.
Therefore it would be legitimate to define wchar_t
as char
,
which might make sense for embedded systems.
But in the GNU C Library wchar_t
is always 32 bits wide and, therefore,
capable of representing all UCS-4 values and, therefore, covering all of
ISO 10646. Some Unix systems define wchar_t
as a 16-bit type
and thereby follow Unicode very strictly. This definition is perfectly
fine with the standard, but it also means that to represent all
characters from Unicode and ISO 10646 one has to use UTF-16 surrogate
characters, which is in fact a multi-wide-character encoding. But
resorting to multi-wide-character encoding contradicts the purpose of the
wchar_t
type.
wint_t
is a data type used for parameters and variables that
contain a single wide character. As the name suggests this type is the
equivalent of int
when using the normal char
strings. The
types wchar_t
and wint_t
often have the same
representation if their size is 32 bits wide but if wchar_t
is
defined as char
the type wint_t
must be defined as
int
due to the parameter promotion.
This type is defined in wchar.h and was introduced in Amendment 1 to ISO C90.
As there are for the char
data type macros are available for
specifying the minimum and maximum value representable in an object of
type wchar_t
.
wint_t
WCHAR_MIN ¶The macro WCHAR_MIN
evaluates to the minimum value representable
by an object of type wint_t
.
This macro was introduced in Amendment 1 to ISO C90.
wint_t
WCHAR_MAX ¶The macro WCHAR_MAX
evaluates to the maximum value representable
by an object of type wint_t
.
This macro was introduced in Amendment 1 to ISO C90.
Another special wide character value is the equivalent to EOF
.
wint_t
WEOF ¶The macro WEOF
evaluates to a constant expression of type
wint_t
whose value is different from any member of the extended
character set.
WEOF
need not be the same value as EOF
and unlike
EOF
it also need not be negative. In other words, sloppy
code like
{ int c; ... while ((c = getc (fp)) < 0) ... }
has to be rewritten to use WEOF
explicitly when wide characters
are used:
{ wint_t c; ... while ((c = getwc (fp)) != WEOF) ... }
This macro was introduced in Amendment 1 to ISO C90 and is defined in wchar.h.
These internal representations present problems when it comes to storage and transmittal. Because each single wide character consists of more than one byte, they are affected by byte-ordering. Thus, machines with different endianesses would see different values when accessing the same data. This byte ordering concern also applies for communication protocols that are all byte-based and therefore require that the sender has to decide about splitting the wide character in bytes. A last (but not least important) point is that wide characters often require more storage space than a customized byte-oriented character set.
For all the above reasons, an external encoding that is different from
the internal encoding is often used if the latter is UCS-2 or UCS-4.
The external encoding is byte-based and can be chosen appropriately for
the environment and for the texts to be handled. A variety of different
character sets can be used for this external encoding (information that
will not be exhaustively presented here–instead, a description of the
major groups will suffice). All of the ASCII-based character sets
fulfill one requirement: they are "filesystem safe." This means that
the character '/'
is used in the encoding only to
represent itself. Things are a bit different for character sets like
EBCDIC (Extended Binary Coded Decimal Interchange Code, a character set
family used by IBM), but if the operating system does not understand
EBCDIC directly the parameters-to-system calls have to be converted
first anyhow.
In most uses of ISO 2022 the defined character sets do not allow state changes that cover more than the next character. This has the big advantage that whenever one can identify the beginning of the byte sequence of a character one can interpret a text correctly. Examples of character sets using this policy are the various EUC character sets (used by Sun’s operating systems, EUC-JP, EUC-KR, EUC-TW, and EUC-CN) or Shift_JIS (SJIS, a Japanese encoding).
But there are also character sets using a state that is valid for more than one character and has to be changed by another byte sequence. Examples for this are ISO-2022-JP, ISO-2022-KR, and ISO-2022-CN.
0xc2 0x61
(non-spacing acute accent, followed by lower-case ‘a’) to get the “small
a with acute” character. To get the acute accent character on its own,
one has to write 0xc2 0x20
(the non-spacing acute followed by a
space).
Character sets like ISO 6937 are used in some embedded systems such as teletex.
There were a few other attempts to encode ISO 10646 such as UTF-7, but UTF-8 is today the only encoding that should be used. In fact, with any luck UTF-8 will soon be the only external encoding that has to be supported. It proves to be universally usable and its only disadvantage is that it favors Roman languages by making the byte string representation of other scripts (Cyrillic, Greek, Asian scripts) longer than necessary if using a specific character set for these scripts. Methods like the Unicode compression scheme can alleviate these problems.
The question remaining is: how to select the character set or encoding to use. The answer: you cannot decide about it yourself, it is decided by the developers of the system or the majority of the users. Since the goal is interoperability one has to use whatever the other people one works with use. If there are no constraints, the selection is based on the requirements the expected circle of users will have. In other words, if a project is expected to be used in only, say, Russia it is fine to use KOI8-R or a similar character set. But if at the same time people from, say, Greece are participating one should use a character set that allows all people to collaborate.
The most widely useful solution seems to be: go with the most general character set, namely ISO 10646. Use UTF-8 as the external encoding and problems about users not being able to use their own language adequately are a thing of the past.
One final comment about the choice of the wide character representation
is necessary at this point. We have said above that the natural choice
is using Unicode or ISO 10646. This is not required, but at least
encouraged, by the ISO C standard. The standard defines at least a
macro __STDC_ISO_10646__
that is only defined on systems where
the wchar_t
type encodes ISO 10646 characters. If this
symbol is not defined one should avoid making assumptions about the wide
character representation. If the programmer uses only the functions
provided by the C library to handle wide character strings there should
be no compatibility problems with other systems.
A Unix C library contains three different sets of functions in two families to handle character set conversion. One of the function families (the most commonly used) is specified in the ISO C90 standard and, therefore, is portable even beyond the Unix world. Unfortunately this family is the least useful one. These functions should be avoided whenever possible, especially when developing libraries (as opposed to applications).
The second family of functions got introduced in the early Unix standards (XPG2) and is still part of the latest and greatest Unix standard: Unix 98. It is also the most powerful and useful set of functions. But we will start with the functions defined in Amendment 1 to ISO C90.
The ISO C standard defines functions to convert strings from a multibyte representation to wide character strings. There are a number of peculiarities:
LC_CTYPE
category of the current locale is used; see
Locale Categories.
Despite these limitations the ISO C functions can be used in many
contexts. In graphical user interfaces, for instance, it is not
uncommon to have functions that require text to be displayed in a wide
character string if the text is not simple ASCII. The text itself might
come from a file with translations and the user should decide about the
current locale, which determines the translation and therefore also the
external encoding used. In such a situation (and many others) the
functions described here are perfect. If more freedom while performing
the conversion is necessary take a look at the iconv
functions
(see Generic Charset Conversion).
We already said above that the currently selected locale for the
LC_CTYPE
category decides the conversion that is performed
by the functions we are about to describe. Each locale uses its own
character set (given as an argument to localedef
) and this is the
one assumed as the external multibyte encoding. The wide character
set is always UCS-4 in the GNU C Library.
A characteristic of each multibyte character set is the maximum number of bytes that can be necessary to represent one character. This information is quite important when writing code that uses the conversion functions (as shown in the examples below). The ISO C standard defines two macros that provide this information.
int
MB_LEN_MAX ¶MB_LEN_MAX
specifies the maximum number of bytes in the multibyte
sequence for a single character in any of the supported locales. It is
a compile-time constant and is defined in limits.h.
int
MB_CUR_MAX ¶MB_CUR_MAX
expands into a positive integer expression that is the
maximum number of bytes in a multibyte character in the current locale.
The value is never greater than MB_LEN_MAX
. Unlike
MB_LEN_MAX
this macro need not be a compile-time constant, and in
the GNU C Library it is not.
MB_CUR_MAX
is defined in stdlib.h.
Two different macros are necessary since strictly ISO C90 compilers do not allow variable length array definitions, but still it is desirable to avoid dynamic allocation. This incomplete piece of code shows the problem:
{
char buf[MB_LEN_MAX];
ssize_t len = 0;
while (! feof (fp))
{
fread (&buf[len], 1, MB_CUR_MAX - len, fp);
/* ... process buf */
len -= used;
}
}
The code in the inner loop is expected to have always enough bytes in
the array buf to convert one multibyte character. The array
buf has to be sized statically since many compilers do not allow a
variable size. The fread
call makes sure that MB_CUR_MAX
bytes are always available in buf. Note that it isn’t
a problem if MB_CUR_MAX
is not a compile-time constant.
In the introduction of this chapter it was said that certain character sets use a stateful encoding. That is, the encoded values depend in some way on the previous bytes in the text.
Since the conversion functions allow converting a text in more than one step we must have a way to pass this information from one call of the functions to another.
A variable of type mbstate_t
can contain all the information
about the shift state needed from one call to a conversion
function to another.
mbstate_t
is defined in wchar.h. It was introduced in
Amendment 1 to ISO C90.
To use objects of type mbstate_t
the programmer has to define such
objects (normally as local variables on the stack) and pass a pointer to
the object to the conversion functions. This way the conversion function
can update the object if the current multibyte character set is stateful.
There is no specific function or initializer to put the state object in any specific state. The rules are that the object should always represent the initial state before the first use, and this is achieved by clearing the whole variable with code such as follows:
{
mbstate_t state;
memset (&state, '\0', sizeof (state));
/* from now on state can be used. */
...
}
When using the conversion functions to generate output it is often necessary to test whether the current state corresponds to the initial state. This is necessary, for example, to decide whether to emit escape sequences to set the state to the initial state at certain sequence points. Communication protocols often require this.
int
mbsinit (const mbstate_t *ps)
¶Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The mbsinit
function determines whether the state object pointed
to by ps is in the initial state. If ps is a null pointer or
the object is in the initial state the return value is nonzero. Otherwise
it is zero.
mbsinit
was introduced in Amendment 1 to ISO C90 and is
declared in wchar.h.
Code using mbsinit
often looks similar to this:
{ mbstate_t state; memset (&state, '\0', sizeof (state)); /* Use state. */ ... if (! mbsinit (&state)) { /* Emit code to return to initial state. */ const wchar_t empty[] = L""; const wchar_t *srcp = empty; wcsrtombs (outbuf, &srcp, outbuflen, &state); } ... }
The code to emit the escape sequence to get back to the initial state is
interesting. The wcsrtombs
function can be used to determine the
necessary output code (see Converting Multibyte and Wide Character Strings). Please note that with
the GNU C Library it is not necessary to perform this extra action for the
conversion from multibyte text to wide character text since the wide
character encoding is not stateful. But there is nothing mentioned in
any standard that prohibits making wchar_t
use a stateful
encoding.
The most fundamental of the conversion functions are those dealing with single characters. Please note that this does not always mean single bytes. But since there is very often a subset of the multibyte character set that consists of single byte sequences, there are functions to help with converting bytes. Frequently, ASCII is a subset of the multibyte character set. In such a scenario, each ASCII character stands for itself, and all other characters have at least a first byte that is beyond the range 0 to 127.
wint_t
btowc (int c)
¶Preliminary: | MT-Safe | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The btowc
function (“byte to wide character”) converts a valid
single byte character c in the initial shift state into the wide
character equivalent using the conversion rules from the currently
selected locale of the LC_CTYPE
category.
If (unsigned char) c
is no valid single byte multibyte
character or if c is EOF
, the function returns WEOF
.
Please note the restriction of c being tested for validity only in
the initial shift state. No mbstate_t
object is used from
which the state information is taken, and the function also does not use
any static state.
The btowc
function was introduced in Amendment 1 to ISO C90
and is declared in wchar.h.
Despite the limitation that the single byte value is always interpreted in the initial state, this function is actually useful most of the time. Most characters are either entirely single-byte character sets or they are extensions to ASCII. But then it is possible to write code like this (not that this specific example is very useful):
wchar_t * itow (unsigned long int val) { static wchar_t buf[30]; wchar_t *wcp = &buf[29]; *wcp = L'\0'; while (val != 0) { *--wcp = btowc ('0' + val % 10); val /= 10; } if (wcp == &buf[29]) *--wcp = L'0'; return wcp; }
Why is it necessary to use such a complicated implementation and not
simply cast '0' + val % 10
to a wide character? The answer is
that there is no guarantee that one can perform this kind of arithmetic
on the character of the character set used for wchar_t
representation. In other situations the bytes are not constant at
compile time and so the compiler cannot do the work. In situations like
this, using btowc
is required.
There is also a function for the conversion in the other direction.
int
wctob (wint_t c)
¶Preliminary: | MT-Safe | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The wctob
function (“wide character to byte”) takes as the
parameter a valid wide character. If the multibyte representation for
this character in the initial state is exactly one byte long, the return
value of this function is this character. Otherwise the return value is
EOF
.
wctob
was introduced in Amendment 1 to ISO C90 and
is declared in wchar.h.
There are more general functions to convert single characters from multibyte representation to wide characters and vice versa. These functions pose no limit on the length of the multibyte representation and they also do not require it to be in the initial state.
size_t
mbrtowc (wchar_t *restrict pwc, const char *restrict s, size_t n, mbstate_t *restrict ps)
¶Preliminary: | MT-Unsafe race:mbrtowc/!ps | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The mbrtowc
function (“multibyte restartable to wide
character”) converts the next multibyte character in the string pointed
to by s into a wide character and stores it in the location
pointed to by pwc. The conversion is performed according
to the locale currently selected for the LC_CTYPE
category. If
the conversion for the character set used in the locale requires a state,
the multibyte string is interpreted in the state represented by the
object pointed to by ps. If ps is a null pointer, a static,
internal state variable used only by the mbrtowc
function is
used.
If the next multibyte character corresponds to the null wide character,
the return value of the function is 0 and the state object is
afterwards in the initial state. If the next n or fewer bytes
form a correct multibyte character, the return value is the number of
bytes starting from s that form the multibyte character. The
conversion state is updated according to the bytes consumed in the
conversion. In both cases the wide character (either the L'\0'
or the one found in the conversion) is stored in the string pointed to
by pwc if pwc is not null.
If the first n bytes of the multibyte string possibly form a valid
multibyte character but there are more than n bytes needed to
complete it, the return value of the function is (size_t) -2
and
no value is stored in *pwc
. The conversion state is
updated and all n input bytes are consumed and should not be
submitted again. Please note that this can happen even if n has a
value greater than or equal to MB_CUR_MAX
since the input might
contain redundant shift sequences.
If the first n
bytes of the multibyte string cannot possibly form
a valid multibyte character, no value is stored, the global variable
errno
is set to the value EILSEQ
, and the function returns
(size_t) -1
. The conversion state is afterwards undefined.
As specified, the mbrtowc
function could deal with multibyte
sequences which contain embedded null bytes (which happens in Unicode
encodings such as UTF-16), but the GNU C Library does not support such
multibyte encodings. When encountering a null input byte, the function
will either return zero, or return (size_t) -1)
and report a
EILSEQ
error. The iconv
function can be used for
converting between arbitrary encodings. See Generic Character Set Conversion Interface.
mbrtowc
was introduced in Amendment 1 to ISO C90 and
is declared in wchar.h.
A function that copies a multibyte string into a wide character string while at the same time converting all lowercase characters into uppercase could look like this:
wchar_t * mbstouwcs (const char *s) { /* Include the null terminator in the conversion. */ size_t len = strlen (s) + 1; wchar_t *result = reallocarray (NULL, len, sizeof (wchar_t)); if (result == NULL) return NULL; wchar_t *wcp = result; mbstate_t state; memset (&state, '\0', sizeof (state)); while (true) { wchar_t wc; size_t nbytes = mbrtowc (&wc, s, len, &state); if (nbytes == 0) { /* Terminate the result string. */ *wcp = L'\0'; break; } else if (nbytes == (size_t) -2) { /* Truncated input string. */ errno = EILSEQ; free (result); return NULL; } else if (nbytes == (size_t) -1) { /* Some other error (including EILSEQ). */ free (result); return NULL; } else { /* A character was converted. */ *wcp++ = towupper (wc); len -= nbytes; s += nbytes; } } return result; }
In the inner loop, a single wide character is stored in wc
, and
the number of consumed bytes is stored in the variable nbytes
.
If the conversion is successful, the uppercase variant of the wide
character is stored in the result
array and the pointer to the
input string and the number of available bytes is adjusted. If the
mbrtowc
function returns zero, the null input byte has not been
converted, so it must be stored explicitly in the result.
The above code uses the fact that there can never be more wide characters in the converted result than there are bytes in the multibyte input string. This method yields a pessimistic guess about the size of the result, and if many wide character strings have to be constructed this way or if the strings are long, the extra memory required to be allocated because the input string contains multibyte characters might be significant. The allocated memory block can be resized to the correct size before returning it, but a better solution might be to allocate just the right amount of space for the result right away. Unfortunately there is no function to compute the length of the wide character string directly from the multibyte string. There is, however, a function that does part of the work.
size_t
mbrlen (const char *restrict s, size_t n, mbstate_t *ps)
¶Preliminary: | MT-Unsafe race:mbrlen/!ps | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The mbrlen
function (“multibyte restartable length”) computes
the number of at most n bytes starting at s, which form the
next valid and complete multibyte character.
If the next multibyte character corresponds to the NUL wide character, the return value is 0. If the next n bytes form a valid multibyte character, the number of bytes belonging to this multibyte character byte sequence is returned.
If the first n bytes possibly form a valid multibyte
character but the character is incomplete, the return value is
(size_t) -2
. Otherwise the multibyte character sequence is invalid
and the return value is (size_t) -1
.
The multibyte sequence is interpreted in the state represented by the
object pointed to by ps. If ps is a null pointer, a state
object local to mbrlen
is used.
mbrlen
was introduced in Amendment 1 to ISO C90 and
is declared in wchar.h.
The attentive reader now will note that mbrlen
can be implemented
as
mbrtowc (NULL, s, n, ps != NULL ? ps : &internal)
This is true and in fact is mentioned in the official specification.
How can this function be used to determine the length of the wide
character string created from a multibyte character string? It is not
directly usable, but we can define a function mbslen
using it:
size_t
mbslen (const char *s)
{
mbstate_t state;
size_t result = 0;
size_t nbytes;
memset (&state, '\0', sizeof (state));
while ((nbytes = mbrlen (s, MB_LEN_MAX, &state)) > 0)
{
if (nbytes >= (size_t) -2)
/* Something is wrong. */
return (size_t) -1;
s += nbytes;
++result;
}
return result;
}
This function simply calls mbrlen
for each multibyte character
in the string and counts the number of function calls. Please note that
we here use MB_LEN_MAX
as the size argument in the mbrlen
call. This is acceptable since a) this value is larger than the length of
the longest multibyte character sequence and b) we know that the string
s ends with a NUL byte, which cannot be part of any other multibyte
character sequence but the one representing the NUL wide character.
Therefore, the mbrlen
function will never read invalid memory.
Now that this function is available (just to make this clear, this function is not part of the GNU C Library) we can compute the number of wide characters required to store the converted multibyte character string s using
wcs_bytes = (mbslen (s) + 1) * sizeof (wchar_t);
Please note that the mbslen
function is quite inefficient. The
implementation of mbstouwcs
with mbslen
would have to
perform the conversion of the multibyte character input string twice, and
this conversion might be quite expensive. So it is necessary to think
about the consequences of using the easier but imprecise method before
doing the work twice.
size_t
wcrtomb (char *restrict s, wchar_t wc, mbstate_t *restrict ps)
¶Preliminary: | MT-Unsafe race:wcrtomb/!ps | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The wcrtomb
function (“wide character restartable to
multibyte”) converts a single wide character into a multibyte string
corresponding to that wide character.
If s is a null pointer, the function resets the state stored in
the object pointed to by ps (or the internal mbstate_t
object) to the initial state. This can also be achieved by a call like
this:
wcrtombs (temp_buf, L'\0', ps)
since, if s is a null pointer, wcrtomb
performs as if it
writes into an internal buffer, which is guaranteed to be large enough.
If wc is the NUL wide character, wcrtomb
emits, if
necessary, a shift sequence to get the state ps into the initial
state followed by a single NUL byte, which is stored in the string
s.
Otherwise a byte sequence (possibly including shift sequences) is written
into the string s. This only happens if wc is a valid wide
character (i.e., it has a multibyte representation in the character set
selected by locale of the LC_CTYPE
category). If wc is no
valid wide character, nothing is stored in the strings s,
errno
is set to EILSEQ
, the conversion state in ps
is undefined and the return value is (size_t) -1
.
If no error occurred the function returns the number of bytes stored in the string s. This includes all bytes representing shift sequences.
One word about the interface of the function: there is no parameter specifying the length of the array s, so the caller has to make sure that there is enough space available, otherwise buffer overruns can occur. This version of the GNU C Library does not assume that s is at least MB_CUR_MAX bytes long, but programs that need to run on GNU C Library versions that have this assumption documented in the manual must comply with this limit.
wcrtomb
was introduced in Amendment 1 to ISO C90 and is
declared in wchar.h.
Using wcrtomb
is as easy as using mbrtowc
. The following
example appends a wide character string to a multibyte character string.
Again, the code is not really useful (or correct), it is simply here to
demonstrate the use and some problems.
char * mbscatwcs (char *s, size_t len, const wchar_t *ws) { mbstate_t state; /* Find the end of the existing string. */ char *wp = strchr (s, '\0'); len -= wp - s; memset (&state, '\0', sizeof (state)); do { size_t nbytes; if (len < MB_CUR_LEN) { /* We cannot guarantee that the next character fits into the buffer, so return an error. */ errno = E2BIG; return NULL; } nbytes = wcrtomb (wp, *ws, &state); if (nbytes == (size_t) -1) /* Error in the conversion. */ return NULL; len -= nbytes; wp += nbytes; } while (*ws++ != L'\0'); return s; }
First the function has to find the end of the string currently in the
array s. The strchr
call does this very efficiently since a
requirement for multibyte character representations is that the NUL byte
is never used except to represent itself (and in this context, the end
of the string).
After initializing the state object the loop is entered where the first
task is to make sure there is enough room in the array s. We
abort if there are not at least MB_CUR_LEN
bytes available. This
is not always optimal but we have no other choice. We might have less
than MB_CUR_LEN
bytes available but the next multibyte character
might also be only one byte long. At the time the wcrtomb
call
returns it is too late to decide whether the buffer was large enough. If
this solution is unsuitable, there is a very slow but more accurate
solution.
... if (len < MB_CUR_LEN) { mbstate_t temp_state; memcpy (&temp_state, &state, sizeof (state)); if (wcrtomb (NULL, *ws, &temp_state) > len) { /* We cannot guarantee that the next character fits into the buffer, so return an error. */ errno = E2BIG; return NULL; } } ...
Here we perform the conversion that might overflow the buffer so that
we are afterwards in the position to make an exact decision about the
buffer size. Please note the NULL
argument for the destination
buffer in the new wcrtomb
call; since we are not interested in the
converted text at this point, this is a nice way to express this. The
most unusual thing about this piece of code certainly is the duplication
of the conversion state object, but if a change of the state is necessary
to emit the next multibyte character, we want to have the same shift state
change performed in the real conversion. Therefore, we have to preserve
the initial shift state information.
There are certainly many more and even better solutions to this problem. This example is only provided for educational purposes.
The functions described in the previous section only convert a single character at a time. Most operations to be performed in real-world programs include strings and therefore the ISO C standard also defines conversions on entire strings. However, the defined set of functions is quite limited; therefore, the GNU C Library contains a few extensions that can help in some important situations.
size_t
mbsrtowcs (wchar_t *restrict dst, const char **restrict src, size_t len, mbstate_t *restrict ps)
¶Preliminary: | MT-Unsafe race:mbsrtowcs/!ps | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The mbsrtowcs
function (“multibyte string restartable to wide
character string”) converts the NUL-terminated multibyte character
string at *src
into an equivalent wide character string,
including the NUL wide character at the end. The conversion is started
using the state information from the object pointed to by ps or
from an internal object of mbsrtowcs
if ps is a null
pointer. Before returning, the state object is updated to match the state
after the last converted character. The state is the initial state if the
terminating NUL byte is reached and converted.
If dst is not a null pointer, the result is stored in the array pointed to by dst; otherwise, the conversion result is not available since it is stored in an internal buffer.
If len wide characters are stored in the array dst before reaching the end of the input string, the conversion stops and len is returned. If dst is a null pointer, len is never checked.
Another reason for a premature return from the function call is if the
input string contains an invalid multibyte sequence. In this case the
global variable errno
is set to EILSEQ
and the function
returns (size_t) -1
.
In all other cases the function returns the number of wide characters
converted during this call. If dst is not null, mbsrtowcs
stores in the pointer pointed to by src either a null pointer (if
the NUL byte in the input string was reached) or the address of the byte
following the last converted multibyte character.
Like mbstowcs
the dst parameter may be a null pointer and
the function can be used to count the number of wide characters that
would be required.
mbsrtowcs
was introduced in Amendment 1 to ISO C90 and is
declared in wchar.h.
The definition of the mbsrtowcs
function has one important
limitation. The requirement that dst has to be a NUL-terminated
string provides problems if one wants to convert buffers with text. A
buffer is not normally a collection of NUL-terminated strings but instead a
continuous collection of lines, separated by newline characters. Now
assume that a function to convert one line from a buffer is needed. Since
the line is not NUL-terminated, the source pointer cannot directly point
into the unmodified text buffer. This means, either one inserts the NUL
byte at the appropriate place for the time of the mbsrtowcs
function call (which is not doable for a read-only buffer or in a
multi-threaded application) or one copies the line in an extra buffer
where it can be terminated by a NUL byte. Note that it is not in general
possible to limit the number of characters to convert by setting the
parameter len to any specific value. Since it is not known how
many bytes each multibyte character sequence is in length, one can only
guess.
There is still a problem with the method of NUL-terminating a line right
after the newline character, which could lead to very strange results.
As said in the description of the mbsrtowcs
function above, the
conversion state is guaranteed to be in the initial shift state after
processing the NUL byte at the end of the input string. But this NUL
byte is not really part of the text (i.e., the conversion state after
the newline in the original text could be something different than the
initial shift state and therefore the first character of the next line
is encoded using this state). But the state in question is never
accessible to the user since the conversion stops after the NUL byte
(which resets the state). Most stateful character sets in use today
require that the shift state after a newline be the initial state–but
this is not a strict guarantee. Therefore, simply NUL-terminating a
piece of a running text is not always an adequate solution and,
therefore, should never be used in generally used code.
The generic conversion interface (see Generic Charset Conversion)
does not have this limitation (it simply works on buffers, not
strings), and the GNU C Library contains a set of functions that take
additional parameters specifying the maximal number of bytes that are
consumed from the input string. This way the problem of
mbsrtowcs
’s example above could be solved by determining the line
length and passing this length to the function.
size_t
wcsrtombs (char *restrict dst, const wchar_t **restrict src, size_t len, mbstate_t *restrict ps)
¶Preliminary: | MT-Unsafe race:wcsrtombs/!ps | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The wcsrtombs
function (“wide character string restartable to
multibyte string”) converts the NUL-terminated wide character string at
*src
into an equivalent multibyte character string and
stores the result in the array pointed to by dst. The NUL wide
character is also converted. The conversion starts in the state
described in the object pointed to by ps or by a state object
local to wcsrtombs
in case ps is a null pointer. If
dst is a null pointer, the conversion is performed as usual but the
result is not available. If all characters of the input string were
successfully converted and if dst is not a null pointer, the
pointer pointed to by src gets assigned a null pointer.
If one of the wide characters in the input string has no valid multibyte
character equivalent, the conversion stops early, sets the global
variable errno
to EILSEQ
, and returns (size_t) -1
.
Another reason for a premature stop is if dst is not a null pointer and the next converted character would require more than len bytes in total to the array dst. In this case (and if dst is not a null pointer) the pointer pointed to by src is assigned a value pointing to the wide character right after the last one successfully converted.
Except in the case of an encoding error the return value of the
wcsrtombs
function is the number of bytes in all the multibyte
character sequences which were or would have been (if dst was
not a null) stored in dst. Before returning, the state in the
object pointed to by ps (or the internal object in case ps
is a null pointer) is updated to reflect the state after the last
conversion. The state is the initial shift state in case the
terminating NUL wide character was converted.
The wcsrtombs
function was introduced in Amendment 1 to
ISO C90 and is declared in wchar.h.
The restriction mentioned above for the mbsrtowcs
function applies
here also. There is no possibility of directly controlling the number of
input characters. One has to place the NUL wide character at the correct
place or control the consumed input indirectly via the available output
array size (the len parameter).
size_t
mbsnrtowcs (wchar_t *restrict dst, const char **restrict src, size_t nmc, size_t len, mbstate_t *restrict ps)
¶Preliminary: | MT-Unsafe race:mbsnrtowcs/!ps | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The mbsnrtowcs
function is very similar to the mbsrtowcs
function. All the parameters are the same except for nmc, which is
new. The return value is the same as for mbsrtowcs
.
This new parameter specifies how many bytes at most can be used from the
multibyte character string. In other words, the multibyte character
string *src
need not be NUL-terminated. But if a NUL byte
is found within the nmc first bytes of the string, the conversion
stops there.
Like mbstowcs
the dst parameter may be a null pointer and
the function can be used to count the number of wide characters that
would be required.
This function is a GNU extension. It is meant to work around the problems mentioned above. Now it is possible to convert a buffer with multibyte character text piece by piece without having to care about inserting NUL bytes and the effect of NUL bytes on the conversion state.
A function to convert a multibyte string into a wide character string and display it could be written like this (this is not a really useful example):
void
showmbs (const char *src, FILE *fp)
{
mbstate_t state;
int cnt = 0;
memset (&state, '\0', sizeof (state));
while (1)
{
wchar_t linebuf[100];
const char *endp = strchr (src, '\n');
size_t n;
/* Exit if there is no more line. */
if (endp == NULL)
break;
n = mbsnrtowcs (linebuf, &src, endp - src, 99, &state);
linebuf[n] = L'\0';
fprintf (fp, "line %d: \"%S\"\n", linebuf);
}
}
There is no problem with the state after a call to mbsnrtowcs
.
Since we don’t insert characters in the strings that were not in there
right from the beginning and we use state only for the conversion
of the given buffer, there is no problem with altering the state.
size_t
wcsnrtombs (char *restrict dst, const wchar_t **restrict src, size_t nwc, size_t len, mbstate_t *restrict ps)
¶Preliminary: | MT-Unsafe race:wcsnrtombs/!ps | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The wcsnrtombs
function implements the conversion from wide
character strings to multibyte character strings. It is similar to
wcsrtombs
but, just like mbsnrtowcs
, it takes an extra
parameter, which specifies the length of the input string.
No more than nwc wide characters from the input string
*src
are converted. If the input string contains a NUL
wide character in the first nwc characters, the conversion stops at
this place.
The wcsnrtombs
function is a GNU extension and just like
mbsnrtowcs
helps in situations where no NUL-terminated input
strings are available.
The example programs given in the last sections are only brief and do
not contain all the error checking, etc. Presented here is a complete
and documented example. It features the mbrtowc
function but it
should be easy to derive versions using the other functions.
int file_mbsrtowcs (int input, int output) { /* Note the use ofMB_LEN_MAX
.MB_CUR_MAX
cannot portably be used here. */ char buffer[BUFSIZ + MB_LEN_MAX]; mbstate_t state; int filled = 0; int eof = 0; /* Initialize the state. */ memset (&state, '\0', sizeof (state)); while (!eof) { ssize_t nread; ssize_t nwrite; char *inp = buffer; wchar_t outbuf[BUFSIZ]; wchar_t *outp = outbuf; /* Fill up the buffer from the input file. */ nread = read (input, buffer + filled, BUFSIZ); if (nread < 0) { perror ("read"); return 0; } /* If we reach end of file, make a note to read no more. */ if (nread == 0) eof = 1; /*filled
is now the number of bytes inbuffer
. */ filled += nread; /* Convert those bytes to wide characters–as many as we can. */ while (1) { size_t thislen = mbrtowc (outp, inp, filled, &state); /* Stop converting at invalid character; this can mean we have read just the first part of a valid character. */ if (thislen == (size_t) -1) break; /* We want to handle embedded NUL bytes but the return value is 0. Correct this. */ if (thislen == 0) thislen = 1; /* Advance past this character. */ inp += thislen; filled -= thislen; ++outp; } /* Write the wide characters we just made. */ nwrite = write (output, outbuf, (outp - outbuf) * sizeof (wchar_t)); if (nwrite < 0) { perror ("write"); return 0; } /* See if we have a real invalid character. */ if ((eof && filled > 0) || filled >= MB_CUR_MAX) { error (0, 0, "invalid multibyte character"); return 0; } /* If any characters must be carried forward, put them at the beginning ofbuffer
. */ if (filled > 0) memmove (buffer, inp, filled); } return 1; }
The functions described in the previous chapter are defined in Amendment 1 to ISO C90, but the original ISO C90 standard also contained functions for character set conversion. The reason that these original functions are not described first is that they are almost entirely useless.
The problem is that all the conversion functions described in the original ISO C90 use a local state. Using a local state implies that multiple conversions at the same time (not only when using threads) cannot be done, and that you cannot first convert single characters and then strings since you cannot tell the conversion functions which state to use.
These original functions are therefore usable only in a very limited set of situations. One must complete converting the entire string before starting a new one, and each string/text must be converted with the same function (there is no problem with the library itself; it is guaranteed that no library function changes the state of any of these functions). For the above reasons it is highly requested that the functions described in the previous section be used in place of non-reentrant conversion functions.
int
mbtowc (wchar_t *restrict result, const char *restrict string, size_t size)
¶Preliminary: | MT-Unsafe race | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The mbtowc
(“multibyte to wide character”) function when called
with non-null string converts the first multibyte character
beginning at string to its corresponding wide character code. It
stores the result in *result
.
mbtowc
never examines more than size bytes. (The idea is
to supply for size the number of bytes of data you have in hand.)
mbtowc
with non-null string distinguishes three
possibilities: the first size bytes at string start with
valid multibyte characters, they start with an invalid byte sequence or
just part of a character, or string points to an empty string (a
null character).
For a valid multibyte character, mbtowc
converts it to a wide
character and stores that in *result
, and returns the
number of bytes in that character (always at least 1 and never
more than size).
For an invalid byte sequence, mbtowc
returns -1. For an
empty string, it returns 0, also storing '\0'
in
*result
.
If the multibyte character code uses shift characters, then
mbtowc
maintains and updates a shift state as it scans. If you
call mbtowc
with a null pointer for string, that
initializes the shift state to its standard initial value. It also
returns nonzero if the multibyte character code in use actually has a
shift state. See States in Non-reentrant Functions.
int
wctomb (char *string, wchar_t wchar)
¶Preliminary: | MT-Unsafe race | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The wctomb
(“wide character to multibyte”) function converts
the wide character code wchar to its corresponding multibyte
character sequence, and stores the result in bytes starting at
string. At most MB_CUR_MAX
characters are stored.
wctomb
with non-null string distinguishes three
possibilities for wchar: a valid wide character code (one that can
be translated to a multibyte character), an invalid code, and
L'\0'
.
Given a valid code, wctomb
converts it to a multibyte character,
storing the bytes starting at string. Then it returns the number
of bytes in that character (always at least 1 and never more
than MB_CUR_MAX
).
If wchar is an invalid wide character code, wctomb
returns
-1. If wchar is L'\0'
, it returns 0
, also
storing '\0'
in *string
.
If the multibyte character code uses shift characters, then
wctomb
maintains and updates a shift state as it scans. If you
call wctomb
with a null pointer for string, that
initializes the shift state to its standard initial value. It also
returns nonzero if the multibyte character code in use actually has a
shift state. See States in Non-reentrant Functions.
Calling this function with a wchar argument of zero when
string is not null has the side-effect of reinitializing the
stored shift state as well as storing the multibyte character
'\0'
and returning 0.
Similar to mbrlen
there is also a non-reentrant function that
computes the length of a multibyte character. It can be defined in
terms of mbtowc
.
int
mblen (const char *string, size_t size)
¶Preliminary: | MT-Unsafe race | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The mblen
function with a non-null string argument returns
the number of bytes that make up the multibyte character beginning at
string, never examining more than size bytes. (The idea is
to supply for size the number of bytes of data you have in hand.)
The return value of mblen
distinguishes three possibilities: the
first size bytes at string start with valid multibyte
characters, they start with an invalid byte sequence or just part of a
character, or string points to an empty string (a null character).
For a valid multibyte character, mblen
returns the number of
bytes in that character (always at least 1
and never more than
size). For an invalid byte sequence, mblen
returns
-1. For an empty string, it returns 0.
If the multibyte character code uses shift characters, then mblen
maintains and updates a shift state as it scans. If you call
mblen
with a null pointer for string, that initializes the
shift state to its standard initial value. It also returns a nonzero
value if the multibyte character code in use actually has a shift state.
See States in Non-reentrant Functions.
The function mblen
is declared in stdlib.h.
For convenience the ISO C90 standard also defines functions to convert entire strings instead of single characters. These functions suffer from the same problems as their reentrant counterparts from Amendment 1 to ISO C90; see Converting Multibyte and Wide Character Strings.
size_t
mbstowcs (wchar_t *wstring, const char *string, size_t size)
¶Preliminary: | MT-Safe | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The mbstowcs
(“multibyte string to wide character string”)
function converts the null-terminated string of multibyte characters
string to an array of wide character codes, storing not more than
size wide characters into the array beginning at wstring.
The terminating null character counts towards the size, so if size
is less than the actual number of wide characters resulting from
string, no terminating null character is stored.
The conversion of characters from string begins in the initial shift state.
If an invalid multibyte character sequence is found, the mbstowcs
function returns a value of -1. Otherwise, it returns the number
of wide characters stored in the array wstring. This number does
not include the terminating null character, which is present if the
number is less than size.
Here is an example showing how to convert a string of multibyte characters, allocating enough space for the result.
wchar_t * mbstowcs_alloc (const char *string) { size_t size = strlen (string) + 1; wchar_t *buf = xmalloc (size * sizeof (wchar_t)); size = mbstowcs (buf, string, size); if (size == (size_t) -1) return NULL; buf = xreallocarray (buf, size + 1, sizeof *buf); return buf; }
If wstring is a null pointer then no output is written and the conversion proceeds as above, and the result is returned. In practice such behaviour is useful for calculating the exact number of wide characters required to convert string. This behaviour of accepting a null pointer for wstring is an XPG4.2 extension that is not specified in ISO C and is optional in POSIX.
size_t
wcstombs (char *string, const wchar_t *wstring, size_t size)
¶Preliminary: | MT-Safe | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The wcstombs
(“wide character string to multibyte string”)
function converts the null-terminated wide character array wstring
into a string containing multibyte characters, storing not more than
size bytes starting at string, followed by a terminating
null character if there is room. The conversion of characters begins in
the initial shift state.
The terminating null character counts towards the size, so if size is less than or equal to the number of bytes needed in wstring, no terminating null character is stored.
If a code that does not correspond to a valid multibyte character is
found, the wcstombs
function returns a value of -1.
Otherwise, the return value is the number of bytes stored in the array
string. This number does not include the terminating null character,
which is present if the number is less than size.
In some multibyte character codes, the meaning of any particular byte sequence is not fixed; it depends on what other sequences have come earlier in the same string. Typically there are just a few sequences that can change the meaning of other sequences; these few are called shift sequences and we say that they set the shift state for other sequences that follow.
To illustrate shift state and shift sequences, suppose we decide that
the sequence 0200
(just one byte) enters Japanese mode, in which
pairs of bytes in the range from 0240
to 0377
are single
characters, while 0201
enters Latin-1 mode, in which single bytes
in the range from 0240
to 0377
are characters, and
interpreted according to the ISO Latin-1 character set. This is a
multibyte code that has two alternative shift states (“Japanese mode”
and “Latin-1 mode”), and two shift sequences that specify particular
shift states.
When the multibyte character code in use has shift states, then
mblen
, mbtowc
, and wctomb
must maintain and update
the current shift state as they scan the string. To make this work
properly, you must follow these rules:
mblen (NULL,
0)
. This initializes the shift state to its standard initial value.
Here is an example of using mblen
following these rules:
void scan_string (char *s) { int length = strlen (s); /* Initialize shift state. */ mblen (NULL, 0); while (1) { int thischar = mblen (s, length); /* Deal with end of string and invalid characters. */ if (thischar == 0) break; if (thischar == -1) { error ("invalid multibyte character"); break; } /* Advance past this character. */ s += thischar; length -= thischar; } }
The functions mblen
, mbtowc
and wctomb
are not
reentrant when using a multibyte code that uses a shift state. However,
no other library functions call these functions, so you don’t have to
worry that the shift state will be changed mysteriously.
The conversion functions mentioned so far in this chapter all had in
common that they operate on character sets that are not directly
specified by the functions. The multibyte encoding used is specified by
the currently selected locale for the LC_CTYPE
category. The
wide character set is fixed by the implementation (in the case of the GNU C Library
it is always UCS-4 encoded ISO 10646).
This has of course several problems when it comes to general character conversion:
LC_CTYPE
category, one has to change the LC_CTYPE
locale using
setlocale
.
Changing the LC_CTYPE
locale introduces major problems for the rest
of the programs since several more functions (e.g., the character
classification functions, see Classification of Characters) use the
LC_CTYPE
category.
LC_CTYPE
selection is global and shared by all
threads.
wchar_t
representation, there is at least a two-step
process necessary to convert a text using the functions above. One would
have to select the source character set as the multibyte encoding,
convert the text into a wchar_t
text, select the destination
character set as the multibyte encoding, and convert the wide character
text to the multibyte (= destination) character set.
Even if this is possible (which is not guaranteed) it is a very tiring work. Plus it suffers from the other two raised points even more due to the steady changing of the locale.
The XPG2 standard defines a completely new set of functions, which has none of these limitations. They are not at all coupled to the selected locales, and they have no constraints on the character sets selected for source and destination. Only the set of available conversions limits them. The standard does not specify that any conversion at all must be available. Such availability is a measure of the quality of the implementation.
In the following text first the interface to iconv
and then the
conversion function, will be described. Comparisons with other
implementations will show what obstacles stand in the way of portable
applications. Finally, the implementation is described in so far as might
interest the advanced user who wants to extend conversion capabilities.
iconv
exampleiconv
Implementationsiconv
Implementation in the GNU C LibraryThis set of functions follows the traditional cycle of using a resource: open–use–close. The interface consists of three functions, each of which implements one step.
Before the interfaces are described it is necessary to introduce a data type. Just like other open–use–close interfaces the functions introduced here work using handles and the iconv.h header defines a special type for the handles used.
This data type is an abstract type defined in iconv.h. The user must not assume anything about the definition of this type; it must be completely opaque.
Objects of this type can be assigned handles for the conversions using
the iconv
functions. The objects themselves need not be freed, but
the conversions for which the handles stand for have to.
The first step is the function to create a handle.
iconv_t
iconv_open (const char *tocode, const char *fromcode)
¶Preliminary: | MT-Safe locale | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem fd | See POSIX Safety Concepts.
The iconv_open
function has to be used before starting a
conversion. The two parameters this function takes determine the
source and destination character set for the conversion, and if the
implementation has the possibility to perform such a conversion, the
function returns a handle.
If the wanted conversion is not available, the iconv_open
function
returns (iconv_t) -1
. In this case the global variable
errno
can have the following values:
EMFILE
The process already has OPEN_MAX
file descriptors open.
ENFILE
The system limit of open files is reached.
ENOMEM
Not enough memory to carry out the operation.
EINVAL
The conversion from fromcode to tocode is not supported.
It is not possible to use the same descriptor in different threads to perform independent conversions. The data structures associated with the descriptor include information about the conversion state. This must not be messed up by using it in different conversions.
An iconv
descriptor is like a file descriptor as for every use a
new descriptor must be created. The descriptor does not stand for all
of the conversions from fromset to toset.
The GNU C Library implementation of iconv_open
has one
significant extension to other implementations. To ease the extension
of the set of available conversions, the implementation allows storing
the necessary files with data and code in an arbitrary number of
directories. How this extension must be written will be explained below
(see The iconv
Implementation in the GNU C Library). Here it is only important to say
that all directories mentioned in the GCONV_PATH
environment
variable are considered only if they contain a file gconv-modules.
These directories need not necessarily be created by the system
administrator. In fact, this extension is introduced to help users
writing and using their own, new conversions. Of course, this does not
work for security reasons in SUID binaries; in this case only the system
directory is considered and this normally is
prefix/lib/gconv. The GCONV_PATH
environment
variable is examined exactly once at the first call of the
iconv_open
function. Later modifications of the variable have no
effect.
The iconv_open
function was introduced early in the X/Open
Portability Guide, version 2. It is supported by all commercial
Unices as it is required for the Unix branding. However, the quality and
completeness of the implementation varies widely. The iconv_open
function is declared in iconv.h.
The iconv
implementation can associate large data structure with
the handle returned by iconv_open
. Therefore, it is crucial to
free all the resources once all conversions are carried out and the
conversion is not needed anymore.
int
iconv_close (iconv_t cd)
¶Preliminary: | MT-Safe | AS-Unsafe corrupt heap lock dlopen | AC-Unsafe corrupt lock mem | See POSIX Safety Concepts.
The iconv_close
function frees all resources associated with the
handle cd, which must have been returned by a successful call to
the iconv_open
function.
If the function call was successful the return value is 0.
Otherwise it is -1 and errno
is set appropriately.
Defined errors are:
EBADF
The conversion descriptor is invalid.
The iconv_close
function was introduced together with the rest
of the iconv
functions in XPG2 and is declared in iconv.h.
The standard defines only one actual conversion function. This has, therefore, the most general interface: it allows conversion from one buffer to another. Conversion from a file to a buffer, vice versa, or even file to file can be implemented on top of it.
size_t
iconv (iconv_t cd, char **inbuf, size_t *inbytesleft, char **outbuf, size_t *outbytesleft)
¶Preliminary: | MT-Safe race:cd | AS-Safe | AC-Unsafe corrupt | See POSIX Safety Concepts.
The iconv
function converts the text in the input buffer
according to the rules associated with the descriptor cd and
stores the result in the output buffer. It is possible to call the
function for the same text several times in a row since for stateful
character sets the necessary state information is kept in the data
structures associated with the descriptor.
The input buffer is specified by *inbuf
and it contains
*inbytesleft
bytes. The extra indirection is necessary for
communicating the used input back to the caller (see below). It is
important to note that the buffer pointer is of type char
and the
length is measured in bytes even if the input text is encoded in wide
characters.
The output buffer is specified in a similar way. *outbuf
points to the beginning of the buffer with at least
*outbytesleft
bytes room for the result. The buffer
pointer again is of type char
and the length is measured in
bytes. If outbuf or *outbuf
is a null pointer, the
conversion is performed but no output is available.
If inbuf is a null pointer, the iconv
function performs the
necessary action to put the state of the conversion into the initial
state. This is obviously a no-op for non-stateful encodings, but if the
encoding has a state, such a function call might put some byte sequences
in the output buffer, which perform the necessary state changes. The
next call with inbuf not being a null pointer then simply goes on
from the initial state. It is important that the programmer never makes
any assumption as to whether the conversion has to deal with states.
Even if the input and output character sets are not stateful, the
implementation might still have to keep states. This is due to the
implementation chosen for the GNU C Library as it is described below.
Therefore an iconv
call to reset the state should always be
performed if some protocol requires this for the output text.
The conversion stops for one of three reasons. The first is that all characters from the input buffer are converted. This actually can mean two things: either all bytes from the input buffer are consumed or there are some bytes at the end of the buffer that possibly can form a complete character but the input is incomplete. The second reason for a stop is that the output buffer is full. And the third reason is that the input contains invalid characters.
In all of these cases the buffer pointers after the last successful conversion, for the input and output buffers, are stored in inbuf and outbuf, and the available room in each buffer is stored in inbytesleft and outbytesleft.
Since the character sets selected in the iconv_open
call can be
almost arbitrary, there can be situations where the input buffer contains
valid characters, which have no identical representation in the output
character set. The behavior in this situation is undefined. The
current behavior of the GNU C Library in this situation is to
return with an error immediately. This certainly is not the most
desirable solution; therefore, future versions will provide better ones,
but they are not yet finished.
If all input from the input buffer is successfully converted and stored
in the output buffer, the function returns the number of non-reversible
conversions performed. In all other cases the return value is
(size_t) -1
and errno
is set appropriately. In such cases
the value pointed to by inbytesleft is nonzero.
EILSEQ
The conversion stopped because of an invalid byte sequence in the input.
After the call, *inbuf
points at the first byte of the
invalid byte sequence.
E2BIG
The conversion stopped because it ran out of space in the output buffer.
EINVAL
The conversion stopped because of an incomplete byte sequence at the end of the input buffer.
EBADF
The cd argument is invalid.
The iconv
function was introduced in the XPG2 standard and is
declared in the iconv.h header.
The definition of the iconv
function is quite good overall. It
provides quite flexible functionality. The only problems lie in the
boundary cases, which are incomplete byte sequences at the end of the
input buffer and invalid input. A third problem, which is not really
a design problem, is the way conversions are selected. The standard
does not say anything about the legitimate names, a minimal set of
available conversions. We will see how this negatively impacts other
implementations, as demonstrated below.
iconv
example ¶The example below features a solution for a common problem. Given that
one knows the internal encoding used by the system for wchar_t
strings, one often is in the position to read text from a file and store
it in wide character buffers. One can do this using mbsrtowcs
,
but then we run into the problems discussed above.
int
file2wcs (int fd, const char *charset, wchar_t *outbuf, size_t avail)
{
char inbuf[BUFSIZ];
size_t insize = 0;
char *wrptr = (char *) outbuf;
int result = 0;
iconv_t cd;
cd = iconv_open ("WCHAR_T", charset);
if (cd == (iconv_t) -1)
{
/* Something went wrong. */
if (errno == EINVAL)
error (0, 0, "conversion from '%s' to wchar_t not available",
charset);
else
perror ("iconv_open");
/* Terminate the output string. */
*outbuf = L'\0';
return -1;
}
while (avail > 0)
{
size_t nread;
size_t nconv;
char *inptr = inbuf;
/* Read more input. */
nread = read (fd, inbuf + insize, sizeof (inbuf) - insize);
if (nread == 0)
{
/* When we come here the file is completely read.
This still could mean there are some unused
characters in the inbuf
. Put them back. */
if (lseek (fd, -insize, SEEK_CUR) == -1)
result = -1;
/* Now write out the byte sequence to get into the
initial state if this is necessary. */
iconv (cd, NULL, NULL, &wrptr, &avail);
break;
}
insize += nread;
/* Do the conversion. */
nconv = iconv (cd, &inptr, &insize, &wrptr, &avail);
if (nconv == (size_t) -1)
{
/* Not everything went right. It might only be
an unfinished byte sequence at the end of the
buffer. Or it is a real problem. */
if (errno == EINVAL)
/* This is harmless. Simply move the unused
bytes to the beginning of the buffer so that
they can be used in the next round. */
memmove (inbuf, inptr, insize);
else
{
/* It is a real problem. Maybe we ran out of
space in the output buffer or we have invalid
input. In any case back the file pointer to
the position of the last processed byte. */
lseek (fd, -insize, SEEK_CUR);
result = -1;
break;
}
}
}
/* Terminate the output string. */
if (avail >= sizeof (wchar_t))
*((wchar_t *) wrptr) = L'\0';
if (iconv_close (cd) != 0)
perror ("iconv_close");
return (wchar_t *) wrptr - outbuf;
}
This example shows the most important aspects of using the iconv
functions. It shows how successive calls to iconv
can be used to
convert large amounts of text. The user does not have to care about
stateful encodings as the functions take care of everything.
An interesting point is the case where iconv
returns an error and
errno
is set to EINVAL
. This is not really an error in the
transformation. It can happen whenever the input character set contains
byte sequences of more than one byte for some character and texts are not
processed in one piece. In this case there is a chance that a multibyte
sequence is cut. The caller can then simply read the remainder of the
takes and feed the offending bytes together with new character from the
input to iconv
and continue the work. The internal state kept in
the descriptor is not unspecified after such an event as is the
case with the conversion functions from the ISO C standard.
The example also shows the problem of using wide character strings with
iconv
. As explained in the description of the iconv
function above, the function always takes a pointer to a char
array and the available space is measured in bytes. In the example, the
output buffer is a wide character buffer; therefore, we use a local
variable wrptr of type char *
, which is used in the
iconv
calls.
This looks rather innocent but can lead to problems on platforms that
have tight restriction on alignment. Therefore the caller of iconv
has to make sure that the pointers passed are suitable for access of
characters from the appropriate character set. Since, in the
above case, the input parameter to the function is a wchar_t
pointer, this is the case (unless the user violates alignment when
computing the parameter). But in other situations, especially when
writing generic functions where one does not know what type of character
set one uses and, therefore, treats text as a sequence of bytes, it might
become tricky.
iconv
Implementations ¶This is not really the place to discuss the iconv
implementation
of other systems but it is necessary to know a bit about them to write
portable programs. The above mentioned problems with the specification
of the iconv
functions can lead to portability issues.
The first thing to notice is that, due to the large number of character sets in use, it is certainly not practical to encode the conversions directly in the C library. Therefore, the conversion information must come from files outside the C library. This is usually done in one or both of the following ways:
This solution is problematic as it requires a great deal of effort to apply to all character sets (potentially an infinite set). The differences in the structure of the different character sets is so large that many different variants of the table-processing functions must be developed. In addition, the generic nature of these functions make them slower than specifically implemented functions.
This solution provides much more flexibility. The C library itself contains only very little code and therefore reduces the general memory footprint. Also, with a documented interface between the C library and the loadable modules it is possible for third parties to extend the set of available conversion modules. A drawback of this solution is that dynamic loading must be available.
Some implementations in commercial Unices implement a mixture of these possibilities; the majority implement only the second solution. Using loadable modules moves the code out of the library itself and keeps the door open for extensions and improvements, but this design is also limiting on some platforms since not many platforms support dynamic loading in statically linked programs. On platforms without this capability it is therefore not possible to use this interface in statically linked programs. The GNU C Library has, on ELF platforms, no problems with dynamic loading in these situations; therefore, this point is moot. The danger is that one gets acquainted with this situation and forgets about the restrictions on other systems.
A second thing to know about other iconv
implementations is that
the number of available conversions is often very limited. Some
implementations provide, in the standard release (not special
international or developer releases), at most 100 to 200 conversion
possibilities. This does not mean 200 different character sets are
supported; for example, conversions from one character set to a set of 10
others might count as 10 conversions. Together with the other direction
this makes 20 conversion possibilities used up by one character set. One
can imagine the thin coverage these platforms provide. Some Unix vendors
even provide only a handful of conversions, which renders them useless for
almost all uses.
This directly leads to a third and probably the most problematic point.
The way the iconv
conversion functions are implemented on all
known Unix systems and the availability of the conversion functions from
character set A to B and the conversion from
B to C does not imply that the
conversion from A to C is available.
This might not seem unreasonable and problematic at first, but it is a quite big problem as one will notice shortly after hitting it. To show the problem we assume to write a program that has to convert from A to C. A call like
cd = iconv_open ("C", "A");
fails according to the assumption above. But what does the program do now? The conversion is necessary; therefore, simply giving up is not an option.
This is a nuisance. The iconv
function should take care of this.
But how should the program proceed from here on? If it tries to convert
to character set B, first the two iconv_open
calls
cd1 = iconv_open ("B", "A");
and
cd2 = iconv_open ("C", "B");
will succeed, but how to find B?
Unfortunately, the answer is: there is no general solution. On some systems guessing might help. On those systems most character sets can convert to and from UTF-8 encoded ISO 10646 or Unicode text. Besides this only some very system-specific methods can help. Since the conversion functions come from loadable modules and these modules must be stored somewhere in the filesystem, one could try to find them and determine from the available file which conversions are available and whether there is an indirect route from A to C.
This example shows one of the design errors of iconv
mentioned
above. It should at least be possible to determine the list of available
conversions programmatically so that if iconv_open
says there is no
such conversion, one could make sure this also is true for indirect
routes.
iconv
Implementation in the GNU C Library ¶After reading about the problems of iconv
implementations in the
last section it is certainly good to note that the implementation in
the GNU C Library has none of the problems mentioned above. What
follows is a step-by-step analysis of the points raised above. The
evaluation is based on the current state of the development (as of
January 1999). The development of the iconv
functions is not
complete, but basic functionality has solidified.
The GNU C Library’s iconv
implementation uses shared loadable
modules to implement the conversions. A very small number of
conversions are built into the library itself but these are only rather
trivial conversions.
All the benefits of loadable modules are available in the GNU C Library
implementation. This is especially appealing since the interface is
well documented (see below), and it, therefore, is easy to write new
conversion modules. The drawback of using loadable objects is not a
problem in the GNU C Library, at least on ELF systems. Since the
library is able to load shared objects even in statically linked
binaries, static linking need not be forbidden in case one wants to use
iconv
.
The second mentioned problem is the number of supported conversions. Currently, the GNU C Library supports more than 150 character sets. The way the implementation is designed the number of supported conversions is greater than 22350 (150 times 149). If any conversion from or to a character set is missing, it can be added easily.
Particularly impressive as it may be, this high number is due to the
fact that the GNU C Library implementation of iconv
does not have
the third problem mentioned above (i.e., whenever there is a conversion
from a character set A to B and from
B to C it is always possible to convert from
A to C directly). If the iconv_open
returns an error and sets errno
to EINVAL
, there is no
known way, directly or indirectly, to perform the wanted conversion.
Triangulation is achieved by providing for each character set a conversion from and to UCS-4 encoded ISO 10646. Using ISO 10646 as an intermediate representation it is possible to triangulate (i.e., convert with an intermediate representation).
There is no inherent requirement to provide a conversion to ISO 10646 for a new character set, and it is also possible to provide other conversions where neither source nor destination character set is ISO 10646. The existing set of conversions is simply meant to cover all conversions that might be of interest.
All currently available conversions use the triangulation method above, making conversion run unnecessarily slow. If, for example, somebody often needs the conversion from ISO-2022-JP to EUC-JP, a quicker solution would involve direct conversion between the two character sets, skipping the input to ISO 10646 first. The two character sets of interest are much more similar to each other than to ISO 10646.
In such a situation one easily can write a new conversion and provide it
as a better alternative. The GNU C Library iconv
implementation
would automatically use the module implementing the conversion if it is
specified to be more efficient.
iconv
iconv
module data structuresiconv
module interfacesAll information about the available conversions comes from a file named
gconv-modules, which can be found in any of the directories along
the GCONV_PATH
. The gconv-modules files are line-oriented
text files, where each of the lines has one of the following formats:
alias
define an alias name for a character
set. Two more words are expected on the line. The first word
defines the alias name, and the second defines the original name of the
character set. The effect is that it is possible to use the alias name
in the fromset or toset parameters of iconv_open
and
achieve the same result as when using the real character set name.
This is quite important as a character set has often many different
names. There is normally an official name but this need not correspond to
the most popular name. Besides this many character sets have special
names that are somehow constructed. For example, all character sets
specified by the ISO have an alias of the form ISO-IR-nnn
where nnn is the registration number. This allows programs that
know about the registration number to construct character set names and
use them in iconv_open
calls. More on the available names and
aliases follows below.
module
introduce an available conversion
module. These lines must contain three or four more words.
The first word specifies the source character set, the second word the destination character set of conversion implemented in this module, and the third word is the name of the loadable module. The filename is constructed by appending the usual shared object suffix (normally .so) and this file is then supposed to be found in the same directory the gconv-modules file is in. The last word on the line, which is optional, is a numeric value representing the cost of the conversion. If this word is missing, a cost of 1 is assumed. The numeric value itself does not matter that much; what counts are the relative values of the sums of costs for all possible conversion paths. Below is a more precise description of the use of the cost value.
Returning to the example above where one has written a module to directly convert from ISO-2022-JP to EUC-JP and back. All that has to be done is to put the new module, let its name be ISO2022JP-EUCJP.so, in a directory and add a file gconv-modules with the following content in the same directory:
module ISO-2022-JP// EUC-JP// ISO2022JP-EUCJP 1 module EUC-JP// ISO-2022-JP// ISO2022JP-EUCJP 1
To see why this is sufficient, it is necessary to understand how the
conversion used by iconv
(and described in the descriptor) is
selected. The approach to this problem is quite simple.
At the first call of the iconv_open
function the program reads
all available gconv-modules files and builds up two tables: one
containing all the known aliases and another that contains the
information about the conversions and which shared object implements
them.
iconv
¶The set of available conversions form a directed graph with weighted
edges. The weights on the edges are the costs specified in the
gconv-modules files. The iconv_open
function uses an
algorithm suitable for search for the best path in such a graph and so
constructs a list of conversions that must be performed in succession
to get the transformation from the source to the destination character
set.
Explaining why the above gconv-modules files allows the
iconv
implementation to resolve the specific ISO-2022-JP to
EUC-JP conversion module instead of the conversion coming with the
library itself is straightforward. Since the latter conversion takes two
steps (from ISO-2022-JP to ISO 10646 and then from ISO 10646 to
EUC-JP), the cost is 1+1 = 2. The above gconv-modules
file, however, specifies that the new conversion modules can perform this
conversion with only the cost of 1.
A mysterious item about the gconv-modules file above (and also
the file coming with the GNU C Library) are the names of the character
sets specified in the module
lines. Why do almost all the names
end in //
? And this is not all: the names can actually be
regular expressions. At this point in time this mystery should not be
revealed, unless you have the relevant spell-casting materials: ashes
from an original DOS 6.2 boot disk burnt in effigy, a crucifix
blessed by St. Emacs, assorted herbal roots from Central America, sand
from Cebu, etc. Sorry! The part of the implementation where
this is used is not yet finished. For now please simply follow the
existing examples. It’ll become clearer once it is. –drepper
A last remark about the gconv-modules is about the names not
ending with //
. A character set named INTERNAL
is often
mentioned. From the discussion above and the chosen name it should have
become clear that this is the name for the representation used in the
intermediate step of the triangulation. We have said that this is UCS-4
but actually that is not quite right. The UCS-4 specification also
includes the specification of the byte ordering used. Since a UCS-4 value
consists of four bytes, a stored value is affected by byte ordering. The
internal representation is not the same as UCS-4 in case the byte
ordering of the processor (or at least the running process) is not the
same as the one required for UCS-4. This is done for performance reasons
as one does not want to perform unnecessary byte-swapping operations if
one is not interested in actually seeing the result in UCS-4. To avoid
trouble with endianness, the internal representation consistently is named
INTERNAL
even on big-endian systems where the representations are
identical.
iconv
module data structures ¶So far this section has described how modules are located and considered to be used. What remains to be described is the interface of the modules so that one can write new ones. This section describes the interface as it is in use in January 1999. The interface will change a bit in the future but, with luck, only in an upwardly compatible way.
The definitions necessary to write new modules are publicly available in the non-standard header gconv.h. The following text, therefore, describes the definitions from this header file. First, however, it is necessary to get an overview.
From the perspective of the user of iconv
the interface is quite
simple: the iconv_open
function returns a handle that can be used
in calls to iconv
, and finally the handle is freed with a call to
iconv_close
. The problem is that the handle has to be able to
represent the possibly long sequences of conversion steps and also the
state of each conversion since the handle is all that is passed to the
iconv
function. Therefore, the data structures are really the
elements necessary to understanding the implementation.
We need two different kinds of data structures. The first describes the conversion and the second describes the state etc. There are really two type definitions like this in gconv.h.
This data structure describes one conversion a module can perform. For each function in a loaded module with conversion functions there is exactly one object of this type. This object is shared by all users of the conversion (i.e., this object does not contain any information corresponding to an actual conversion; it only describes the conversion itself).
struct __gconv_loaded_object *__shlib_handle
const char *__modname
int __counter
All these elements of the structure are used internally in the C library to coordinate loading and unloading the shared object. One must not expect any of the other elements to be available or initialized.
const char *__from_name
const char *__to_name
__from_name
and __to_name
contain the names of the source and
destination character sets. They can be used to identify the actual
conversion to be carried out since one module might implement conversions
for more than one character set and/or direction.
gconv_fct __fct
gconv_init_fct __init_fct
gconv_end_fct __end_fct
These elements contain pointers to the functions in the loadable module. The interface will be explained below.
int __min_needed_from
int __max_needed_from
int __min_needed_to
int __max_needed_to;
These values have to be supplied in the init function of the module. The
__min_needed_from
value specifies how many bytes a character of
the source character set at least needs. The __max_needed_from
specifies the maximum value that also includes possible shift sequences.
The __min_needed_to
and __max_needed_to
values serve the
same purpose as __min_needed_from
and __max_needed_from
but
this time for the destination character set.
It is crucial that these values be accurate since otherwise the conversion functions will have problems or not work at all.
int __stateful
This element must also be initialized by the init function.
int __stateful
is nonzero if the source character set is stateful.
Otherwise it is zero.
void *__data
This element can be used freely by the conversion functions in the
module. void *__data
can be used to communicate extra information
from one call to another. void *__data
need not be initialized if
not needed at all. If void *__data
element is assigned a pointer
to dynamically allocated memory (presumably in the init function) it has
to be made sure that the end function deallocates the memory. Otherwise
the application will leak memory.
It is important to be aware that this data structure is shared by all
users of this specification conversion and therefore the __data
element must not contain data specific to one specific use of the
conversion function.
This is the data structure that contains the information specific to each use of the conversion functions.
char *__outbuf
char *__outbufend
These elements specify the output buffer for the conversion step. The
__outbuf
element points to the beginning of the buffer, and
__outbufend
points to the byte following the last byte in the
buffer. The conversion function must not assume anything about the size
of the buffer but it can be safely assumed there is room for at
least one complete character in the output buffer.
Once the conversion is finished, if the conversion is the last step, the
__outbuf
element must be modified to point after the last byte
written into the buffer to signal how much output is available. If this
conversion step is not the last one, the element must not be modified.
The __outbufend
element must not be modified.
int __flags
This field is a set of flags. The __GCONV_IS_LAST
bit is set if
this conversion step is the last one. This information is necessary for
the recursion. See the description of the conversion function internals
below. This element must never be modified.
int __invocation_counter
The conversion function can use this element to see how many calls of the conversion function already happened. Some character sets require a certain prolog when generating output, and by comparing this value with zero, one can find out whether it is the first call and whether, therefore, the prolog should be emitted. This element must never be modified.
int __internal_use
This element is another one rarely used but needed in certain
situations. It is assigned a nonzero value in case the conversion
functions are used to implement mbsrtowcs
et.al. (i.e., the
function is not used directly through the iconv
interface).
This sometimes makes a difference as it is expected that the
iconv
functions are used to translate entire texts while the
mbsrtowcs
functions are normally used only to convert single
strings and might be used multiple times to convert entire texts.
But in this situation we would have problem complying with some rules of
the character set specification. Some character sets require a prolog,
which must appear exactly once for an entire text. If a number of
mbsrtowcs
calls are used to convert the text, only the first call
must add the prolog. However, because there is no communication between the
different calls of mbsrtowcs
, the conversion functions have no
possibility to find this out. The situation is different for sequences
of iconv
calls since the handle allows access to the needed
information.
The int __internal_use
element is mostly used together with
__invocation_counter
as follows:
if (!data->__internal_use
&& data->__invocation_counter == 0)
/* Emit prolog. */
...
This element must never be modified.
mbstate_t *__statep
The __statep
element points to an object of type mbstate_t
(see Representing the state of the conversion). The conversion of a stateful character
set must use the object pointed to by __statep
to store
information about the conversion state. The __statep
element
itself must never be modified.
mbstate_t __state
This element must never be used directly. It is only part of this structure to have the needed space allocated.
iconv
module interfaces ¶With the knowledge about the data structures we now can describe the conversion function itself. To understand the interface a bit of knowledge is necessary about the functionality in the C library that loads the objects with the conversions.
It is often the case that one conversion is used more than once (i.e.,
there are several iconv_open
calls for the same set of character
sets during one program run). The mbsrtowcs
et.al. functions in
the GNU C Library also use the iconv
functionality, which
increases the number of uses of the same functions even more.
Because of this multiple use of conversions, the modules do not get
loaded exclusively for one conversion. Instead a module once loaded can
be used by an arbitrary number of iconv
or mbsrtowcs
calls
at the same time. The splitting of the information between conversion-
function-specific information and conversion data makes this possible.
The last section showed the two data structures used to do this.
This is of course also reflected in the interface and semantics of the functions that the modules must provide. There are three functions that must have the following names:
gconv_init
The gconv_init
function initializes the conversion function
specific data structure. This very same object is shared by all
conversions that use this conversion and, therefore, no state information
about the conversion itself must be stored in here. If a module
implements more than one conversion, the gconv_init
function will
be called multiple times.
gconv_end
The gconv_end
function is responsible for freeing all resources
allocated by the gconv_init
function. If there is nothing to do,
this function can be missing. Special care must be taken if the module
implements more than one conversion and the gconv_init
function
does not allocate the same resources for all conversions.
gconv
This is the actual conversion function. It is called to convert one
block of text. It gets passed the conversion step information
initialized by gconv_init
and the conversion data, specific to
this use of the conversion functions.
There are three data types defined for the three module interface functions and these define the interface.
int
(*__gconv_init_fct) (struct __gconv_step *)
¶This specifies the interface of the initialization function of the module. It is called exactly once for each conversion the module implements.
As explained in the description of the struct __gconv_step
data
structure above the initialization function has to initialize parts of
it.
__min_needed_from
__max_needed_from
__min_needed_to
__max_needed_to
These elements must be initialized to the exact numbers of the minimum and maximum number of bytes used by one character in the source and destination character sets, respectively. If the characters all have the same size, the minimum and maximum values are the same.
__stateful
This element must be initialized to a nonzero value if the source character set is stateful. Otherwise it must be zero.
If the initialization function needs to communicate some information
to the conversion function, this communication can happen using the
__data
element of the __gconv_step
structure. But since
this data is shared by all the conversions, it must not be modified by
the conversion function. The example below shows how this can be used.
#define MIN_NEEDED_FROM 1 #define MAX_NEEDED_FROM 4 #define MIN_NEEDED_TO 4 #define MAX_NEEDED_TO 4 int gconv_init (struct __gconv_step *step) { /* Determine which direction. */ struct iso2022jp_data *new_data; enum direction dir = illegal_dir; enum variant var = illegal_var; int result; if (__strcasecmp (step->__from_name, "ISO-2022-JP//") == 0) { dir = from_iso2022jp; var = iso2022jp; } else if (__strcasecmp (step->__to_name, "ISO-2022-JP//") == 0) { dir = to_iso2022jp; var = iso2022jp; } else if (__strcasecmp (step->__from_name, "ISO-2022-JP-2//") == 0) { dir = from_iso2022jp; var = iso2022jp2; } else if (__strcasecmp (step->__to_name, "ISO-2022-JP-2//") == 0) { dir = to_iso2022jp; var = iso2022jp2; } result = __GCONV_NOCONV; if (dir != illegal_dir) { new_data = (struct iso2022jp_data *) malloc (sizeof (struct iso2022jp_data)); result = __GCONV_NOMEM; if (new_data != NULL) { new_data->dir = dir; new_data->var = var; step->__data = new_data; if (dir == from_iso2022jp) { step->__min_needed_from = MIN_NEEDED_FROM; step->__max_needed_from = MAX_NEEDED_FROM; step->__min_needed_to = MIN_NEEDED_TO; step->__max_needed_to = MAX_NEEDED_TO; } else { step->__min_needed_from = MIN_NEEDED_TO; step->__max_needed_from = MAX_NEEDED_TO; step->__min_needed_to = MIN_NEEDED_FROM; step->__max_needed_to = MAX_NEEDED_FROM + 2; } /* Yes, this is a stateful encoding. */ step->__stateful = 1; result = __GCONV_OK; } } return result; }
The function first checks which conversion is wanted. The module from which this function is taken implements four different conversions; which one is selected can be determined by comparing the names. The comparison should always be done without paying attention to the case.
Next, a data structure, which contains the necessary information about
which conversion is selected, is allocated. The data structure
struct iso2022jp_data
is locally defined since, outside the
module, this data is not used at all. Please note that if all four
conversions this module supports are requested there are four data
blocks.
One interesting thing is the initialization of the __min_
and
__max_
elements of the step data object. A single ISO-2022-JP
character can consist of one to four bytes. Therefore the
MIN_NEEDED_FROM
and MAX_NEEDED_FROM
macros are defined
this way. The output is always the INTERNAL
character set (aka
UCS-4) and therefore each character consists of exactly four bytes. For
the conversion from INTERNAL
to ISO-2022-JP we have to take into
account that escape sequences might be necessary to switch the character
sets. Therefore the __max_needed_to
element for this direction
gets assigned MAX_NEEDED_FROM + 2
. This takes into account the
two bytes needed for the escape sequences to signal the switching. The
asymmetry in the maximum values for the two directions can be explained
easily: when reading ISO-2022-JP text, escape sequences can be handled
alone (i.e., it is not necessary to process a real character since the
effect of the escape sequence can be recorded in the state information).
The situation is different for the other direction. Since it is in
general not known which character comes next, one cannot emit escape
sequences to change the state in advance. This means the escape
sequences have to be emitted together with the next character.
Therefore one needs more room than only for the character itself.
The possible return values of the initialization function are:
__GCONV_OK
The initialization succeeded
__GCONV_NOCONV
The requested conversion is not supported in the module. This can happen if the gconv-modules file has errors.
__GCONV_NOMEM
Memory required to store additional information could not be allocated.
The function called before the module is unloaded is significantly easier. It often has nothing at all to do; in which case it can be left out completely.
void
(*__gconv_end_fct) (struct gconv_step *)
¶The task of this function is to free all resources allocated in the
initialization function. Therefore only the __data
element of
the object pointed to by the argument is of interest. Continuing the
example from the initialization function, the finalization function
looks like this:
void gconv_end (struct __gconv_step *data) { free (data->__data); }
The most important function is the conversion function itself, which can get quite complicated for complex character sets. But since this is not of interest here, we will only describe a possible skeleton for the conversion function.
int
(*__gconv_fct) (struct __gconv_step *, struct __gconv_step_data *, const char **, const char *, size_t *, int)
¶The conversion function can be called for two basic reasons: to convert
text or to reset the state. From the description of the iconv
function it can be seen why the flushing mode is necessary. What mode
is selected is determined by the sixth argument, an integer. This
argument being nonzero means that flushing is selected.
Common to both modes is where the output buffer can be found. The
information about this buffer is stored in the conversion step data. A
pointer to this information is passed as the second argument to this
function. The description of the struct __gconv_step_data
structure has more information on the conversion step data.
What has to be done for flushing depends on the source character set.
If the source character set is not stateful, nothing has to be done.
Otherwise the function has to emit a byte sequence to bring the state
object into the initial state. Once this all happened the other
conversion modules in the chain of conversions have to get the same
chance. Whether another step follows can be determined from the
__GCONV_IS_LAST
flag in the __flags
field of the step
data structure to which the first parameter points.
The more interesting mode is when actual text has to be converted. The first step in this case is to convert as much text as possible from the input buffer and store the result in the output buffer. The start of the input buffer is determined by the third argument, which is a pointer to a pointer variable referencing the beginning of the buffer. The fourth argument is a pointer to the byte right after the last byte in the buffer.
The conversion has to be performed according to the current state if the
character set is stateful. The state is stored in an object pointed to
by the __statep
element of the step data (second argument). Once
either the input buffer is empty or the output buffer is full the
conversion stops. At this point, the pointer variable referenced by the
third parameter must point to the byte following the last processed
byte (i.e., if all of the input is consumed, this pointer and the fourth
parameter have the same value).
What now happens depends on whether this step is the last one. If it is
the last step, the only thing that has to be done is to update the
__outbuf
element of the step data structure to point after the
last written byte. This update gives the caller the information on how
much text is available in the output buffer. In addition, the variable
pointed to by the fifth parameter, which is of type size_t
, must
be incremented by the number of characters (not bytes) that were
converted in a non-reversible way. Then, the function can return.
In case the step is not the last one, the later conversion functions have to get a chance to do their work. Therefore, the appropriate conversion function has to be called. The information about the functions is stored in the conversion data structures, passed as the first parameter. This information and the step data are stored in arrays, so the next element in both cases can be found by simple pointer arithmetic:
int gconv (struct __gconv_step *step, struct __gconv_step_data *data, const char **inbuf, const char *inbufend, size_t *written, int do_flush) { struct __gconv_step *next_step = step + 1; struct __gconv_step_data *next_data = data + 1; ...
The next_step
pointer references the next step information and
next_data
the next data record. The call of the next function
therefore will look similar to this:
next_step->__fct (next_step, next_data, &outerr, outbuf, written, 0)
But this is not yet all. Once the function call returns the conversion
function might have some more to do. If the return value of the function
is __GCONV_EMPTY_INPUT
, more room is available in the output
buffer. Unless the input buffer is empty, the conversion functions start
all over again and process the rest of the input buffer. If the return
value is not __GCONV_EMPTY_INPUT
, something went wrong and we have
to recover from this.
A requirement for the conversion function is that the input buffer pointer (the third argument) always point to the last character that was put in converted form into the output buffer. This is trivially true after the conversion performed in the current step, but if the conversion functions deeper downstream stop prematurely, not all characters from the output buffer are consumed and, therefore, the input buffer pointers must be backed off to the right position.
Correcting the input buffers is easy to do if the input and output character sets have a fixed width for all characters. In this situation we can compute how many characters are left in the output buffer and, therefore, can correct the input buffer pointer appropriately with a similar computation. Things are getting tricky if either character set has characters represented with variable length byte sequences, and it gets even more complicated if the conversion has to take care of the state. In these cases the conversion has to be performed once again, from the known state before the initial conversion (i.e., if necessary the state of the conversion has to be reset and the conversion loop has to be executed again). The difference now is that it is known how much input must be created, and the conversion can stop before converting the first unused character. Once this is done the input buffer pointers must be updated again and the function can return.
One final thing should be mentioned. If it is necessary for the
conversion to know whether it is the first invocation (in case a prolog
has to be emitted), the conversion function should increment the
__invocation_counter
element of the step data structure just
before returning to the caller. See the description of the struct
__gconv_step_data
structure above for more information on how this can
be used.
The return value must be one of the following values:
__GCONV_EMPTY_INPUT
All input was consumed and there is room left in the output buffer.
__GCONV_FULL_OUTPUT
No more room in the output buffer. In case this is not the last step this value is propagated down from the call of the next conversion function in the chain.
__GCONV_INCOMPLETE_INPUT
The input buffer is not entirely empty since it contains an incomplete character sequence.
The following example provides a framework for a conversion function. In case a new conversion has to be written the holes in this implementation have to be filled and that is it.
int gconv (struct __gconv_step *step, struct __gconv_step_data *data, const char **inbuf, const char *inbufend, size_t *written, int do_flush) { struct __gconv_step *next_step = step + 1; struct __gconv_step_data *next_data = data + 1; gconv_fct fct = next_step->__fct; int status; /* If the function is called with no input this means we have to reset to the initial state. The possibly partly converted input is dropped. */ if (do_flush) { status = __GCONV_OK; /* Possible emit a byte sequence which put the state object into the initial state. */ /* Call the steps down the chain if there are any but only if we successfully emitted the escape sequence. */ if (status == __GCONV_OK && ! (data->__flags & __GCONV_IS_LAST)) status = fct (next_step, next_data, NULL, NULL, written, 1); } else { /* We preserve the initial values of the pointer variables. */ const char *inptr = *inbuf; char *outbuf = data->__outbuf; char *outend = data->__outbufend; char *outptr; do { /* Remember the start value for this round. */ inptr = *inbuf; /* The outbuf buffer is empty. */ outptr = outbuf; /* For stateful encodings the state must be safe here. */ /* Run the conversion loop.status
is set appropriately afterwards. */ /* If this is the last step, leave the loop. There is nothing we can do. */ if (data->__flags & __GCONV_IS_LAST) { /* Store information about how many bytes are available. */ data->__outbuf = outbuf; /* If any non-reversible conversions were performed, add the number to*written
. */ break; } /* Write out all output that was produced. */ if (outbuf > outptr) { const char *outerr = data->__outbuf; int result; result = fct (next_step, next_data, &outerr, outbuf, written, 0); if (result != __GCONV_EMPTY_INPUT) { if (outerr != outbuf) { /* Reset the input buffer pointer. We document here the complex case. */ size_t nstatus; /* Reload the pointers. */ *inbuf = inptr; outbuf = outptr; /* Possibly reset the state. */ /* Redo the conversion, but this time the end of the output buffer is atouterr
. */ } /* Change the status. */ status = result; } else /* All the output is consumed, we can make another run if everything was ok. */ if (status == __GCONV_FULL_OUTPUT) status = __GCONV_OK; } } while (status == __GCONV_OK); /* We finished one use of this step. */ ++data->__invocation_counter; } return status; }
This information should be sufficient to write new modules. Anybody doing so should also take a look at the available source code in the GNU C Library sources. It contains many examples of working and optimized modules.
Different countries and cultures have varying conventions for how to communicate. These conventions range from very simple ones, such as the format for representing dates and times, to very complex ones, such as the language spoken.
Internationalization of software means programming it to be able to adapt to the user’s favorite conventions. In ISO C, internationalization works by means of locales. Each locale specifies a collection of conventions, one convention for each purpose. The user chooses a set of conventions by specifying a locale (via environment variables).
All programs inherit the chosen locale as part of their environment. Provided the programs are written to obey the choice of locale, they will follow the conventions preferred by the user.
Each locale specifies conventions for several purposes, including the following:
Some aspects of adapting to the specified locale are handled
automatically by the library subroutines. For example, all your program
needs to do in order to use the collating sequence of the chosen locale
is to use strcoll
or strxfrm
to compare strings.
Other aspects of locales are beyond the comprehension of the library. For example, the library can’t automatically translate your program’s output messages into other languages. The only way you can support output in the user’s favorite language is to program this more or less by hand. The C library provides functions to handle translations for multiple languages easily.
This chapter discusses the mechanism by which you can modify the current locale. The effects of the current locale on specific library functions are discussed in more detail in the descriptions of those functions.
The simplest way for the user to choose a locale is to set the
environment variable LANG
. This specifies a single locale to use
for all purposes. For example, a user could specify a hypothetical
locale named ‘espana-castellano’ to use the standard conventions of
most of Spain.
The set of locales supported depends on the operating system you are using, and so do their names, except that the standard locale called ‘C’ or ‘POSIX’ always exist. See Locale Names.
In order to force the system to always use the default locale, the
user can set the LC_ALL
environment variable to ‘C’.
A user also has the option of specifying different locales for different purposes—in effect, choosing a mixture of multiple locales. See Locale Categories.
For example, the user might specify the locale ‘espana-castellano’ for most purposes, but specify the locale ‘usa-english’ for currency formatting. This might make sense if the user is a Spanish-speaking American, working in Spanish, but representing monetary amounts in US dollars.
Note that both locales ‘espana-castellano’ and ‘usa-english’, like all locales, would include conventions for all of the purposes to which locales apply. However, the user can choose to use each locale for a particular subset of those purposes.
The purposes that locales serve are grouped into categories, so
that a user or a program can choose the locale for each category
independently. Here is a table of categories; each name is both an
environment variable that a user can set, and a macro name that you can
use as the first argument to setlocale
.
The contents of the environment variable (or the string in the second
argument to setlocale
) has to be a valid locale name.
See Locale Names.
LC_COLLATE
¶This category applies to collation of strings (functions strcoll
and strxfrm
); see Collation Functions.
LC_CTYPE
¶This category applies to classification and conversion of characters, and to multibyte and wide characters; see Character Handling, and Character Set Handling.
LC_MONETARY
¶This category applies to formatting monetary values; see Generic Numeric Formatting Parameters.
LC_NUMERIC
¶This category applies to formatting numeric values that are not monetary; see Generic Numeric Formatting Parameters.
LC_TIME
¶This category applies to formatting date and time values; see Formatting Calendar Time.
LC_MESSAGES
¶This category applies to selecting the language used in the user interface for message translation (see The Uniforum approach to Message Translation; see X/Open Message Catalog Handling) and contains regular expressions for affirmative and negative responses.
LC_ALL
¶This is not a category; it is only a macro that you can use
with setlocale
to set a single locale for all purposes. Setting
this environment variable overwrites all selections by the other
LC_*
variables or LANG
.
LANG
¶If this environment variable is defined, its value specifies the locale to use for all purposes except as overridden by the variables above.
When developing the message translation functions it was felt that the
functionality provided by the variables above is not sufficient. For
example, it should be possible to specify more than one locale name.
Take a Swedish user who better speaks German than English, and a program
whose messages are output in English by default. It should be possible
to specify that the first choice of language is Swedish, the second
German, and if this also fails to use English. This is
possible with the variable LANGUAGE
. For further description of
this GNU extension see User influence on gettext
.
A C program inherits its locale environment variables when it starts up.
This happens automatically. However, these variables do not
automatically control the locale used by the library functions, because
ISO C says that all programs start by default in the standard ‘C’
locale. To use the locales specified by the environment, you must call
setlocale
. Call it as follows:
setlocale (LC_ALL, "");
to select a locale based on the user choice of the appropriate environment variables.
You can also use setlocale
to specify a particular locale, for
general use or for a specific category.
The symbols in this section are defined in the header file locale.h.
char *
setlocale (int category, const char *locale)
¶Preliminary: | MT-Unsafe const:locale env | AS-Unsafe init lock heap corrupt | AC-Unsafe init corrupt lock mem fd | See POSIX Safety Concepts.
The function setlocale
sets the current locale for category
category to locale.
If category is LC_ALL
, this specifies the locale for all
purposes. The other possible values of category specify a
single purpose (see Locale Categories).
You can also use this function to find out the current locale by passing
a null pointer as the locale argument. In this case,
setlocale
returns a string that is the name of the locale
currently selected for category category.
The string returned by setlocale
can be overwritten by subsequent
calls, so you should make a copy of the string (see Copying Strings and Arrays) if you want to save it past any further calls to
setlocale
. (The standard library is guaranteed never to call
setlocale
itself.)
You should not modify the string returned by setlocale
. It might
be the same string that was passed as an argument in a previous call to
setlocale
. One requirement is that the category must be
the same in the call the string was returned and the one when the string
is passed in as locale parameter.
When you read the current locale for category LC_ALL
, the value
encodes the entire combination of selected locales for all categories.
If you specify the same “locale name” with LC_ALL
in a
subsequent call to setlocale
, it restores the same combination
of locale selections.
To be sure you can use the returned string encoding the currently selected locale at a later time, you must make a copy of the string. It is not guaranteed that the returned pointer remains valid over time.
When the locale argument is not a null pointer, the string returned
by setlocale
reflects the newly-modified locale.
If you specify an empty string for locale, this means to read the appropriate environment variable and use its value to select the locale for category.
If a nonempty string is given for locale, then the locale of that name is used if possible.
The effective locale name (either the second argument to
setlocale
, or if the argument is an empty string, the name
obtained from the process environment) must be a valid locale name.
See Locale Names.
If you specify an invalid locale name, setlocale
returns a null
pointer and leaves the current locale unchanged.
Here is an example showing how you might use setlocale
to
temporarily switch to a new locale.
#include <stddef.h>
#include <locale.h>
#include <stdlib.h>
#include <string.h>
void
with_other_locale (char *new_locale,
void (*subroutine) (int),
int argument)
{
char *old_locale, *saved_locale;
/* Get the name of the current locale. */
old_locale = setlocale (LC_ALL, NULL);
/* Copy the name so it won’t be clobbered by setlocale
. */
saved_locale = strdup (old_locale);
if (saved_locale == NULL)
fatal ("Out of memory");
/* Now change the locale and do some stuff with it. */
setlocale (LC_ALL, new_locale);
(*subroutine) (argument);
/* Restore the original locale. */
setlocale (LC_ALL, saved_locale);
free (saved_locale);
}
Portability Note: Some ISO C systems may define additional locale categories, and future versions of the library will do so. For portability, assume that any symbol beginning with ‘LC_’ might be defined in locale.h.
The only locale names you can count on finding on all operating systems are these three standard ones:
"C"
This is the standard C locale. The attributes and behavior it provides are specified in the ISO C standard. When your program starts up, it initially uses this locale by default.
"POSIX"
This is the standard POSIX locale. Currently, it is an alias for the standard C locale.
""
The empty name says to select a locale based on environment variables. See Locale Categories.
Defining and installing named locales is normally a responsibility of the system administrator at your site (or the person who installed the GNU C Library). It is also possible for the user to create private locales. All this will be discussed later when describing the tool to do so.
If your program needs to use something other than the ‘C’ locale, it will be more portable if you use whatever locale the user specifies with the environment, rather than trying to specify some non-standard locale explicitly by name. Remember, different machines might have different sets of locales installed.
The following command prints a list of locales supported by the system:
locale -a
Portability Note: With the notable exception of the standard locale names ‘C’ and ‘POSIX’, locale names are system-specific.
Most locale names follow XPG syntax and consist of up to four parts:
language[_territory[.codeset]][@modifier]
Beside the first part, all of them are allowed to be missing. If the full specified locale is not found, less specific ones are looked for. The various parts will be stripped off, in the following order:
For example, the locale name ‘de_AT.iso885915@euro’ denotes a German-language locale for use in Austria, using the ISO-8859-15 (Latin-9) character set, and with the Euro as the currency symbol.
In addition to locale names which follow XPG syntax, systems may provide aliases such as ‘german’. Both categories of names must not contain the slash character ‘/’.
If the locale name starts with a slash ‘/’, it is treated as a
path relative to the configured locale directories; see LOCPATH
below. The specified path must not contain a component ‘..’, or
the name is invalid, and setlocale
will fail.
Portability Note: POSIX suggests that if a locale name starts
with a slash ‘/’, it is resolved as an absolute path. However,
the GNU C Library treats it as a relative path under the directories listed
in LOCPATH
(or the default locale directory if LOCPATH
is unset).
Locale names which are longer than an implementation-defined limit are
invalid and cause setlocale
to fail.
As a special case, locale names used with LC_ALL
can combine
several locales, reflecting different locale settings for different
categories. For example, you might want to use a U.S. locale with ISO
A4 paper format, so you set LANG
to ‘en_US.UTF-8’, and
LC_PAPER
to ‘de_DE.UTF-8’. In this case, the
LC_ALL
-style combined locale name is
LC_CTYPE=en_US.UTF-8;LC_TIME=en_US.UTF-8;LC_PAPER=de_DE.UTF-8;...
followed by other category settings not shown here.
The path used for finding locale data can be set using the
LOCPATH
environment variable. This variable lists the
directories in which to search for locale definitions, separated by a
colon ‘:’.
The default path for finding locale data is system specific. A typical
value for the LOCPATH
default is:
/usr/share/locale
The value of LOCPATH
is ignored by privileged programs for
security reasons, and only the default directory is used.
There are several ways to access locale information. The simplest way is to let the C library itself do the work. Several of the functions in this library implicitly access the locale data, and use what information is provided by the currently selected locale. This is how the locale model is meant to work normally.
As an example take the strftime
function, which is meant to nicely
format date and time information (see Formatting Calendar Time).
Part of the standard information contained in the LC_TIME
category is the names of the months. Instead of requiring the
programmer to take care of providing the translations the
strftime
function does this all by itself. %A
in the format string is replaced by the appropriate weekday
name of the locale currently selected by LC_TIME
. This is an
easy example, and wherever possible functions do things automatically
in this way.
But there are quite often situations when there is simply no function
to perform the task, or it is simply not possible to do the work
automatically. For these cases it is necessary to access the
information in the locale directly. To do this the C library provides
two functions: localeconv
and nl_langinfo
. The former is
part of ISO C and therefore portable, but has a brain-damaged
interface. The second is part of the Unix interface and is portable in
as far as the system follows the Unix standards.
localeconv
: It is portable but … ¶Together with the setlocale
function the ISO C people
invented the localeconv
function. It is a masterpiece of poor
design. It is expensive to use, not extensible, and not generally
usable as it provides access to only LC_MONETARY
and
LC_NUMERIC
related information. Nevertheless, if it is
applicable to a given situation it should be used since it is very
portable. The function strfmon
formats monetary amounts
according to the selected locale using this information.
struct lconv *
localeconv (void)
¶Preliminary: | MT-Unsafe race:localeconv locale | AS-Unsafe | AC-Safe | See POSIX Safety Concepts.
The localeconv
function returns a pointer to a structure whose
components contain information about how numeric and monetary values
should be formatted in the current locale.
You should not modify the structure or its contents. The structure might
be overwritten by subsequent calls to localeconv
, or by calls to
setlocale
, but no other function in the library overwrites this
value.
localeconv
’s return value is of this data type. Its elements are
described in the following subsections.
If a member of the structure struct lconv
has type char
,
and the value is CHAR_MAX
, it means that the current locale has
no value for that parameter.
These are the standard members of struct lconv
; there may be
others.
char *decimal_point
char *mon_decimal_point
These are the decimal-point separators used in formatting non-monetary
and monetary quantities, respectively. In the ‘C’ locale, the
value of decimal_point
is "."
, and the value of
mon_decimal_point
is ""
.
char *thousands_sep
char *mon_thousands_sep
These are the separators used to delimit groups of digits to the left of
the decimal point in formatting non-monetary and monetary quantities,
respectively. In the ‘C’ locale, both members have a value of
""
(the empty string).
char *grouping
char *mon_grouping
These are strings that specify how to group the digits to the left of
the decimal point. grouping
applies to non-monetary quantities
and mon_grouping
applies to monetary quantities. Use either
thousands_sep
or mon_thousands_sep
to separate the digit
groups.
Each member of these strings is to be interpreted as an integer value of
type char
. Successive numbers (from left to right) give the
sizes of successive groups (from right to left, starting at the decimal
point.) The last member is either 0
, in which case the previous
member is used over and over again for all the remaining groups, or
CHAR_MAX
, in which case there is no more grouping—or, put
another way, any remaining digits form one large group without
separators.
For example, if grouping
is "\04\03\02"
, the correct
grouping for the number 123456787654321
is ‘12’, ‘34’,
‘56’, ‘78’, ‘765’, ‘4321’. This uses a group of 4
digits at the end, preceded by a group of 3 digits, preceded by groups
of 2 digits (as many as needed). With a separator of ‘,’, the
number would be printed as ‘12,34,56,78,765,4321’.
A value of "\03"
indicates repeated groups of three digits, as
normally used in the U.S.
In the standard ‘C’ locale, both grouping
and
mon_grouping
have a value of ""
. This value specifies no
grouping at all.
char int_frac_digits
char frac_digits
These are small integers indicating how many fractional digits (to the right of the decimal point) should be displayed in a monetary value in international and local formats, respectively. (Most often, both members have the same value.)
In the standard ‘C’ locale, both of these members have the value
CHAR_MAX
, meaning “unspecified”. The ISO standard doesn’t say
what to do when you find this value; we recommend printing no
fractional digits. (This locale also specifies the empty string for
mon_decimal_point
, so printing any fractional digits would be
confusing!)
These members of the struct lconv
structure specify how to print
the symbol to identify a monetary value—the international analog of
‘$’ for US dollars.
Each country has two standard currency symbols. The local currency symbol is used commonly within the country, while the international currency symbol is used internationally to refer to that country’s currency when it is necessary to indicate the country unambiguously.
For example, many countries use the dollar as their monetary unit, and when dealing with international currencies it’s important to specify that one is dealing with (say) Canadian dollars instead of U.S. dollars or Australian dollars. But when the context is known to be Canada, there is no need to make this explicit—dollar amounts are implicitly assumed to be in Canadian dollars.
char *currency_symbol
The local currency symbol for the selected locale.
In the standard ‘C’ locale, this member has a value of ""
(the empty string), meaning “unspecified”. The ISO standard doesn’t
say what to do when you find this value; we recommend you simply print
the empty string as you would print any other string pointed to by this
variable.
char *int_curr_symbol
The international currency symbol for the selected locale.
The value of int_curr_symbol
should normally consist of a
three-letter abbreviation determined by the international standard
ISO 4217 Codes for the Representation of Currency and Funds,
followed by a one-character separator (often a space).
In the standard ‘C’ locale, this member has a value of ""
(the empty string), meaning “unspecified”. We recommend you simply print
the empty string as you would print any other string pointed to by this
variable.
char p_cs_precedes
char n_cs_precedes
char int_p_cs_precedes
char int_n_cs_precedes
These members are 1
if the currency_symbol
or
int_curr_symbol
strings should precede the value of a monetary
amount, or 0
if the strings should follow the value. The
p_cs_precedes
and int_p_cs_precedes
members apply to
positive amounts (or zero), and the n_cs_precedes
and
int_n_cs_precedes
members apply to negative amounts.
In the standard ‘C’ locale, all of these members have a value of
CHAR_MAX
, meaning “unspecified”. The ISO standard doesn’t say
what to do when you find this value. We recommend printing the
currency symbol before the amount, which is right for most countries.
In other words, treat all nonzero values alike in these members.
The members with the int_
prefix apply to the
int_curr_symbol
while the other two apply to
currency_symbol
.
char p_sep_by_space
char n_sep_by_space
char int_p_sep_by_space
char int_n_sep_by_space
These members are 1
if a space should appear between the
currency_symbol
or int_curr_symbol
strings and the
amount, or 0
if no space should appear. The
p_sep_by_space
and int_p_sep_by_space
members apply to
positive amounts (or zero), and the n_sep_by_space
and
int_n_sep_by_space
members apply to negative amounts.
In the standard ‘C’ locale, all of these members have a value of
CHAR_MAX
, meaning “unspecified”. The ISO standard doesn’t say
what you should do when you find this value; we suggest you treat it as
1 (print a space). In other words, treat all nonzero values alike in
these members.
The members with the int_
prefix apply to the
int_curr_symbol
while the other two apply to
currency_symbol
. There is one specialty with the
int_curr_symbol
, though. Since all legal values contain a space
at the end of the string one either prints this space (if the currency
symbol must appear in front and must be separated) or one has to avoid
printing this character at all (especially when at the end of the
string).
These members of the struct lconv
structure specify how to print
the sign (if any) of a monetary value.
char *positive_sign
char *negative_sign
These are strings used to indicate positive (or zero) and negative monetary quantities, respectively.
In the standard ‘C’ locale, both of these members have a value of
""
(the empty string), meaning “unspecified”.
The ISO standard doesn’t say what to do when you find this value; we
recommend printing positive_sign
as you find it, even if it is
empty. For a negative value, print negative_sign
as you find it
unless both it and positive_sign
are empty, in which case print
‘-’ instead. (Failing to indicate the sign at all seems rather
unreasonable.)
char p_sign_posn
char n_sign_posn
char int_p_sign_posn
char int_n_sign_posn
These members are small integers that indicate how to
position the sign for nonnegative and negative monetary quantities,
respectively. (The string used for the sign is what was specified with
positive_sign
or negative_sign
.) The possible values are
as follows:
0
The currency symbol and quantity should be surrounded by parentheses.
1
Print the sign string before the quantity and currency symbol.
2
Print the sign string after the quantity and currency symbol.
3
Print the sign string right before the currency symbol.
4
Print the sign string right after the currency symbol.
CHAR_MAX
“Unspecified”. Both members have this value in the standard ‘C’ locale.
The ISO standard doesn’t say what you should do when the value is
CHAR_MAX
. We recommend you print the sign after the currency
symbol.
The members with the int_
prefix apply to the
int_curr_symbol
while the other two apply to
currency_symbol
.
When writing the X/Open Portability Guide the authors realized that the
localeconv
function is not enough to provide reasonable access to
locale information. The information which was meant to be available
in the locale (as later specified in the POSIX.1 standard) requires more
ways to access it. Therefore the nl_langinfo
function
was introduced.
char *
nl_langinfo (nl_item item)
¶Preliminary: | MT-Safe locale | AS-Safe | AC-Safe | See POSIX Safety Concepts.
The nl_langinfo
function can be used to access individual
elements of the locale categories. Unlike the localeconv
function, which returns all the information, nl_langinfo
lets the caller select what information it requires. This is very
fast and it is not a problem to call this function multiple times.
A second advantage is that in addition to the numeric and monetary
formatting information, information from the
LC_TIME
and LC_MESSAGES
categories is available.
The type nl_item
is defined in nl_types.h. The argument
item is a numeric value defined in the header langinfo.h.
The X/Open standard defines the following values:
CODESET
¶nl_langinfo
returns a string with the name of the coded character
set used in the selected locale.
ABDAY_1
¶ABDAY_2
¶ABDAY_3
¶ABDAY_4
¶ABDAY_5
¶ABDAY_6
¶ABDAY_7
¶nl_langinfo
returns the abbreviated weekday name. ABDAY_1
corresponds to Sunday.
DAY_1
¶DAY_2
¶DAY_3
¶DAY_4
¶DAY_5
¶DAY_6
¶DAY_7
¶Similar to ABDAY_1
, etc., but here the return value is the
unabbreviated weekday name.
ABMON_1
¶ABMON_2
¶ABMON_3
¶ABMON_4
¶ABMON_5
¶ABMON_6
¶ABMON_7
¶ABMON_8
¶ABMON_9
¶ABMON_10
¶ABMON_11
¶ABMON_12
¶The return value is the abbreviated name of the month, in the
grammatical form used when the month forms part of a complete date.
ABMON_1
corresponds to January.
MON_1
¶MON_2
¶MON_3
<