librdesc copies token seminfo by value internally. The seminfo is any user-defined object; e.g. it may contain heap-allocated fields. The token_destroyer callback is responsible for performing any cleanup required for those objects.

The ownership of each seminfo copy depends on the result of rdesc_pump() or rdesc_resume():

Result	Ownership
RDESC_CONTINUE	Parser took a copy of `seminfo`. The caller's original is no longer holds the resource ownership and destructor of it, if present, should not be called.
RDESC_READY	Parser releases ownership of `seminfo` in the matched CST. `token_destroyer` is not called on them if parser is destroyed or reset. The caller takes ownership and is responsible for destroying them during traversal.
RDESC_NOMATCH	Parser retains ownership of all held tokens. `token_destroyer` is invoked on them upon `rdesc_reset()` or `rdesc_destroy()`. Caller owes nothing.
RDESC_ENOMEM	Allocation failed mid-parse. The unprocessed token's `seminfo` is saved internally – the caller must not destroy it.

In dirty Ready state, the parser still holds tokens in its backtrack stack. The caller can:

Call rdesc_start() to begin a new parse. The parser will not call token_destroyer on any remaining held tokens.
Call rdesc_destroy() to release all resources, including any remaining held tokens.

To find out when the parser enters the dirty Ready state, refer to its documentation.

Note: token_destroyer is set at rdesc_init() time and may be NULL if tokens require no cleanup.