Metadata-Version: 2.4
Name: ctypesgen
Version: 5.4.0
Summary: Python wrapper generator for ctypes
License: BSD-2-Clause
Project-URL: Homepage, https://github.com/ctypesgen/ctypesgen
Project-URL: Repository, https://github.com/ctypesgen/ctypesgen.git
Project-URL: Issues, https://github.com/ctypesgen/ctypesgen/issues
Project-URL: Changelog, https://github.com/ctypesgen/ctypesgen/blob/master/CHANGELOG.md
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Development Status :: 4 - Beta
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Code Generators
Classifier: Topic :: Software Development :: Pre-processors
Classifier: Topic :: Software Development :: Build Tools
Classifier: Environment :: Console
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license
Dynamic: license-file

_**Important:** This repository's actual main branch is `pypdfium2`, whereas `master` is the old upstream codebase. So if you want to use this fork, make sure you are on `pypdfium2`, or a branch derived thereof. This layout has historical reasons, and makes it easier to sync the fork._

# ctypesgen (pypdfium2-team fork)

ctypesgen is a ctypes wrapper generator for Python.

This is a fork with the objective to better suit the needs of pypdfium2, and address some of the technical debt and (in our opinion) design issues that have accumulated due to highly conservative maintenance.

Here are some notes on our development intents:
* We do not mind API-breaking changes at this time.
* We endeavor to use plain ctypes as much as possible and keep the template lean.
* For now, we only envisage to work with ctypesgen's higher-level parts. The parser backend may be out of our scope.


### System Dependencies

ctypesgen depends on the presence of an external C pre-processor, by default `gcc` or `clang`, as available.
Alternatively, you may specify a custom pre-processor command using the `--cpp` option (e.g. `--cpp "clang -E"` to always use clang).

See also [`docs/pcpp.md`](docs/pcpp.md) for an experimental pure-python option.


### Tips & Tricks

<!-- TODO update info on relative modules (mention multiple `.`s and --linkage-anchor) -->

* If you have multiple libraries that are supposed to interoperate with shared symbols, first create bindings to any shared headers and then use the `-m / --link-modules` option on dependants. (Otherwise, you'd create duplicate symbols that are formally different types, with need to cast between them.)
  If the module is not installed separately, you may prefix the module name with `.` for a relative import, and share boilerplate code using `--no-embed-templates`. Relative modules will be expected to be present in the output directory at compile time.
  Note, this strategy can also be used to bind to same-library headers separately; however, you'll need to resolve the dependency tree on your own.
* Extra include search paths can be provided using the `-I` option or by setting `$CPATH`/`$C_INCLUDE_PATH`.
  You could use this to add a header spoofing an external symbol via `typedef void* SYMBOL;` (`c_void_p`) that may be provided by a third-party binding at runtime.
* If building with `--no-macro-guards` and you encounter broken macros, you may use `--symbol-rules` (see below) or replace them manually. This can be necessary on C constructs like `#define NAN (0.0f / 0.0f)` that don't play well with python. In particular, you are likely to run into this with `--all-headers`.

Also, check out [`docs/demo/`](docs/demo/) for a minimalistic example of using ctypesgen,
and [`docs/python_api.md`](docs/python_api.md) on how to produce bindings for Python's C API.


#### Symbol inclusion

* ctypesgen works with the following symbol rules:
  - `yes`: The symbol is eagerly included.
  - `if_needed`: The symbol is included if other included symbols depend on it (e.g. a type used in a function signature).
  - `never`: The symbol is always excluded, and implicitly all its dependants.
* Roughly speaking, symbols from caller-given headers get assigned the include rule `yes`, and any others `if_needed`. When building with `--all-headers`, all symbols default to `yes` regardless of their origin.
* `--no-macros` sets the include rule of all macro objects to `never`.
* Finally, the `--symbol-rules` option is applied, which can be used to assign symbol rules by regex fullmatch expressions, providing callers with powerful means of control over symbol inclusion.
* To filter out excess symbols, you'll usually want to use `if_needed` rather than `never` to avoid accidental exclusion of dependants. Use `never` only where this side effect is actually wanted, e.g. to exclude a broken symbol.


### Known Limitations

*ctypes*
* Rare calling conventions other than `cdecl` or `stdcall` are not supported.
* ctypes does not support custom pointer result types in callbacks. Therefore, we use a template to remap these to `c_void_p`.

*pypdfium2-ctypesgen*
* The DLL class is assumed to be `CDLL`, otherwise it needs to be given by the caller. We do not currently support mixed calling conventions, because it does not match the API layer of ctypes.
* We do not support binding to multiple binaries in the same output file. Instead, you'll want to create separate output files sharing the loader template, and possibly use module linking, as described above.

*ctypesgen*
* ctypesgen's parser was originally written for C99. Support for later standards (C11 etc.) is probably incomplete.
* The conflicting names resolver is largely untested, in particular the handling of dependants. Please report success or failure.
* Linked modules are naively prioritized in dependency resolver and conflicting names handler, i.e. intentional overrides are ignored. The position of includes is not honored; ctypesgen always imports linked modules at top level.


### History and Friends

- ctypesgen has its roots in [`wraptypes`](https://github.com/pyglet/pyglet/tree/master/tools/wraptypes) from pyglet, which was originally written by Alex Holkner for C99, and is still around today.
  Some documentation can be found [here](https://docs.pyglet.org/en/development/internal/wraptypes.html). This may still be a valuable source of information even for today's ctypesgen.

- ctypesgen is also used by the GRASS project, which has its own copy of ctypesgen [here](https://github.com/OSGeo/grass/tree/main/python/libgrass_interface_generator).

- [`ctypeslib2`](https://github.com/trolldbois/ctypeslib) is an independent alternative to ctypesgen.
It has its own parser backend using Clang API. This may be more reliable in a way, at the cost of being impure and tied to a specific compiler.
  `ctypeslib2` has some fancy features ctypesgen currently does not, such as taking over Doxygen comments.

- The original `ctypeslib` (by former ctypes maintainer Thomas Heller) used GCC's XML output, but the author is not aware of a maintained version of this codebase. `ctypeslib2` is derived from ctypeslib, but its backend has been thoroughly changed for Clang.


### Issues / Patches

Oversights or unintentional breakage can happen at times. If you think a change introduces logical issues, feel free to file a bug report or submit a patch.

Note though, the response/contributions policy is [basically the same as for pypdfium2](https://github.com/pypdfium2-team/pypdfium2/?tab=readme-ov-file#response-policy).


### Code Style

The pypdfium2-ctypesgen codebase *intentionally* does not hard wrap long lines.
You'll want to configure your editor to soft wrap e.g. at 100 columns.

We also intentionally maintain indentation levels. This allows to paste code excerpts into the Python console (REPL), and makes editing more convenient.

Future maintainers, please keep it this way. Also, please do NOT run formatters like Black on this codebase ever again.
Take care of code style yourself while editing, this tends to yield much smarter results.


### Fork rationale

Trying to get through changes upstream is tedious, with unclear outcome, and often not applicable due to mismatched intents (e.g. regarding backwards compatibility). Also consider that isolating commits in separate branches is not feasible anymore as merge conflicts arise (e.g. due to code cleanups and interfering changes).

Contrast this to a fork, which allows us to keep focused and effect improvements quickly, so as to invest developer time rationally.

However, we would be glad if our work could eventually be merged back upstream once the change set has matured, if upstream can arrange themselves with the radical changes.
See https://github.com/ctypesgen/ctypesgen/issues/195 for discussion.


### Syncing with upstream

- First, sync the fork's master branch using GitHub's web interface.
- View changes on [GitHub's compare page](https://github.com/pypdfium2-team/ctypesgen/compare/pypdfium2...master).
- Pull and merge locally, then push the result.

Last time we had to do this, `git merge origin/master -Xours` did a good job.
`-Xignore-all-space` may also be of interest.

For those parts of ctypesgen that we have barely modified (e.g. the parser core), it should be largely possible to pull in changes as-is.
Otherwise, you'll have to manually look through the changes and pick what you consider worthwhile on a case by case basis.

Note, it is important to verify the resulting merge commit for correctness - automatic merge strategies might produce mistakes!

Copyright (c) 2007-2022, Ctypesgen Developers
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice,
   this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
   notice, this list of conditions and the following disclaimer in the
   documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

# Changelog


## Unreleased (v2)

_**NOTE:** Unfortunately, keeping track of differences to the old upstream is difficult and time-taking with a constantly evolving codebase. So, this changelog may be outdated/incomplete. Please check `ctypesgen --help` for current usage info._

- Large-scale rewrite of ctypesgen by `@mara004`, aiming to unbloat, simplify and modernize the codebase.
- Many issues fixed. See https://github.com/pypdfium2-team/ctypesgen/issues/1 and https://github.com/ctypesgen/ctypesgen/issues/195 and below for an overview.

### Usage differences (selection)

* CLI
  - Headers have been converted from positional to flag `-i`/`--headers`, to avoid confusion with options that take a variadic number of params.
  - Beware: Historically, `--include` did something different and is now called `--system-headers` here.
  - `--symbol-rules` replaces `--include-symbols` (yes) / `--exclude-symbols` (never).
  - `--no-embed-preamble` renamed to `--no-embed-templates`.
  - `--allow-gnu-c` replaced by `-X __GNUC__`.
  - More flags changed or renamed.
* Rewrote library loader. Avoid implicit searches. Take full paths with formatting options, rather than directories. Use standard `ctypes.util.find_library()` means to look for system libraries, rather than bloated custom code. Resolve `.` to the module directory, not the caller's CWD. Don't add compile libraries to runtime. Handle iOS (PEP 730).
* The bloated string wrappers have been removed. By default, no implicit string encoding/decoding is being done anymore, because `char*` is not necessarily a UTF-8 string or even NUL-terminated. However, the `--string-template` option allows to plug in your own string helpers (e.g. `c_char_p`, or a custom wrapper). However, we recommend that new code assume the raw `POINTER(c_char)` and cast/decode on the caller side as necessary.
* We declare `c_void_p` as restype directly, which ctypes auto-converts to int/None. Previously, ctypesgen would use `POINTER(c_ubyte)` and cast to `c_void_p` via errcheck to bypass the auto-conversion. However, a `c_void_p` programatically is just that: an integer or null pointer, so the behavior of ctypes seems fine. Note that we can seamlessly `ctypes.cast()` an int to a pointer type. The API difference is that there is no `.value` property anymore. Instead, the object itself is the value, removing a layer of indirection.

### New features and improvements (selection)

* Implemented relative imports with `--link-modules`, and library handle sharing with `--no-embed-templates`. Removed incorrect `POINTER` override that breaks the type system.
* Prevent assignment of invalid struct fields by setting slots *in the class body*.
* Slimmed up template by removing many avoidable wrappers.
* Better control over symbol inclusion via `--symbol-rules` (exposes `if_needed` strategy, allows free order of actions).
* Symbol regex matching uses `fullmatch()` rather than `match()` (more explicit).
* Eagerly include direct members with `--system-headers`. This helps lower the need for `--all-headers` (which generally includes a lot more than necessary).
* Auto-detect default pre-processor.
* Handle FAMs (Flexible Array members) as zero-sized arrays. See https://github.com/ctypesgen/ctypesgen/issues/219.
* Tightened `UNCHECKED()` template to only remap pointer types, and pass through anything else as-is. This avoids erroneously changing `None` or custom non-pointer types to `c_void_p`.
* `-X`: Ability to override arbitrary pre-processor default flags added by ctypesgen.
* Pass through `-D/-U` in given order.


## Historical

### v1.1.1

- Fixed inconsistency in version output in released packages

### v1.1.0

This release has a number of bug fixes in addition to a few new features.
Following a complete transition to Python 3, with dropped Python 2 support,
major work was made towards code modernization and quality.

- The code is now Black formatted and Flake8 tested
- Greatly improved unittest framework
- Embedded PLY version updated to 3.11
- New option: `--no-embed-preamble` create separate files for preamble and
  loader instead of embedding in each output file
- New option: `--allow-gnu-c` do not undefine `__GNUC__`
- Fixed library loader search path on macOS
- Fixed rare bug, processing (legacy) header files with MacRoman encoding
  on macOS
- Added full support for floating and integer constants
- Added support for sized integer types on Windows
- Added support to handle `restrict` and `_Noreturn` keywords
- Added name formats to posix library loader
- Fixed mapping of 'short int' to c_short
- Git tags are now using `x.y.z` format

### v1.0.2

Many issues fixed. Parse gcc attributes more

Implements automatic calling convention selection based on gcc attributes for
stdcall/cdecl.

- Simplify and unify library loader for various platforms. Improve library path
  searches on Linux (parsed ld.so.conf includes now).
- First implementaion of #pragma pack
- First implemenation of #undef
- Adds several command line options:
  `-D` `--define`
  `-U` `--undefine`
  `--no-undefs`
  `-P` `--strip-prefix`
  `--debug-level`

### v1.0.1

Fix handling of function prototypes

Other minor improvments included.

### v1.0.0

Py2/Py3 support

Various development branches merged back

In addition to the various developments from the different branches, this
tag also represents a code state that:

- ties in with Travis CI to watch code developments
- improves testsuite, including moving all JSON tests to testsuite
- includes a decent Debian package build configuration
- automatically creates a man page to be included in the Debian package
